Datadog: A Comprehensive Guide to Optimization and Monitoring

Introduction:

For IT and DevOps teams, Datadog is a flexible analytics and monitoring solution that allows event tracking and performance metric determination across a range of cloud and infrastructure services. Datadog supports Windows, Linux, and Mac OS and is compatible with major cloud service providers like AWS, Azure, Red Hat OpenShift, and Google Cloud. It may be implemented on-premises or as a SaaS.

What is Datadog?

Datadog is a monitoring and analytics tool for information technology (IT) and DevOps teams that can be used to determine performance metrics as well as event monitoring for infrastructure and cloud services. The software can monitor services such as servers, databases, and tools.

Datadog monitoring software is available for deployment on-premises or as a software as a service (SaaS). Datadog supports Windows, Linux, and Mac operating systems. Support for cloud service providers includes AWS, Microsoft Azure, Red Hat OpenShift, and Google Cloud Platform.

Datadog can also send users notifications of performance issues on any set metric, such as compute rates. Users are notified through means such as email, Slack, or PagerDuty.

Pricing Details: https://www.datadoghq.com/pricing/

Prerequisites

Datadog is providing a 14-day trial account in this account you will be getting the Datadog controller SAAS product once you register in the Datadog account https://www.datadoghq.com/.

• Launch the Datadog SAAS product and Install the Agent with HostOnce you have completed the sign-in you will able to see the below content you can skip this.
• Click on Next below screenshot you will able to see different kinds of hosts.
• Here you have to select your host, where it will display the agent details to run those commands on your host, and then start your agent.

$ DD_AGENT_MAJOR_VERSION=7 
DD_API_KEY=7e99f076612449c8d47e7bfc14a966 DD_SITE="us3.datadoghq.com" 
bash -c 
"$(curl -L https://s3.amazonaws.com/dd-agent/scripts/install_script.sh)"

#this command will verify depending upon your OS and the API key will differ in every new account.

$systemctl start datadog-agent

• Once the Agent is installed on the host Under the Infrastructure List we can collect all the added hosts.
•  

Server Monitoring

• Monitoring the performance of your servers in real-time is critical to keeping your applications up and running smoothly.
• From identifying overloaded hosts to making rightsizing decisions, it is important to have a deep understanding of the status of your servers and the role they are playing in your architecture.
• Datadog helps you gain visibility into your servers, no matter the complexity of the environment.

Click on the host > metrics

 

Enabling the Live process for the OS

• Datadog’s Live Processes gives you real-time visibility into the process running on your infrastructure.
• Use Live Processes to View all of your running processes in one place.
• Once the Datadog Agent is installed, enable Live Processes collection by editing the Agent main configuration file by setting the following parameter to true:

process_config: enabled: true

The enabled value is a string with the following options:

• “true”: Enable the Process Agent to collect processes and containers.
• “false” (default): Only collect containers if available.
• “disabled”: Don’t run the Process Agent at all.

Additionally, some configuration options may be set as environment variables.

 

Web Server Monitoring (Apache Integration with Datadog and Log monitoring)

The Apache check is packaged with the Datadog Agent. To start gathering your Apache metrics and logs, you need to:

• Install the Agent on your Apache servers.
• Install mod_status on your Apache servers and enable ExtendedStatus.
• Refer below link to add the mod_status to the Apache configuration file.
  https://www.tecmint.com/monitor-apache-web-server-load-and-page-statistics/

To configure the check for an Agent running on a host:

Metric collection:

• Edit the apache.d/conf.yaml file in the conf.d/ folder at the root of your Agent’s configuration directory to start collecting your Apache metrics. See the sample apache.d/conf.yaml for all available configuration options.

init_config: instances:   ## @param apache_status_url - string - required   ## Status url of your Apache server.   #   - apache_status_url: http://localhost/server-status?auto

• On the Datadog UI go to the Integration list search for Apache and install the integration
• Restart the Agent.
• You have to be able to get the Apache metrics on the Datadog UI once the above step is successful.

 

Enabling Log collection for Apache

Available for Agent versions >6.0

• Collecting logs is disabled by default in the Datadog Agent. Enable it in datadog.yaml: logs_enabled: true
• Add this configuration block to your apache.d/conf.yaml file to start collecting your Apache logs, and adjusting the path and service values to configure them for your environment:

logs: - type: file path: /path/to/your/apache/access.log source: apache service: apache sourcecategory: http_web_access - type: file path: /path/to/your/apache/error.log source: apache service: apache sourcecategory: http_web_error

• Restart the Agent.
• You will get the logs on the Datadog UI once the above step is successful as shown in the below screenshot.
•  

Datadog APM

Datadog APM enables you to monitor service dependencies and health metrics, reduce latency, and eliminate errors so that your users get the best possible experience. Datadog supports monitoring APM levels for Java, Python, Go, Ruby, Node.js, (beta) PHP, (beta). NET.

 

Datadog APM for Java application running with Tomcat

Step 1 – Install Datadog Agent in your host

Step 2 – Install Java Application

$ sudo apt-get install openjdk-8-jdk

Step 3 – Install and Run Apache Tomcat

$ sudo apt install wget unzip -y $ cd /opt $ wget https://mirrors.estointernet.in/apache/tomcat/tomcat-8/v8.5.65/bin/apache-tomcat-8.5.65.zip $ unzip apache-tomcat-8.5.65.zip

Step 4 – Download dd-java-agent.jar into /opt/ and give permission.

$ cd /opt $ wget -O dd-java-agent.jar 'https://dtdg.co/latest-java-tracer' $ chmod 777 dd-java-agent.jar

Step 5 – Edit catalina.sh and the following entry

$ vi /opt/apache-tomcat-8.5.61/bin/catalina.sh # Make sure dd.service and dd.env as per your choice. CATALINA_OPTS="$CATALINA_OPTS -javaagent:/opt/dd-java-agent.jar -Ddd.profiling.enabled=true -Ddd.logs.injection=true -Ddd.trace.sample.rate=1 -Ddd.service=apm-tomcat -Ddd.env=test"

Step 6 – Start Tomcat and Datadog Agent

$ systemctl restart datadog-agent $ cd /opt/apache-tomcat-8.5.65/bin $ chmod -R 775 . $ ./startup.sh $ ps -eaf | grep tomcat | grep dd

 

Review the Result :

Reporting services appear on the APM services page.

• The services list is a high-level view of all services reporting from your infrastructure.
• Select an individual service to view detailed performance insights.

Datadog offers complete Application Performance Monitoring (APM) and distributed tracing for organizations operating at any scale.

• You can use Datadog’s auto-instrumentation libraries to collect performance data or integrate Datadog with open-source instrumentation and tracing tools.

 

Datadog Synthetic API :

Datadog Synthetic API tests and browser tests help you proactively identify issues in application endpoints and key business workflows (e.g., signing up for a new account) before your end users encounter them.

Synthetic API tests generate active requests to your web properties or application endpoints at periodic, configurable intervals. Synthetic Monitoring can continuously simulate several different types of traffic:HTTP: verifies that applications are up and respond to requests.

• DNS: monitors the performance and resolvability of DNS nameservers.
• TCP: ensures the availability of critical service ports (e.g., SSH (22), DNS (53), VPN over HTTPS (443)).
• SSL: validates SSL/TLS certificates and tracks their expiration dates.
• ICMP: pings endpoints to detect connectivity or latency issues, unexpected changes to security firewall settings, and the number of network hops to a host

To add the Endpoint URL to monitor follow the below steps:

UX Monitoring > Synthetic Test > New API Test > Provide URL name and select particular protocol > Test URL once after the test check is passed > Select the region >Mention the required details such as frequency and Enter the message > whom should notify > Create

 

To add the Endpoint URL to monitor follow the below steps :

UX Monitoring > Synthetic Test > New API Test > Provide URL name and select particular protocol > Test URL once after the test check is passed > Select the region >Mention the required details such as frequency and Enter the message > whom should notify > Create

How you will see the uptime in every region?

 

Dashboard Creation

A dashboard is Datadog’s tool for visually tracking, analyzing, and displaying key performance metrics, which enables you to monitor the health of your infrastructure.

• To create a dashboard, click +New Dashboard on the Dashboard List page or New Dashboard from the navigation menu. Enter a dashboard name and choose a layout option.
• Select the widget.
• Select the Metrics and Host for the creation of the Dashboard and click Create.

 

Alerting

Monitoring all of your infrastructure in one place wouldn’t be complete without the ability to know when critical changes are occurring. Datadog gives you the ability to create monitors that actively check metrics, integration availability, network endpoints, and more. Configure monitors, notify your teams, and manage alerts at a glance on the Alerting platform.

Manage Monitors

• Edit, clone, delete, mute, and resolve monitors all in the same place. Focus on high-priority alerts by using advanced faceted search.
• Explore monitor details and alerts over time on the Monitor Status page.
• Alert condition whether the Status is critical or normal can be seen from the UI as well and if the condition is critical it will trigger the alert based on the threshold.

 

Add new members and manage invites

To add members to your organization:

• Go to the Organization Settings page, then click the Users tab.
• Click Invite Users in the upper right corner of the page.
• Enter the email address of the user you wish to invite to your Datadog account.
• Assign one or more user roles to the users.
  (Note: Users with the Invite User permission can invite a user to any role they have themselves. Users with both the Invite User and Access Management permissions can invite a user to any role.)
• Click Send Invites.

 

Un-install/remove Datadog Agent from the host

• sudo yum remove Datadog-agent

This command removes the Agent, but does not remove:

• The datadog.yaml configuration file.
• user-created files in the /etc/dd-agent configuration folder.
• user-created files in the /opt/datadog-agent folder.
• the dd-agent user.

#sudo userdel dd-agent \ && sudo rm -rf /opt/datadog-agent/ \ && sudo rm -rf /etc/dd-agent/ \ && sudo rm -rf /var/log/datadog/

 

Datadog Monitoring EKS Cluster

Prerequisite:

Install the Helm and AWS CLI on your OS to connect the EKS. On the UI > Integration > Kubernetes > Install Integration.

• Install using helm
  helm repo add datadog https://helm.datadoghq.com helm repo update.
• Helm v3+

$ helm install datadog --set datadog.site='http://datadoghq.com ' --set datadog.apiKey=your_api_key --set datadog.apm.enabled=true --set datadog.log.enabled=true datadog/datadog $ helm repo update 

Enable Cluster Log :

• git clone https://github.com/DataDog/helm-charts/blob/main/charts/datadog/values.yaml update the yaml file as below.

datadog: ## @param logs - object - required ## Enable logs agent and provide custom configs # logs: ## @param enabled - boolean - optional - default: false ## Enables this to activate Datadog Agent log collection. # enabled: true ## @param containerCollectAll - boolean - optional - default: false ## Enable this to allow log collection for all containers. # containerCollectAll: true

$ helm install pod-name -f values.yaml --set datadog.site='datadoghq.com' --set datadog.apiKey='your_api_key' datadog/datadog

Reference: https://docs.datadoghq.com/containers/kubernetes/

 

Conclusion:

With Datadog, you may go on an enlightening journey from installation to advanced functionality. Examine configurable dashboards, live tracking of metrics, and notification systems. You can monitor, diagnose, and optimize your infrastructure with Datadog, which offers services like APM, synthetic API testing, and server monitoring.

Datadog can track Apache metrics and orchestrate APM for Java applications, among many other things. Develop a strong monitoring strategy that is specific to the requirements of your company to get the most out of your Datadog experience. Cheers to your monitoring!