Puppet Metrics
Ship Puppet Server Metrics using Metricbeat to Logstash
Metricbeat is a lightweight shipper that helps you monitor Puppet performance metrics by collecting metrics running on the Puppet Agents. Configure Metricbeat using the pre-defined examples below to collect and ship Puppet metrics to Logstash or Elasticsearch.
Install Integration
Requirements
In order for puppet metrics to be shipped correctly you have to have the following server pre-requisites.
- Puppet Server 5.0 or above, tested on version 6
- Jolokia agent installed
- Tested on Ubuntu server
Install Jolokia Agent
Download Tomcat 7 (opens in a new tab) and then extract it.
tar zxvf apache-tomcat-7.*.tar.gz
cd apache-tomcat-7.*
TC=`pwd`
Download the Jolokia WAR-agent and copy it into Tomcat's webapp directory.
cd $TC/webapps
wget https://repo1.maven.org/maven2/org/jolokia/jolokia-war/1.2.3/jolokia-war-1.2.3.war
mv jolokia-war-1.2.3.war jolokia.war
Verify Jolokia Installation
Now the container can be started with.
$TC/bin/catalina.sh start
In order to verify the installation was successful, point your browser to:
http://localhost:8080/jolokia/version (opens in a new tab)
Your response should have status 200 in the json data.
Configure Jolokia
To change the security access policy for Jolokia.
Create the /etc/puppetlabs/puppetserver/jolokia-access.xml
file with contents below
that follow the Jolokia access policy.
<?xml version="1.0" encoding="utf-8"?>
<restrict>
<remote>
<host>${PUPPET-HOST-SERVER}</host>
</remote>
<http>
<method>post</method>
</http>
<commands>
<command>read</command>
<command>list</command>
</commands>
<allow>
<mbean>
<name>java.lang:type=Memory</name>
<operation>gc</operation>
</mbean>
</allow>
<deny>
<mbean>
<name>com.mchange.v2.c3p0:type=PooledDataSource,*</name>
<attribute>properties</attribute>
</mbean>
</deny>
</restrict>
Uncomment the metrics.metrics-webservice.jolokia.servlet-init-params.policyLocation
parameter and then restart the puppet server.
Install Metricbeat
curl -L -O https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-oss-7.6.2-amd64.deb
sudo dpkg -i metricbeat-oss-7.6.2-amd64.deb
Locate Configuration File
/etc/metricbeat/metricbeat.yml
Configure Output
We'll be shipping to your hosted Logstash so that we have the option to apply filters before the data is indexed into Elasticsearch.
Comment out the elasticsearch output block:
## Comment out elasticsearch output
#output.elasticsearch:
# hosts: ["localhost:9200"]
No input available! Your stack is missing the required input for this data source.
Talk to support to add the inputEnable Module
There are several built-in metricbeat modules you can use. To enable the jolokia module, run the following:
metricbeat modules list
metricbeat modules enable @module
Configure Module
Each module has its own configuration file where different metricsets can be enabled / disabled. Locate the configuration file for the jolokia module.
/etc/metricbeat/modules.d/@module.yml
By default metricsets are disabled. To enable or disable a metric simply comment or uncomment the line out.
- module: jolokia
metricsets: ["jmx"]
enabled: true
period: 10s
hosts: ["localhost:8080"]
namespace: "jolokia_metrics"
path: "/jolokia"
http_method: 'GET'
jmx.mappings:
- mbean: 'java.lang:type=Memory'
attributes:
- attr: HeapMemoryUsage
field: memory.heap_usage
- attr: NonHeapMemoryUsage
field: memory.non_heap_usage
- mbean: 'Catalina:name=*,type=ThreadPool'
attributes:
- attr: port
field: catalina.port
- attr: maxConnections
field: catalina.maxConnections
- mbean: 'java.lang:type=Runtime'
attributes:
- attr: Uptime
field: uptime
Start Metricbeat
Start or restart Metricbeat (opens in a new tab) to apply the configuration changes.
How to diagnose no data in Stack
If you don't see data appearing in your stack after following this integration, take a look at the troubleshooting guide for steps to diagnose and resolve the problem or contact our support team and we'll be happy to assist.
Puppet Overview
Puppet is a leading configuration management (CM) system that makes the process of automating common tasks for DevOps & Sysadmins manageable. The platform can also be used to eliminate the need for manual configuration in order to deploy servers automatically.
This IT automation software can manage many of the common operations for maintaining system infrastructure including patching, provisioning, and management of operating systems & servers across on-premise and cloud environments.
Puppet are technology partners with many of the biggest leaders in the cloud computing space, including Microsoft Azure & Amazon Web Services. They work with over 30,000 companies globally helping them to secure their infrastructure, including Red Hat, Cisco, & Salesforce.
Puppet has been deployed across especially large infrastructures managing in excess of 5000+ machines. As Puppet is built for dynamically scaling and centralising machine activity, their platform works hand in hand with Logit.io to scale your infrastructure and centralise your data. Logit.io provides you with a scalable alternative for inspecting Puppet metrics data fast. Logit.io also enables you to visualise key performance metrics for Puppet using our Hosted Kibana (opens in a new tab) integration.
Puppet metrics include key insights on the number of changes the application has made, resources managed, and the type of events that have occurred. Each of the metric’s log messages has a name, label, and values that make it easy to parse when using Logstash to transform your files.
If you need any further assistance with migrating your Puppet metric logs to Logit.io we're here to help you get started. Feel free to get in contact with our support team by sending us a message via live chat and we'll be happy to assist.