Puppet Metrics

Ship Puppet Server Metrics using Metricbeat to Logstash

Metricbeat is a lightweight shipper that helps you monitor Puppet performance metrics by collecting metrics running on the Puppet Agents. Configure Metricbeat using the pre-defined examples below to collect and ship Puppet metrics to Logstash or Elasticsearch.

Install Integration

Please click on the Install Integration button to configure your stack for this source.

Requirements

In order for puppet metrics to be shipped correctly you have to have the following server pre-requisites.

Puppet Server 5.0 or above, tested on version 6
Jolokia agent installed
Tested on Ubuntu server

Install Jolokia Agent

Download Tomcat 7 (opens in a new tab) and then extract it.

tar zxvf apache-tomcat-7.*.tar.gz 
cd apache-tomcat-7.*
TC=`pwd`

Download the Jolokia WAR-agent and copy it into Tomcat's webapp directory.

cd $TC/webapps
wget https://repo1.maven.org/maven2/org/jolokia/jolokia-war/1.2.3/jolokia-war-1.2.3.war
mv jolokia-war-1.2.3.war jolokia.war

Verify Jolokia Installation

Now the container can be started with.

$TC/bin/catalina.sh start

In order to verify the installation was successful, point your browser to:

http://localhost:8080/jolokia/version (opens in a new tab)

Your response should have status 200 in the json data.

Configure Jolokia

To change the security access policy for Jolokia.

Create the /etc/puppetlabs/puppetserver/jolokia-access.xml file with contents below that follow the Jolokia access policy.

/etc/puppetlabs/puppetserver/jolokia-access.xml

<?xml version="1.0" encoding="utf-8"?>
<restrict>
  <remote>
    <host>${PUPPET-HOST-SERVER}</host>
  </remote>
  <http>
    <method>post</method>
  </http>
  <commands>
    <command>read</command>
    <command>list</command>
  </commands>
  <allow>
    <mbean>
      <name>java.lang:type=Memory</name>
      <operation>gc</operation>
    </mbean>
  </allow>
  <deny>
    <mbean>
      <name>com.mchange.v2.c3p0:type=PooledDataSource,*</name>
      <attribute>properties</attribute>
    </mbean>
  </deny>
</restrict>

Uncomment the metrics.metrics-webservice.jolokia.servlet-init-params.policyLocation parameter and then restart the puppet server.

Install Metricbeat

curl -L -O https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-oss-7.6.2-amd64.deb
sudo dpkg -i metricbeat-oss-7.6.2-amd64.deb

Locate Configuration File

/etc/metricbeat/metricbeat.yml

Configure Output

We'll be shipping to your hosted Logstash so that we have the option to apply filters before the data is indexed into Elasticsearch.

Comment out the elasticsearch output block:

## Comment out elasticsearch output
#output.elasticsearch:
#  hosts: ["localhost:9200"]

⚠️

No input available! Your stack is missing the required input for this data source.

Talk to support to add the input

Enable Module

There are several built-in metricbeat modules you can use. To enable the jolokia module, run the following:

metricbeat modules list
metricbeat modules enable @module

Configure Module

Each module has its own configuration file where different metricsets can be enabled / disabled. Locate the configuration file for the jolokia module.

/etc/metricbeat/modules.d/@module.yml

By default metricsets are disabled. To enable or disable a metric simply comment or uncomment the line out.

- module: jolokia
  metricsets: ["jmx"]
  enabled: true
  period: 10s
  hosts: ["localhost:8080"]
  namespace: "jolokia_metrics"
  path: "/jolokia"
  http_method: 'GET'
  jmx.mappings:
    - mbean: 'java.lang:type=Memory'
      attributes:
        - attr: HeapMemoryUsage
          field: memory.heap_usage
        - attr: NonHeapMemoryUsage
          field: memory.non_heap_usage
      - mbean: 'Catalina:name=*,type=ThreadPool'
      attributes:
        - attr: port
          field: catalina.port
        - attr: maxConnections
          field: catalina.maxConnections
      - mbean: 'java.lang:type=Runtime'
      attributes:
        - attr: Uptime
          field: uptime

Start Metricbeat

Start or restart Metricbeat (opens in a new tab) to apply the configuration changes.

How to diagnose no data in Stack

If you don't see data appearing in your stack after following this integration, take a look at the troubleshooting guide for steps to diagnose and resolve the problem or contact our support team and we'll be happy to assist.

Puppet Overview

Puppet is a leading configuration management (CM) system that makes the process of automating common tasks for DevOps & Sysadmins manageable. The platform can also be used to eliminate the need for manual configuration in order to deploy servers automatically.

This IT automation software can manage many of the common operations for maintaining system infrastructure including patching, provisioning, and management of operating systems & servers across on-premise and cloud environments.

Puppet are technology partners with many of the biggest leaders in the cloud computing space, including Microsoft Azure & Amazon Web Services. They work with over 30,000 companies globally helping them to secure their infrastructure, including Red Hat, Cisco, & Salesforce.

Puppet has been deployed across especially large infrastructures managing in excess of 5000+ machines. As Puppet is built for dynamically scaling and centralising machine activity, their platform works hand in hand with Logit.io to scale your infrastructure and centralise your data. Logit.io provides you with a scalable alternative for inspecting Puppet metrics data fast. Logit.io also enables you to visualise key performance metrics for Puppet using our Hosted Kibana (opens in a new tab) integration.

Puppet metrics include key insights on the number of changes the application has made, resources managed, and the type of events that have occurred. Each of the metric’s log messages has a name, label, and values that make it easy to parse when using Logstash to transform your files.

If you need any further assistance with migrating your Puppet metric logs to Logit.io we're here to help you get started. Feel free to get in contact with our support team by sending us a message via live chat and we'll be happy to assist.

PostgreSQL Metrics Azure