Amazon CloudWatch

Amazon CloudWatch

Ship logs from Amazon AWS to Logstash using the Logstash Cloudwatch plugin

Follow the steps below to send your observability data to Logit.io

Metrics

Install Integration

Please click on the Install Integration button to configure your stack for this source.

Create IAM Policy

In the top left corner of your aws console you will notice a services drop down arrow. Open it and from that menu choose IAM.

AWS Services

Next, in the left hand menu, select policies. Once you have reached the policies page you want to hit the Create Policy that appears towards the top of the page.

Go to policies page

Creating a policy is essentially defining permissions that you can then assign to an identity (user, group or roles.)

On the create policy screen choose the json tab and enter the following:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1444715676000",
      "Effect": "Allow",
      "Action": [
        "cloudwatch:GetMetricStatistics",
        "cloudwatch:ListMetrics"
      ],
      "Resource": "*"
    },
    {
      "Sid": "Stmt1444716576170",
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeInstances"
      ],
      "Resource": "*"
    }
  ]
}

Create policy editor

At the bottom of the page select review policy and finally on the next page you need to give your policy a name, now hit create policy.

Review new policy

Create IAM User

You are now going to need to create a new user so that you can attach the policy you created in step one, this will outline the responsibilities of this user.

While on the IAM page, in the left hand menu choose users.

Now at the top of the page select Add User.

View users and press Add user button

Enter a username and make sure to check Programmatic Access, continue onto the next page of creation.

Set user details in add user form

In the next section you want to attach an exisiting policy. Highlight attach exisiting policies and search the policy list below for your newly created policy.

Attach policies to the new user

Continue onwards to the next step, you can choose to set any Tags here but they aren't necessary. Continue onto the User Review, check all settings are correct and select create user.

On the next screen you will be given your Access Key ID and Secret Access Key. You will need to make a note of these or alternatively download the .csv file provided.

Success, and view your access key ID

Confirm EC2 Instance (Optional)

If you already have an EC2 Instance running you can skip this step.

The easiest way to for the new user to generate some data is to create an EC2 instance so that it can gather the metrics. Select the services drop down and look for EC2, it should come under the heading Compute. On the left hand menu select instances and choose to create a new instance and follow the steps provided.

Once you've created your instance you can move onto the next step.

Configure Logstash for Amazon CloudWatch

To start sending logs and metrics from AWS to your Stack you need to configure an AWS Input on your Logit.io Stack.

Go to Dashboard

Logit.io will verify your input before it is applied, we will contact you to confirm when this has been completed.

Check Logit.io for your metrics

Data should now have been sent to your Stack.

View My Data

If you don't see take a look at How to diagnose no data in Stack below for how to diagnose common issues.

CloudWatch Logging Overview

Sending data to Logit.io from Amazon CloudWatch offers a powerful way to centralize log data for enhanced insights and analysis. Organizations can seamlessly forward CloudWatch metrics to Logit.io, enabling real-time log analysis and visualization, improving operational efficiency, and strengthening security measures. For organizations using Amazon RDS, integrating these services can optimize database performance, while combining CloudTrail with Logit.io further enhances security and compliance efforts. Enjoy the efficient operation of these integrations within Logit.io's AWS logging (opens in a new tab).