AWS Beanstalk Configuration

A log shipper designed for files

Follow the steps below to send your observability data to Logit.io

Logs

Filebeat is an open source shipping agent that lets you ship logs from AWS Beanstalk to one or more destinations, including Logstash.

Install Integration

Please click on the Install Integration button to configure your stack for this source.

AWS Beanstalk

To efficiently transport logs to your stack, Filebeat stands out as a streamlined choice. A member of Elastic's Beats family, Filebeat excels at collecting diverse metrics and logs across environments. Its lightweight nature, encryption capabilities, and user-friendly configuration make it an optimal solution.

Specifically, if you're keen on applying filters to logs, Filebeat offers a straightforward approach. Installation involves minimal effort, and the tool efficiently trails specified files. This is particularly advantageous in an AWS Beanstalk environment, where various log files are generated. Opting for Filebeat ensures a smooth log shipping process and facilitates easy differentiation between log types in OpenSearch.

Installing Filebeat via an .ebextension

Rather than engaging in manual SSH access to your EC2 instance for Filebeat installation, Elastic Beanstalk streamlines the process by enabling automatic deployment of AWS services and additional software through its extension system, known as .ebextension.

To implement this, include a .ebextensions folder in the root directory of your application and add a filebeat.config file in YAML format.

Copy the configuration file below and overwrite the contents of your

files:
"/etc/filebeat/filebeat.yml":
  mode: "000755"
  owner: root
  group: root
  content: |
    filebeat.inputs:
    - input_type: filestream
      enabled: true
      paths:
      # REQUIRED: Change the path to your logs
      - /var/log/*.log
      fields:
        type: logfile
    output.logstash:
      hosts: ["@logstash.host:@logstash.sslPort"]
      loadbalance: true
      ssl.enabled: true
 
commands:
1_command:
  command: "curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-8.11.3-x86_64.rpm"
  cwd: /home/ec2-user
2_command:
  command: "rpm -ivh --replacepkgs filebeat-8.11.3-x86_64.rpm"
  cwd: /home/ec2-user
3_command:
  command: "/etc/init.d/filebeat start"

Validate your YAML

It's a good idea to run the configuration file through a YAML validator to rule out indentation errors, clean up extra characters, and check if your YAML file is valid. Yamllint.com (opens in a new tab) is a great choice.

Check Logit.io for your logs

Data should now have been sent to your Stack.

View My Data

If you don't see take a look at How to diagnose no data in Stack below for how to diagnose common issues.

How to diagnose no data in Stack

If you don't see data appearing in your stack after following this integration, take a look at the troubleshooting guide for steps to diagnose and resolve the problem or contact our support team and we'll be happy to assist.

AWS Beanstalk Logging Overview

AWS Elastic Beanstalk simplifies application deployment and management in the AWS Cloud. As a fully managed platform-as-a-service (PaaS) solution, it frees developers from infrastructure complexities, enabling a focus on code creation.

Deployment is effortless, with Elastic Beanstalk handling resource provisioning, capacity management, and automatic scaling based on demand. Supporting diverse programming languages, it offers flexibility for various development needs.

Managed environments cover web servers, application servers, and databases, making Elastic Beanstalk suitable for hosting a variety of applications. Seamlessly integrated with other AWS services, it interfaces with Amazon RDS, Amazon S3, and CloudWatch for comprehensive functionality.

Automatic updates ensure a secure and up-to-date environment, and customization options allow users to modify configurations and integrate with existing AWS resources. Multi-tier application management is simplified, supporting separate environments for distinct tiers.

Built-in monitoring tools provide insights into application performance, and auto-scaling adjusts instances based on load. In a DevOps-friendly environment, Elastic Beanstalk supports continuous integration and deployment (CI/CD), streamlining the software delivery pipeline.

If you need any further assistance with migrating your log data to ELK we're here to help you get started. Feel free to get in contact with our support team by sending us a message via live chat & we'll be happy to assist.

Avast AWS ECS Fargate Metrics