Log Management Overview
What is a Log Management Stack?
In the context of Log Management on Logit.io, a stack refers to a collection or grouping of related logs or log entries. Logs are records of events, activities, or messages generated by software applications, servers, and other systems. They are crucial for troubleshooting, monitoring, and analyzing the behavior of these systems.
The Ingestion Pipeline
The ingestion pipeline is responsible for collecting log data from various sources, processing it, and then forwarding it to the storage component. This stage often involves log collectors or agents installed on servers or systems to capture logs, as well as parsing and transformation processes to make the data more structured and usable. Here, you can utilize services such as managed Logstash and view individual instance details and their health.
Storage of Log Data
Once the log data is ingested, it needs to be stored efficiently for future analysis and retrieval. Depending on the scale and requirements of the organization, various storage solutions can be used. You can utilize OpenSearch for powerful searching and storage capabilities, as well as view individual nodes and health details.
Analysis and Visualization
After storing log data, it's essential to be able to analyze and visualize it effectively. Visualization tools provide dashboards, graphs, and charts that make it easier for administrators, DevOps teams, and other stakeholders to monitor system health, detect anomalies, and troubleshoot issues. Here, you can harness the potential of OpenSearch Dashboards to produce detailed visualizations and view node and version details.
Working with Your Stack in Log Management
Locating Your Stack
Finding your stack in Log Management is simple. From the initial dashboard, it should be the first section you see, below your account title and details. Here, you can view your entire stack. There's a significant amount of information in this section, so we have broken it down for you.
Stack Plan
To start, just beneath the section titled 'Log Management', you will encounter the 'Stack Plan.' Within this section, you will find comprehensive information about your stack, which includes details about your subscription, such as the monthly log volume and data retention.
Logstash
Below 'Stack Plan' is the heading 'Services'. The first of these services is 'Logstash'. Here you can utilize a variety of Logstash endpoints. Additionally, you can select 'View All Inputs' to view the full list of your inputs and configure new inputs. You can also view your Logstash IP addresses.
OpenSearch
The next of these 'Services' is 'OpenSearch.' From here you can copy your account details such as your OpenSearch endpoint, password, and username. Finally, if you select 'View Full Details' you will reach a page that allows you to observe the current OpenSearch version running on your nodes, toggle between different authentication modes, and gain access to OpenSearch's API endpoints.
OpenSearch Dashboards
The final service listed is 'OpenSearch Dashboards'. From here you can copy the endpoint, as well as select 'Launch' which will take you to your OpenSearch Dashboards from where you can analyze your log data and build interactive dashboards.
View Data Integrations
To the right of the 'Log Management' section you will notice 'View Data Integrations.' If you click this link, it will take you to our source integration tool, where you'll find a range of sources with instructions to effectively configure them.
View Statistics
Next to 'View Data Integrations' you will find 'View Statistics.' On this page, you have the ability to check data utilization statistics for transmitting data to your Logs Stack on a weekly basis, as well as review historical usage data for a period of up to one year. The usage table located towards the end of this page provides a more detailed exploration of these trends. It presents a daily breakdown of sent logs, their corresponding size in gigabytes, and a comparison against your current plan's data and retention limits.
Manage Stack in Log Management
The final option available to you in this section is 'Manage Stack'. If you select this, you will view a page where you can see your Logs Stack version information, as well as rebuild or delete your Stack.