Python Logging Configuration Example

Ship logs from Python to Logit.io

Send Python logs to your Logit.io Stack.

This sample app was created and tested with Python 3.11.2 and pip 24.2 and python-logstash-async 3.3.0

Install

Create a new directory for your project and name it python_logs_example.

Open a Terminal window or Command Prompt and navigate into the new python_logs_example folder. Create a virtual environment for your project to manage dependencies, this is done by entering the following command.

python -m venv venv ^
venv\Scripts\activate

We now need to install the necessary logging and Logstash handler packages using pip:

python-logstash-async: Provides a handler for sending logs to Logstash

The main advantage of using an asynchronous handler is that it sends logs to Logstash without blocking the main application thread. We install it with the following command:

pip install python-logstash-async

Create two new files in the folder and call them app.py and config.py. There should now be two files (and a folder called venv) in the python_logs_example folder.

Configuring the App

Copy and Paste the code below into config.py:

⚠️

No Beats/Beats-SSL input available! Your stack is missing the required input for this data source.

Talk to support to add the input

Copy and Paste the code below into app.py:

import logging
from logstash_async.handler import AsynchronousLogstashHandler
from logstash_async.handler import LogstashFormatter
 
# Importing configuration variables from config.py
from config import HOST, PORT, SSL_ENABLE
 
def main():
  try:
    # Create the logger and set it's logging level
    logger = logging.getLogger("logstash")
    logger.setLevel(logging.DEBUG)   
    
    handler = AsynchronousLogstashHandler(
      host=HOST, 
      port=PORT, 
      ssl_enable=SSL_ENABLE, 
      ssl_verify=False,
      transport='logstash_async.transport.BeatsTransport',
      database_path='')
    
    # Here you can specify additional formatting on your log record/message
    formatter = LogstashFormatter()
    handler.setFormatter(formatter)
    
    # Assign handler to the logger
    logger.addHandler(handler)
 
    print("Sending logs to Logit.io.")
    
    # Send log records to Logstash 
    logger.error('python-logstash-async: test error message.')
    logger.info('python-logstash-async: test info message.')
    logger.warning('python-logstash-async: test warning message.')
    logger.debug('python-logstash-async: test debug message.')
    
    # Flush and close the handler to ensure all logs are sent
    # If the application crashes before buffered logs are flushed, 
    # some logs could be lost unless explicitly handled through mechanisms 
    # like flush intervals or proper shutdown processes
    handler.flush()  # Flush the log queue
    handler.close()  # Wait for background thread to finish
 
  except Exception as e:
    print(f"Failed to send log to Logit.io: {e}")
  
if __name__ == '__main__':
    main()

Run the Python App

Run the Python app with the following command in Terminal or the Command Prompt window.

python app.py

You will see feedback from the app so that you know that is running.

Once the app has finished running, logs will have been sent to your Stack. The app sends an example of an error log, an info log, a warning log and a debug log.

Launch Logs to view your data

Launch Logs

How to diagnose no data in Stack

If you don't see data appearing in your stack after following this integration, take a look at the troubleshooting guide for steps to diagnose and resolve the problem or contact our support team and we'll be happy to assist.

Node.js Ruby