Python Logging: How to Use Logging in Python

Python Logging Tutorial With Example | Logging in Python

Python logging is an inbuilt module that defines functions and classes which implement the flexible event logging system for applications and libraries. The key benefit of having a logging API provided by a standard library module is that all Python modules can participate in the logging, so your application log can include your messages integrated with messages from third-party modules.

Python Logging

Although logging is essential, not all developers know how to use them correctly. I saw some developers insert the print statements when developing and remove those statements when it is finished.

It works when a program is a simple script, but for complex systems, you better not to use this print approach.

First of all, you cannot leave only essential messages in the log, you may see the lots of garbage messages in the log, but can’t find anything fruitful.

You also cannot control those print statements without modifying the code, and you may forget to remove those unused prints and all the printed messages go into stdout, which is terrible when you have the data to output to the stdout.

Of course, you can print the messages to stderr, but still, it is not a good practice to use print for logging.

Logging is a handy tool in the programmer’s toolbox. It can help you develop the flow of the program and discover scenarios that you might not even have thought of while developing an app.

Logs provide developers with the extra set of eyes that are always looking at the flow that the application is going through.

They can store the information, like which user or IP accessed an app.

If the error occurs, then they can provide more insights than the stack trace by telling you what the state of a program was before it arrived at the line of code where an error occurred.

By logging the useful data from the right places, you can not only debug the errors easily but also use the data to analyze the performance of an application to plan for scaling or look at the usage patterns to plan for marketing.

#Logging Module in Python

The logging module in Python is the ready-to-use and powerful module that is designed to meet the needs of the beginners as well as the enterprise teams.

You can import the Logging module using the following line of code.

import logging

After the logging module imported, you can use something called a “logger” to log the messages that you want to see. By default, there is a total of five standard levels which are indicating the severity of events.

Each has the corresponding method that can be used to log the events at that level of seriousness. The defined levels are the following:

  1. DEBUG
  2. INFO
  4. ERROR

The logging module provides you with the default logger that allows you to get started without needing to do many configurations.


import logging

logging.debug('This is a debug message')'This is an info message')
logging.warning('This is a warning message')
logging.error('This is an error message')
logging.critical('This is a critical message')

Now run the above file in the terminal, and you can see the following output.

Python Logging Tutorial With Example

One thing from the above output you can note that we only got the three output and not five.

The above output shows that the severity level before each message along with a root, which is the name, the logging module gives to its default logger. (Loggers are discussed in detail in the later sections.)

In this format, which shows the level, name, and message separated by the colon (:), is a default output format that can be configured to include things like timestamp, line number, and the other details.

Notice that the debug() and info() messages didn’t get logged. It is because, by default, the logging module logs the messages with the severity level of WARNING or above.

You can change that by configuring a logging module to log the events of all levels if you want. You can also define your severity levels by changing configurations, but it is generally not recommended as it can confuse with logs of some third-party libraries that you might be using.

#Logging Basic Configurations

You can use the basicConfig(args) method to configure the logging.

Commonly used parameters for the basicConfig() are the following.

  • level: The root logger will be set to a specified severity level.
  • filename: This specifies the file.
  • filemode: If the filename is given, the file is opened in this mode. The default is a, which means append.
  • form: This is the format of the log message.

Now, see the following example of Python Logging.


import logging
logger = logging.getLogger(__name__)'Start reading database')

# read database here
records = {'krunal': 26, 'ankit': 24}
logger.debug('Records: %s', records)'Updating records ...')
records = {'krunal': 27, 'ankit': 25}

# update records here'Finish updating records')

Now, run the above file and see the output.

Logging Basic Configurations

So, By using a level parameter, you can set what level of log messages you want to record.

It can be done by passing one of the constants available in the class, and this would enable all the logging calls at or above that level to be logged.

The debug(), info(), warning(), error(), and critical() also call the basicConfig() without arguments automatically if it has not been called before. What this means that after the first time one of the above functions is called, you can no longer configure a root logger because they would have called a basicConfig() function internally.

You can use the FileHandler to write records to a file. See the following code example.


import logging

logger = logging.getLogger(__name__)

# create a file handler
handler = logging.FileHandler('info.log')

# create a logging format
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')

# add the handlers to the logger
logger.addHandler(handler)'See the info.log file')

Now, run the file and one file is created in the same directory as your file called info.log and now see the output of the file. The output is the following.

2019-06-07 23:02:03,617 - __main__ - INFO - See the info.log file

So, you can create log files in your file system to track the changes in your applications.

Most web frameworks and CMS follow this approach so that when the app will crash, they will see this kind of log file to figure out what went wrong.

#Formatting the Logging Output in Python

Format string in Python is effortless. Same as the Logging module. You can pass the variable that can be represented as the string from your program as the message to your logs, and some essential elements are already a part of the LogRecord and can be easily added to an output format.

If you want to log a process ID along with the level and message. See the following code example.


import logging

logging.warning('This statement is a Warning')

See the below output.

Formatting the Logging Output in Python

Here is another example where you can add the current date and time info.

import logging

logging.basicConfig(format='%(asctime)s - %(message)s', level=logging.INFO)'Krunal has logged in')

See the output.

Logging Format in Python

The %(asctime) adds the time of the creation of the LogRecord.

The format can be changed using a datefmt attribute, which uses the same formatting language as the formatting functions in the Python datetime module, such as the time.strftime().

#Logging Variable Data

The logging methods take the string as an argument, and it might seem natural to format the string with variable data in a separate line and pass it to the log method.

But this can be done directly by using a format string for the message and appending the variable data as arguments. See the following example.


import logging

name = 'Krunal'

logging.warning('%s raised a warning', name)

See the following output.

Logging Variable Data

The arguments passed to a method would be included as the variable data in the message.

While you can use any formatting style you want, the f-strings introduced in Python 3.6 are an excellent way to format the strings as they can help keep the formatting short and easy to read.


import logging

name = 'Krunal'

logging.warning(f'{name} raised an warning')

logging methods in Python

#Classes and Functions

You can (and should) define your logger by creating an object of a Logger class, especially if your application has multiple modules.

Let’s have the look at some of the classes and functions in the Python module.

The most common classes defined in the logging module are the following.

  • Logger: This is the class whose objects will be used in the application code directly to call the functions.

  • LogRecord: Loggers automatically create the LogRecord objects that have all the information related to an event being logged, like the name of a logger, the function and the line number, the message, and more.
  • Handler: Handlers send a LogRecord to the required output destination, like the console or the file. The handler is the base for subclasses like the StreamHandler, FileHandler, SMTPHandler, HTTPHandler, and more. These subclasses send the logging outputs to the corresponding destinations, like sys.stdout or the disk file.
  • Formatter: This is where you specify the format of the output by specifying the string format that lists out the attributes that the output should contain.

We mostly deal with the objects of a Logger class, which are instantiated using a module-level function logging.getLogger().

Multiple calls to getLogger() with the same name will return a reference to the same Logger object, which saves us from passing the logger objects to every part where it’s needed.

See the following example.


import logging

logger = logging.getLogger('demo logger')
logger.error('This is an error')

See the output.

Logging in Python example

So, in this tutorial, we have discussed the Python logging module in, Logging basic configurations, classes, and functions, Python logging variable and data, and formatting the Python logging output.

Finally, Python Logging Example Tutorial is over.

Recommended Posts

Python Time sleep()

Python subprocess

Python OS Module

Python SimpleHTTPServer

Python Enumerate

Leave A Reply

Please enter your comment!
Please enter your name here

This site uses Akismet to reduce spam. Learn how your comment data is processed.