Multithreading in Python

Python threading helps streamline your design by enabling the simultaneous execution of many program components. Multithreading in Python allows two or more threads to perform independently with the same process resource. Read object-oriented programming in Python for a better understanding of concepts and techniques in coding. 

This article is for you if you have some Python knowledge and wish to use threads to speed up your program! You will learn the basics that cover threading and multithreading, along with examples of tested codes. 

What is a thread?

A thread is an independent execution sequence. This implies that two things will be happening simultaneously in your program. However, in the majority of Python 3 implementations, the many threads only appear to execute concurrently. Explore why you should learn Python and gain expertise for trending software jobs.

In concurrent programming, an execution unit is called a thread. A CPU can carry out multiple tasks related to a single process simultaneously, thanks to the method known as multithreading. These threads can share processing resources and run independently.

What is Multithreading in Python?

The goal of multithreading in Python programming is to run multiple tasks and function cells at the same time. Threads are lighter than processes, and multiple threads may execute individually while sharing their process resources. Multithreading is a well-known technique in which multiple threads in a process share their data space with the main thread, making information sharing and communication within threads easy and efficient.

The Purpose of Multithreading

You can divide an application into several smaller jobs and execute them all at once using multithreading. Your application’s speed, performance, and rendering can all be enhanced by using multithreading effectively.

Suggested Read: What is Multithreading in Java?

Multithreading in Python

Python has built-in multiprocessing and multithreading constructs. The main focus of this course will be on utilizing Python to create multithreaded applications. Python has two primary modules that are useful for managing threads:

• The thread module: 
• The threading module.

But there’s also something called a global interpreter lock (GIL) in Python. It doesn’t increase performance and might potentially make some multithreaded apps perform worse. Discover method overloading in Python, along with examples.

The Thread Module

The thread module is no longer supported in Python. It is only available in backward compatibility as ‘_thread’ and has been declared outdated as Python 3.

Syntax:

thread.start_new_thread(function_name, arguments)

Example

import time

import _thread

def thread_test(name, wait):

   i = 0

   while i <= 3:

      time.sleep(wait)

      print(“Running %sn” %name)

      i = i + 1

   print(“%s has finished execution” %name)

if __name__ == “__main__”:

    _thread.start_new_thread(thread_test, (“First Thread”, 1))

    _thread.start_new_thread(thread_test, (“Second Thread”, 2))

    _thread.start_new_thread(thread_test, (“Third Thread”, 3))

Code Explanation:

• The time and thread modules, which manage the starting and stopping of Python threads, are imported into these statements.
• In this case, the start_new_thread method will call the thread_test function that you defined. 
• The method prints the name of the thread that called it after four iterations of a while loop. 
• The thread has finished executing when the iteration is over and a message is printed.
• This is your program’s primary section. Here, you just pass the thread_test function as an input when using the start_new_thread method. 
• This will initiate the execution of the function you supply as an argument on a new thread. 

Keep in mind that you may substitute any other function you wish to run as a thread for this one (thread_test).

Related Articles: Goto statement in Python | Jump statement in Python

The Threading Module

The higher-level threading module is what you should use for programs that you plan to release. This is merely an educational overview of the thread module.

Structure of Threading Module

A list of some useful functions and definitions in this threading module

Getting Started with Multithreading in Python

First, let’s create the download.py Python module. All the functions required to retrieve and download the list of photos will be included in this file. Multithreading in Python is one of the frequently asked Python interview questions and answers. These features will be divided into three distinct functions:

• get_links
• download_link
• setup_download_dir

If there isn’t currently a download destination directory, one will be created using the third function, setup_download_dir.

Example

HTTP requests must include the authorization header containing the client ID to use Imgur’s API. The application dashboard on Imgur is where you can locate this client ID. The response will be encoded in JSON. To decode it, we can use the built-in JSON library in Python. It’s even easier to get the image because all you need to do is put it into a file and retrieve it via its URL.

import json

import logging

import os

from pathlib import Path

from urllib.request import urlopen, Request

logger = logging.getLogger(__name__)

types = {‘image/jpeg’, ‘image/png’}

def get_links(client_id):

    headers = {‘Authorization’: ‘Client-ID {}’.format(client_id)}

    req = Request(‘https://api.imgur.com/3/gallery/random/random/’, headers=headers, method=’GET’)

    with urlopen(req) as resp:

        data = json.loads(resp.read().decode(‘utf-8’))

    return [item[‘link’] for item in data[‘data’] if ‘type’ in item and item[‘type’] in types]

def download_link(directory, link):

    download_path = directory / os.path.basename(link)

    with urlopen(link) as image, download_path.open(‘wb’) as f:

        f.write(image.read())

    logger.info(‘Downloaded %s’, link)

def setup_download_dir():

    download_dir = Path(‘images’)

    if not download_dir.exists():

        download_dir.mkdir()

    return download_dir

The next step is to create a module that will use these functions to download each image individually. This will be called single.py. This will include the primary feature of our initial, crude Imgur picture downloader. The environment variable IMGUR_CLIENT_ID will be used by the module to retrieve the Imgur client ID. To build the download destination directory, it will call the setup_download_dir function. Ultimately, it will utilize the get_links function to retrieve a list of photos, filter out any GIF and album URLs, and then download and save each image to the disc using download_link. Explore how you can work with data in Python in this article. This is how single.py appears:

import logging

import os

from time import time

from download import setup_download_dir, get_links, download_link

logging.basicConfig(level=logging.INFO, format=’%(asctime)s – %(name)s – %(levelname)s – %(message)s’)

logger = logging.getLogger(__name__)

def main():

    ts = time()

    client_id = os.getenv(‘IMGUR_CLIENT_ID’)

    if not client_id:

        raise Exception(“Couldn’t find IMGUR_CLIENT_ID environment variable!”)

    download_dir = setup_download_dir()

    links = get_links(client_id)

    for link in links:

        download_link(download_dir, link)

    logging.info(‘Took %s seconds’, time() – ts)

if __name__ == ‘__main__’:

    main()

Parallelism and Concurrency in Python

One of the most well-known methods for achieving concurrency and parallelism in Python is threading. The operating system typically offers threading as a feature. Sharing the same memory space as processes, threads are lighter.

To replace single.py, we will create a new module for this multithreading in the Python example. With the addition of the main thread, this module will generate eight additional threads, for a total of nine threads. Since our machine has eight CPU cores, it was decided to run eight worker threads because it looked like a reasonable amount of threads to run concurrently—one worker thread per core. In actuality, a lot more consideration goes into selecting this figure, taking into account other elements like other programs and services that may be running on the same system. Here are some of the popular Python libraries used for data analysis and they’ll be useful for your in-depth understanding of Python programming. 

• Except for the addition of a new class called DownloadWorker, which is a descendant of the Python Thread class, this is nearly identical to the previous one. 
• The infinite looping run method has been overridden by this override. It calls self.queue.get() to attempt to fetch a URL from a thread-safe queue on each iteration. 
• Until a worker has an item in the queue to process, it blocks. The worker invokes the same download_link function from the previous script to download the picture to the images directory as soon as it pulls an item from the queue. 
• The worker notifies the queue that the download is complete when it is. Because the queue keeps track of the number of jobs that were enqueued, this is crucial. 

If workers failed to indicate when a job was finished, the call to ‘queue.join()’ would permanently stop the main thread.

import logging

import os

from queue import Queue

from threading import Thread

from time import time

from download import setup_download_dir, get_links, download_link

logging.basicConfig(level=logging.INFO, format=’%(asctime)s – %(name)s – %(levelname)s – %(message)s’)

logger = logging.getLogger(__name__)

class DownloadWorker(Thread):

    def __init__(self, queue):

        Thread.__init__(self)

        self.queue = queue

    def run(self):

        while True:

            directory, link = self.queue.get()

            try:

                download_link(directory, link)

            finally:

                self.queue.task_done(

def main():

    ts = time()

    client_id = os.getenv(‘IMGUR_CLIENT_ID’)

    if not client_id:

        raise Exception(“Couldn’t find IMGUR_CLIENT_ID environment variable!”)

    download_dir = setup_download_dir()

    links = get_links(client_id)

    queue = Queue()

    for x in range(8):

        worker = DownloadWorker(queue)

        worker.daemon = True

        worker.start()

    for link in links:

        logger.info(‘Queueing {}’.format(link))

        queue.put((download_dir, link))

    queue.join()

    logging.info(‘Took %s’, time() – ts)

if __name__ == ‘__main__’:

    main()

This sample script for Python multithreading takes 4.1 seconds to download on the same computer as before! Compared to the prior example, that is 4.7 times faster. Even though this is far faster, it’s important to note that the GIL meant that only one thread was operating at a time during this operation. This code is therefore parallel but concurrent. This task is IO-bound, which is why it is still faster. While downloading these photographs, the processor is hardly making a sweat, and most of the time is spent waiting on the network. We have a guide to help you with building a web application with the popular Python framework, Django.

For this reason, multithreading in Python can result in significant speed gains. When a thread is prepared to work, the processor can switch between them. It is possible to experience decreased performance while using the threading module in Python or any other interpreted language that has a GIL. Using the threading module will cause your code to execute more slowly if it is carrying out a CPU-bound activity like decompressing gzip files.

Recommended Read: Python Developer Salary for Freshers

Conclusion

The capacity of a processor to run many threads simultaneously is known as multithreading. This is accomplished with a basic CPU with a single core by often switching between threads. We sincerely hope that this post will help you in your understanding of multithreading in Python. With our Best Python training in Chennai at SLA Institute, you may gain practical experience and learn more.