Using Blocking and Non-Blocking Linux FIFOs in Python

FIFOs (Named pipes) is one of the Inter Process Communication (IPC) functionalities built into the Linux platform for one-way communication between processes. The FIFO appears as a file which can be accessed by normal file IO libraries for exchanging data.

FIFO allows running micro-services that communicate with each other rather than a single monolithic code, helping in maintaining the code. Running FIFO allows processes written in different languages to access each other.

Creating Pipes

mkfifo “pipe” command in bash creates a FIFO named “pipe”

mkfifo pipe

Lets test the FIFO we created, opening two terminals and entering the following commands

Terminal 1:

echo "Hello World!" > pipe

Terminal 2:

cat pipe

The blocking nature of the FIFO can be observed, the ‘data write’ to the pipe from Terminal 1 doesn’t complete until the data is read from Terminal 2. Now we are done with the bash commands lets see the implementation of FIFOs in Python.

Blocking FIFOs

A read call to a FIFO is blocked until a data is available in the pipe and a write call is blocked until the data is read from another process. The following python code demonstrates a blocking FIFO

#fifo.py
import os

bufferSize = 100;
PATH = "pipe";

pipe = os.open(PATH, os.O_RDONLY);
input = os.read(pipe,bufferSize);
print(input);
os.close(pipe);

Lets run this code in the background and write some data into the “pipe” FIFO file.

 sudo python fifo.py&;
echo "Hello World!" > pipe 

Note: Adding an ampersand (&) to the end of the command runs it in the background but still prints to the console.

The os.open() function blocks the process when data is unavailable rather than the os.read() function. Observe this by interrupting the code before writing data into the FIFO.

The blocking nature of FIFOs is useful only when the code is executed after a data is received. Scenarios where there are other functions that need to run continuously independent of the data received  requires the use of non-blocking FIFO operations.

Non-Blocking FIFOs

The following python code implements a non-blocking FIFO.

#fifo.py
import os, errno, time

bufferSize = 100;
PATH = "pipe";
    while 1:
    try:
        pipe = os.open(PATH, os.O_RDONLY | os.O_NONBLOCK);
        input = os.read(pipe,bufferSize);
    except OSError as err:
        if err.errno == 11:
            continue;
        else:
            raise err;
    if input:
        print(input);

os.close(pipe);

#Other functions
print "Sleep 500 ms"
time.sleep(0.5);

The code when executed is not blocked when there is no data and proceeds to print a string and sleep for 500ms

bash fifos
Bash output for writing and reading FIFO data using non-blocking method

Implementing a non-blocking FIFO in a loop would throw out a “Resource temporarily unavailable” (Error number :11) error and returns an empty string (“”) when there is no data on the FIFO. Implement an exception block for the os_open() and os_read() functions to ignore the particular error.

Refer to this post and this post for an example of using a non-blocking FIFO for inter process communication.

The code is accessible in my Github page.

Leave a Reply

Your email address will not be published.