Skip to content


Posts on Matplotlib and ML

Using PyQt with QtAgg in Jupyterlab – IV – simple PyQt and MPL application with background worker and receiver threads

As you read this post you are probably interested in Machine Learning [ML] and hopefully in Linux systems as a ML-platform as well. This post series wants to guide you over a bridge between the standard tool-set of Python3 notebooks in Jupyterlab for the control of ML-algorithms and graphical Qt-applications on your Linux desktop. The objective is to become more independent of some limitations of the browser based Jupyterlab notebooks.

One aspect is the use of graphical Qt-based control elements (as e.g. buttons, etc.) in desktop windows. On the other hand we want to use background threads to produce (ML) data which we later, e.g. during training runs, display in Qt windows. Background threads will also enable us to run smaller code in other cells of our notebook during long ML-runs. We are also confident that we can keep up the interactivity of both our Qt windows and Jupyterlab during such runs.

We will later use the callback machinery of Keras based ML-runs to produce ML-data and other information about a running ML-algorithm in the background of Jupyterlab. These data will be sent to Matplotlib- and Qt callback-functions in Jupyterlab which then update Qt windows.

Knowledge gained so far …

During the previous posts we have gathered enough information to now build an example PyQt application, which utilizes two background threads.

We have seen that QtAgg, a backend bridge for producing Matplotlib [MPL] plots in Qt windows, can be used for full fledged PyQt applications, too. In the first post we became familiar with some useful Qt-widgets and the general structure of Qt-Apps.

In the 2nd and 3rd posts we have learned that both Matplotlib figures and Qt-widgets must be controlled by the main thread associated with our Jupyterlab notebook. A Qt event loop is started in this thread by QtAgg for us. We have also noted that background threads controlled by QThread-objects can send signals which end up serialized in the Qt event queue of the main thread. From there they can be handled asynchronously, but in timely order by callbacks, which in turn update Qt-widgets for MPL-plots and other information. The 3rd post discussed a general pattern to employ both a raw data producing worker thread and a receiver thread to prepare the data for eventual foreground handling.

Objective of this post

In this post I will discuss a simple application that produces data with the help of two background threads according to the pattern discussed in the previous post. All data and information will periodically be sent from the background to callbacks in the main thread. Although we only use one main Qt window the structure of the application includes all key elements to serve as a blueprint for more complex situations. We will in particular discuss how to stop the background jobs and their threads in a regular way. An interesting side topic will be how one captures print output to stdout from background jobs.

Level of this post: Advanced. Some experience with Jupyterlab, QtAgg, Matplotlib and (asynchronous) PyQt is required. The first three posts of this series provide (in my opinion) a quick, though steep learning curve for PyQt newbies.

Application elements

Our PyQt application will contain three major elements in a vertical layout:

  • Two buttons to start and stop two background threads. These threads provide data for a sine-curve with steadily growing frequency and some related information text.
  • A Qt-widget for a Matplotlib figure to display the changing sine curve.
  • A series of QTextEdit widgets to display messages from the background and from callbacks in the foreground.

Our pattern requires the following threads: A “worker thread” periodically creates raw data and puts them into Python queues. A “receiver thread” reads out the queues and refines the data.

In our case the receiver thread will add additional information and data. Then signals are used to communicate with callbacks in the main thread. We send all data for widget and figure updates directly with the signals. This is done for demonstration purposes. We could also have used supplemental data queues for the purpose of inter-thread data exchange. For plotting we use Matplotlib and the related Figure.canvas-widget provided by QtAgg.

So, we have a main thread with a Qt event loop (and of course a loop for Jupyterlab REPL interaction) and two background threads which perform some (simple) asynchronous data production for us.

Our challenge: Qt and Matplotlib control with Python code in a Jupyterlab notebook

The application looks pretty simple. And its structure will indeed be simple. However, as always the devil is an expert for faults in details. In our particular situation with Jupyterlab we need to get control over the following tasks:

  • setup and start of two background threads – a worker thread and a receiver thread,
  • association of worker and receiver objects to the named threads with a respective affinity,
  • asynchronous inter-thread communication and data exchange via signals,
  • updates of Qt-widgets and integrated Matplotlib figures,
  • spinning the Qt-event-loop in the main thread to ensure quick widget updates,
  • a regular stop of thread activities and a removal of thread-related objects,
  • checking interactivity of both the Jupyterlab and the Qt-interface,
  • stability of the plot-production against potentially conflicting commands from the main thread.

All via code executed in cells of a Python notebook. An additional topic is:

  • capturing print-commands in the background and transmission of the text to the foreground.
Read More »Using PyQt with QtAgg in Jupyterlab – IV – simple PyQt and MPL application with background worker and receiver threads

Jupyterlab, matplotlib, dynamic plots – II – external Qt-windows and figure updates from foreground jobs

The work on this post series has been delayed a bit. One of my objectives was to use background jobs to directly redraw or to at least trigger a redrawing of Matplotlib figures with backends like Qt5Agg. By using background jobs I wanted to circumvent a blocking of code execution in other further Juypter notebook cells. This would to e.g. perform data analysis tasks in the foreground whilst long running Python jobs are executed in the background (e.g. jobs for training a ML-algorithm). This challenge gave me some headache in the meantime. I just say one word which tells experienced Python PyQt and Matplotlib users probably enough: thread safety.

In this post series we focus on the QtAgg backend first. Well, I found a thread safe solution for dynamic plot updates from background jobs in the context of Jupyterlab and QtAgg. But it became much more complicated than I had expected. It involves a combination of PyQT and Matplotlib. For anything simpler than that you as the user, who controls what is done in a notebook, have to be very (!) cautious.

A growing nest of problems which have to be solved

There are some key problems which we must must solve:

  1. Interactive mode of Matplotlib with the Qt(5)Agg-backend does not support automatic dynamic canvas updates of Matplotlib figures.
  2. Interactive mode together with Qt(5)Agg has negative side effects on the general behavior of Python notebooks in Jupyterlab.
  3. Jupyterlab/IPython has to use its CPU time carefully and must split it between user interactions and GUI processing. This splitting is done in the controlling (event based) loop of your Jupyter notebook, i.e. in the main thread. Whatever other background threads you may start … It is the main loop there which controls the Matplotlib backend interaction with the GUI-data-processing for screen output and the Matplotlib handling of events that may have happened on the displayed figure (GUI-Loop).
  4. Most GUI related frameworks like PyQt and Matplotib (e.g. with a QtAgg-backend) are not thread safe. It is pretty easy to crash both Matplotlib and PyQt related jobs by firing certain update command to the same figure from two running threads.
  5. Drawing actions of very many of the so called Matplotlib “artists” and other contributors to continuous updates of Matplotlib figures and as well as of PyQt elements most often must take place in the main thread. In our case: In the main loop of Jupyterlab, where you started your Matplotlib figures or even a main PyQT-window on your (Linux) desktop.
  6. The draw commands must in almost all practically relevant cases be processed sequentially. This requires a blocking communication mechanism between threads. Otherwise you take a risk of race conditions and complicated side-effects, which all may lead to errors and even may even crash of the IPython kernel. I experienced this quite often the last days.
  7. Starting side tasks with asyncio in the main asyncio loop of the Jupyter notebook will not really help you either. In some respects and regarding Matplotlib or PyQt asyncio jobs are very comparable to threads. And you may run across the same kind of problems. But I admit that for a careful user and only synchronized requests you may get quite a long way with asyncio. We will study this on our way.

This all sounds complicated – and it indeed is. But you need not yet understand all of the points made above. We have to approach a working Qt5-based solution for non-cell-blocking Python plot jobs in the background of a Juypterlab notebook step by step over some more posts. And I hope your knowledge will grow with every post. 🙂

Topics of this post

In the current post we will use the QtAgg-backend to display Matplotlib Qt5-windows on a (KDE) Linux desktop.

Note: Using “QtAgg” is the preferred mode of invoking the Qt5/6 backend. See the code below. It translates to the available version, in my case to “Qt5Agg“. I therefore will use the term Qt(5)Agg below.

We will try to update two Matplotlib figures, each in its own Qt5-window, and their sub-plots dynamically from one common loop in a notebook cell. Already this simple task requires special commands as Matplotlib’s interactive mode is only partially supported by Qt(5)Agg. In addition we will also come across a dirty side effect of QtAgg on the general behavior of notebooks in Jupyterlab 4.0.x.

Level of this post: Beginner. Python3, Jupyterlab and Matplotlib should be familiar. And you, of course, work with Linux

Windows for Matplotlib figures outside Jupyterlab and the browser?

If you work as a non-professional in the field of Machine Learning the probability is high that you use Jupyterlab with Python3 notebooks as a convenient development environment. In the first post of this series I have listed up some relevant Matplotlib graphics backends which we can use within Jupyterlab to dynamically update already existing plot figures with graphical representations of new or changed data. A practical application in the context of Machine Learning is e.g. to update plots of metric and other data in your own way during the training of Artificial Neural Networks [ANNs].

While you may be used to display Matplotlib graphics inside a Jupyter notebook (as the output of a certain cell) it may be much more convenient to get a visualization of continuously changing information in (Linux) desktop windows outside the notebook, well, even outside the browser. You may want such external windows even if you accept that the Python code executed in a notebook cell is blocking the use of other cells before all cells commands have been executed.

One reason could be that during ML training runs or other long calculation periods you may want to minimize your Jupyterlab window and work with another application. Another situation could be that the graphics window shall be updated from multiple background tasks of your notebook. Or you may just want to avoid scrolling forth and back in your notebook.

This post shows how we can use the Matplotlib Qt5-backend on a Linux/KDE system for this purpose. The basic idea is to support relatively fast changes of plot figures which are independently placed on the KDE screen outside the Jupyterlab interface in a browser.

Objectives of this post

The following graphics illustrates our wishes. It shows two Qt5 windows aside a browser window with a Jupyterlab tab.


The Qt5-windows contain figures with sub-plots (axis-frames). All sub-plots show different data series. All displayed data will later change with some frequency – we want to get a periodic update of the plots from a common update loop. All in all we will study the following aspects:

  • Creation of the external windows on the desktop with Matplotlib and Qt(5)Agg.
  • Initially filling and singular updates of the sub-plots with the help of code from various notebook cells.
  • Periodical update of all figures and their subplots from a common loop in a notebook cell.

Below I will discuss the necessary steps with the help of a simple example.

Read More »Jupyterlab, matplotlib, dynamic plots – II – external Qt-windows and figure updates from foreground jobs

Jupyterlab, matplotlib, dynamic plots – I – relevant backends

When we work with Deep Neural Networks on limited HW-resources we must get an overview over CPU- and VRAM-consumption, training progress, change of metrical variables of our network models, etc. Most of us will probably want to see the development of our system- and model-related variables in a graphical way. All of this requires dynamic plots, which are updated periodically and thus display monitored data live.

As non-professionals we probably use Python code in standalone Jupyter notebooks or (multiple) Python notebooks in a common Jupyterlab environment. All within a browser.

The Jupyterlab interface resembles a typical IDE. Its structure and code are more complicated than those of pure Jupyter Notebooks. Jupyterlab comes with more configuration options, but also with more SW-problems. But Jupyterlab has some notable advantages. Once you turned to it you probably won’t go back to isolated Jupyter notebooks.

In this post series I want to discuss how you can create, update and organize multiple dynamic plots with Jupyterlab 4 (4.0.8 in a Python 3.9 environment), Python 3 and Matplotlib. There are various options and – depending on which way you want to go – one must also overcome some obstacles. I will describe options only for a Linux KDE environment. But things should work on Gnome and other Linux GUIs, too. I do not care about Microbesoft.

In this first post I show you how to get an overview over Matplotlib’s relevant graphics backends. Further posts will then describe whether and how the backends “Qt5Agg”, “TkAgg”, “Gtk3Agg”, “WebAgg” and “ipympl” work. Only two of these backends automatically and completely update plot figures after each of a series of multiple plot commands in a notebook cell. The Gtk3-backend poses a specific problem – but this gives me a welcome opportunity to discuss a method to trigger periodic plot updates with the help of user-controlled asynchronous background tasks.

Addendum and major changes, 11/25/23 and 11/28/23: This post was updated and partially rewritten according to new insights. I have also added an objective regarding plot updates from background jobs.

Objectives – updates of multiple plot frames with live data

We need dynamic, i.e. live plots, whose frames are once created in various Jupyter cells – and which later can be updated based on changed and extended data values.

  1. We will organize multiple plot figures at some central place on our desktop screen – within Jupyterlab or outside Jupyterlab on the Linux desktop GUI.
  2. We will perform updates of multiple plots with changed data – first one after the other with code in separate and independent Jupyter cells (and with the help of respective functions).
  3. We will perform live updates of multiple plots in parallel and continuously by one common loop that gathers new data and updates related figures.
  4. We will perform continuous updates of our plot figures from background jobs.
Read More »Jupyterlab, matplotlib, dynamic plots – I – relevant backends