Skip to content

qt5Agg

Jupyterlab, matplotlib, dynamic plots – II – external Qt-windows and figure updates from foreground jobs

The work on this post series has been delayed a bit. One of my objectives was to use background jobs to directly redraw or to at least trigger a redrawing of Matplotlib figures with backends like Qt5Agg. By using background jobs I wanted to circumvent a blocking of code execution in other further Juypter notebook cells. This would to e.g. perform data analysis tasks in the foreground whilst long running Python jobs are executed in the background (e.g. jobs for training a ML-algorithm). This challenge gave me some headache in the meantime. I just say one word which tells experienced Python PyQt and Matplotlib users probably enough: thread safety.

In this post series we focus on the QtAgg backend first. Well, I found a thread safe solution for dynamic plot updates from background jobs in the context of Jupyterlab and QtAgg. But it became much more complicated than I had expected. It involves a combination of PyQT and Matplotlib. For anything simpler than that you as the user, who controls what is done in a notebook, have to be very (!) cautious.

A growing nest of problems which have to be solved

There are some key problems which we must must solve:

  1. Interactive mode of Matplotlib with the Qt(5)Agg-backend does not support automatic dynamic canvas updates of Matplotlib figures.
  2. Interactive mode together with Qt(5)Agg has negative side effects on the general behavior of Python notebooks in Jupyterlab.
  3. Jupyterlab/IPython has to use its CPU time carefully and must split it between user interactions and GUI processing. This splitting is done in the controlling (event based) loop of your Jupyter notebook, i.e. in the main thread. Whatever other background threads you may start … It is the main loop there which controls the Matplotlib backend interaction with the GUI-data-processing for screen output and the Matplotlib handling of events that may have happened on the displayed figure (GUI-Loop).
  4. Most GUI related frameworks like PyQt and Matplotib (e.g. with a QtAgg-backend) are not thread safe. It is pretty easy to crash both Matplotlib and PyQt related jobs by firing certain update command to the same figure from two running threads.
  5. Drawing actions of very many of the so called Matplotlib “artists” and other contributors to continuous updates of Matplotlib figures and as well as of PyQt elements most often must take place in the main thread. In our case: In the main loop of Jupyterlab, where you started your Matplotlib figures or even a main PyQT-window on your (Linux) desktop.
  6. The draw commands must in almost all practically relevant cases be processed sequentially. This requires a blocking communication mechanism between threads. Otherwise you take a risk of race conditions and complicated side-effects, which all may lead to errors and even may even crash of the IPython kernel. I experienced this quite often the last days.
  7. Starting side tasks with asyncio in the main asyncio loop of the Jupyter notebook will not really help you either. In some respects and regarding Matplotlib or PyQt asyncio jobs are very comparable to threads. And you may run across the same kind of problems. But I admit that for a careful user and only synchronized requests you may get quite a long way with asyncio. We will study this on our way.

This all sounds complicated – and it indeed is. But you need not yet understand all of the points made above. We have to approach a working Qt5-based solution for non-cell-blocking Python plot jobs in the background of a Juypterlab notebook step by step over some more posts. And I hope your knowledge will grow with every post. 🙂

Topics of this post

In the current post we will use the QtAgg-backend to display Matplotlib Qt5-windows on a (KDE) Linux desktop.

Note: Using “QtAgg” is the preferred mode of invoking the Qt5/6 backend. See the code below. It translates to the available version, in my case to “Qt5Agg“. I therefore will use the term Qt(5)Agg below.

We will try to update two Matplotlib figures, each in its own Qt5-window, and their sub-plots dynamically from one common loop in a notebook cell. Already this simple task requires special commands as Matplotlib’s interactive mode is only partially supported by Qt(5)Agg. In addition we will also come across a dirty side effect of QtAgg on the general behavior of notebooks in Jupyterlab 4.0.x.

Level of this post: Beginner. Python3, Jupyterlab and Matplotlib should be familiar. And you, of course, work with Linux

Windows for Matplotlib figures outside Jupyterlab and the browser?

If you work as a non-professional in the field of Machine Learning the probability is high that you use Jupyterlab with Python3 notebooks as a convenient development environment. In the first post of this series I have listed up some relevant Matplotlib graphics backends which we can use within Jupyterlab to dynamically update already existing plot figures with graphical representations of new or changed data. A practical application in the context of Machine Learning is e.g. to update plots of metric and other data in your own way during the training of Artificial Neural Networks [ANNs].

While you may be used to display Matplotlib graphics inside a Jupyter notebook (as the output of a certain cell) it may be much more convenient to get a visualization of continuously changing information in (Linux) desktop windows outside the notebook, well, even outside the browser. You may want such external windows even if you accept that the Python code executed in a notebook cell is blocking the use of other cells before all cells commands have been executed.

One reason could be that during ML training runs or other long calculation periods you may want to minimize your Jupyterlab window and work with another application. Another situation could be that the graphics window shall be updated from multiple background tasks of your notebook. Or you may just want to avoid scrolling forth and back in your notebook.

This post shows how we can use the Matplotlib Qt5-backend on a Linux/KDE system for this purpose. The basic idea is to support relatively fast changes of plot figures which are independently placed on the KDE screen outside the Jupyterlab interface in a browser.

Objectives of this post

The following graphics illustrates our wishes. It shows two Qt5 windows aside a browser window with a Jupyterlab tab.

 

The Qt5-windows contain figures with sub-plots (axis-frames). All sub-plots show different data series. All displayed data will later change with some frequency – we want to get a periodic update of the plots from a common update loop. All in all we will study the following aspects:

  • Creation of the external windows on the desktop with Matplotlib and Qt(5)Agg.
  • Initially filling and singular updates of the sub-plots with the help of code from various notebook cells.
  • Periodical update of all figures and their subplots from a common loop in a notebook cell.

Below I will discuss the necessary steps with the help of a simple example.

Read More »Jupyterlab, matplotlib, dynamic plots – II – external Qt-windows and figure updates from foreground jobs

Jupyterlab, matplotlib, dynamic plots – I – relevant backends

When we work with Deep Neural Networks on limited HW-resources we must get an overview over CPU- and VRAM-consumption, training progress, change of metrical variables of our network models, etc. Most of us will probably want to see the development of our system- and model-related variables in a graphical way. All of this requires dynamic plots, which are updated periodically and thus display monitored data live.

As non-professionals we probably use Python code in standalone Jupyter notebooks or (multiple) Python notebooks in a common Jupyterlab environment. All within a browser.

The Jupyterlab interface resembles a typical IDE. Its structure and code are more complicated than those of pure Jupyter Notebooks. Jupyterlab comes with more configuration options, but also with more SW-problems. But Jupyterlab has some notable advantages. Once you turned to it you probably won’t go back to isolated Jupyter notebooks.

In this post series I want to discuss how you can create, update and organize multiple dynamic plots with Jupyterlab 4 (4.0.8 in a Python 3.9 environment), Python 3 and Matplotlib. There are various options and – depending on which way you want to go – one must also overcome some obstacles. I will describe options only for a Linux KDE environment. But things should work on Gnome and other Linux GUIs, too. I do not care about Microbesoft.

In this first post I show you how to get an overview over Matplotlib’s relevant graphics backends. Further posts will then describe whether and how the backends “Qt5Agg”, “TkAgg”, “Gtk3Agg”, “WebAgg” and “ipympl” work. Only two of these backends automatically and completely update plot figures after each of a series of multiple plot commands in a notebook cell. The Gtk3-backend poses a specific problem – but this gives me a welcome opportunity to discuss a method to trigger periodic plot updates with the help of user-controlled asynchronous background tasks.

Addendum and major changes, 11/25/23 and 11/28/23: This post was updated and partially rewritten according to new insights. I have also added an objective regarding plot updates from background jobs.

Objectives – updates of multiple plot frames with live data

We need dynamic, i.e. live plots, whose frames are once created in various Jupyter cells – and which later can be updated based on changed and extended data values.

  1. We will organize multiple plot figures at some central place on our desktop screen – within Jupyterlab or outside Jupyterlab on the Linux desktop GUI.
  2. We will perform updates of multiple plots with changed data – first one after the other with code in separate and independent Jupyter cells (and with the help of respective functions).
  3. We will perform live updates of multiple plots in parallel and continuously by one common loop that gathers new data and updates related figures.
  4. We will perform continuous updates of our plot figures from background jobs.
Read More »Jupyterlab, matplotlib, dynamic plots – I – relevant backends