Skip to content

matplotlib backend

Using PyQt with QtAgg in Jupyterlab – I – a first simple example

As my readers know I presently study how to work with background jobs for plot and information updates in Jupyterlab. The reason for this endeavor is that I want to organize my work with ML- training and long evaluation runs a bit differently in the future. In particular I want to have the freedom of working in other regions (other cells) of a Python3 notebook while the long running jobs do their work in the background. This includes that these other cells may also start major background jobs or run code in parallel to the background tasks.

After some experiments I think that for my particular purposes a combination of QTAgg, Matplotlib and PyQt provides the best solution.

What I like about PyQt is its very clear structure. And you can use QTextEdit-widgets as a kind of canvas for print output. And e special Figure-widget for Matplotlib. Using PyQt opens up for many new options to build nice application windows around your ML-algorithms on your Linux desktop. Which will be extremely valuable for presentations or customer end applications.

In this post series I will present a few test applications that cover a lot of topics around Qt-widgets, widget- communication, threads, signals and events. We will start with a simple foreground application. Then we turn to threads and signals between threads. Afterward we integrate Matplotlib and deepen our understanding of handling threads in PyQt applications for Matplotlib and print output.

Objectives of this post

This first post translates a question posed at Stackoverflow [1] into a more complex example. With it I want to show

  • how to set up your own PyQt application in a Jupyterlab (Python) environment and display it directly as a Qt-window on your desktop,
  • how to create a nested PyQt “sub-window”-structure with the help of a PyQt-window layout,
  • how to react to events,
  • how to give a central widget its own paintEvent and a painter for drawing graphical primitives,
  • how to “hide” a widget element,
  • how you control which of your graphical primitives is drawn and which not.

This application does not yet use background jobs. So, due to its present structure it will block running any Python code in other notebook cells while we have some action going on in the application’s PyQt-widgets.

We are going to change this during the course of this post series. For the time being my main objective is only to show that PyQt can be used at all with QtAgg in Jupyterlab notebooks.

My “application” is a bit stupid in so far as 2 main button only trigger a continuous dynamic painting of a red line from a sub-window’s center to the mouse pointer or to the corners of their surrounding sub-window areas. It is simple and trivial, but it nevertheless gave me some fun constructing it.

If you like, you may think of the red line a “laser beam” shooting at the mouse pointer or the frame corners. I will use this term below when I discuss the functionality. There is also a video link at the end of the post.

All of this has obviously nothing to do with ML, yet, but it gives you a feeling how we can compose PyQt applications in a Jupyterlab environment. In other posts of this blog we will use our acquired PyQt knowledge to put Keras based ML-machinery into our own graphical interfaces on the Linux desktop. I.e. outside the browser tab for Jupyterlab.

Level of this post: In parts advanced. Not because the example is so complicated. Nevertheless some background information on Qt-Window layouts and Widgets is required. Good introductions are given here. Details for all Qt-widgets that I use below can be found at https://wiki.qt.io/.

But note: For a Jupyterlab Python notebook with an QtAgg-backend you do not need the whole app.exec() stuff described in the tutorial examples.

Read More »Using PyQt with QtAgg in Jupyterlab – I – a first simple example

Jupyterlab, matplotlib, dynamic plots – II – external Qt-windows and figure updates from foreground jobs

The work on this post series has been delayed a bit. One of my objectives was to use background jobs to directly redraw or to at least trigger a redrawing of Matplotlib figures with backends like Qt5Agg. By using background jobs I wanted to circumvent a blocking of code execution in other further Juypter notebook cells. This would to e.g. perform data analysis tasks in the foreground whilst long running Python jobs are executed in the background (e.g. jobs for training a ML-algorithm). This challenge gave me some headache in the meantime. I just say one word which tells experienced Python PyQt and Matplotlib users probably enough: thread safety.

In this post series we focus on the QtAgg backend first. Well, I found a thread safe solution for dynamic plot updates from background jobs in the context of Jupyterlab and QtAgg. But it became much more complicated than I had expected. It involves a combination of PyQT and Matplotlib. For anything simpler than that you as the user, who controls what is done in a notebook, have to be very (!) cautious.

A growing nest of problems which have to be solved

There are some key problems which we must must solve:

  1. Interactive mode of Matplotlib with the Qt(5)Agg-backend does not support automatic dynamic canvas updates of Matplotlib figures.
  2. Interactive mode together with Qt(5)Agg has negative side effects on the general behavior of Python notebooks in Jupyterlab.
  3. Jupyterlab/IPython has to use its CPU time carefully and must split it between user interactions and GUI processing. This splitting is done in the controlling (event based) loop of your Jupyter notebook, i.e. in the main thread. Whatever other background threads you may start … It is the main loop there which controls the Matplotlib backend interaction with the GUI-data-processing for screen output and the Matplotlib handling of events that may have happened on the displayed figure (GUI-Loop).
  4. Most GUI related frameworks like PyQt and Matplotib (e.g. with a QtAgg-backend) are not thread safe. It is pretty easy to crash both Matplotlib and PyQt related jobs by firing certain update command to the same figure from two running threads.
  5. Drawing actions of very many of the so called Matplotlib “artists” and other contributors to continuous updates of Matplotlib figures and as well as of PyQt elements most often must take place in the main thread. In our case: In the main loop of Jupyterlab, where you started your Matplotlib figures or even a main PyQT-window on your (Linux) desktop.
  6. The draw commands must in almost all practically relevant cases be processed sequentially. This requires a blocking communication mechanism between threads. Otherwise you take a risk of race conditions and complicated side-effects, which all may lead to errors and even may even crash of the IPython kernel. I experienced this quite often the last days.
  7. Starting side tasks with asyncio in the main asyncio loop of the Jupyter notebook will not really help you either. In some respects and regarding Matplotlib or PyQt asyncio jobs are very comparable to threads. And you may run across the same kind of problems. But I admit that for a careful user and only synchronized requests you may get quite a long way with asyncio. We will study this on our way.

This all sounds complicated – and it indeed is. But you need not yet understand all of the points made above. We have to approach a working Qt5-based solution for non-cell-blocking Python plot jobs in the background of a Juypterlab notebook step by step over some more posts. And I hope your knowledge will grow with every post. 🙂

Topics of this post

In the current post we will use the QtAgg-backend to display Matplotlib Qt5-windows on a (KDE) Linux desktop.

Note: Using “QtAgg” is the preferred mode of invoking the Qt5/6 backend. See the code below. It translates to the available version, in my case to “Qt5Agg“. I therefore will use the term Qt(5)Agg below.

We will try to update two Matplotlib figures, each in its own Qt5-window, and their sub-plots dynamically from one common loop in a notebook cell. Already this simple task requires special commands as Matplotlib’s interactive mode is only partially supported by Qt(5)Agg. In addition we will also come across a dirty side effect of QtAgg on the general behavior of notebooks in Jupyterlab 4.0.x.

Level of this post: Beginner. Python3, Jupyterlab and Matplotlib should be familiar. And you, of course, work with Linux

Windows for Matplotlib figures outside Jupyterlab and the browser?

If you work as a non-professional in the field of Machine Learning the probability is high that you use Jupyterlab with Python3 notebooks as a convenient development environment. In the first post of this series I have listed up some relevant Matplotlib graphics backends which we can use within Jupyterlab to dynamically update already existing plot figures with graphical representations of new or changed data. A practical application in the context of Machine Learning is e.g. to update plots of metric and other data in your own way during the training of Artificial Neural Networks [ANNs].

While you may be used to display Matplotlib graphics inside a Jupyter notebook (as the output of a certain cell) it may be much more convenient to get a visualization of continuously changing information in (Linux) desktop windows outside the notebook, well, even outside the browser. You may want such external windows even if you accept that the Python code executed in a notebook cell is blocking the use of other cells before all cells commands have been executed.

One reason could be that during ML training runs or other long calculation periods you may want to minimize your Jupyterlab window and work with another application. Another situation could be that the graphics window shall be updated from multiple background tasks of your notebook. Or you may just want to avoid scrolling forth and back in your notebook.

This post shows how we can use the Matplotlib Qt5-backend on a Linux/KDE system for this purpose. The basic idea is to support relatively fast changes of plot figures which are independently placed on the KDE screen outside the Jupyterlab interface in a browser.

Objectives of this post

The following graphics illustrates our wishes. It shows two Qt5 windows aside a browser window with a Jupyterlab tab.

 

The Qt5-windows contain figures with sub-plots (axis-frames). All sub-plots show different data series. All displayed data will later change with some frequency – we want to get a periodic update of the plots from a common update loop. All in all we will study the following aspects:

  • Creation of the external windows on the desktop with Matplotlib and Qt(5)Agg.
  • Initially filling and singular updates of the sub-plots with the help of code from various notebook cells.
  • Periodical update of all figures and their subplots from a common loop in a notebook cell.

Below I will discuss the necessary steps with the help of a simple example.

Read More »Jupyterlab, matplotlib, dynamic plots – II – external Qt-windows and figure updates from foreground jobs