When we work with Deep Neural Networks on limited HW-resources we must get an overview over CPU- and VRAM-consumption, training progress, change of metrical variables of our network models, etc. Most of us will probably want to see the development of our system- and model-related variables in a graphical way. All of this requires dynamic plots, which are updated periodically and thus display monitored data live.
As non-professionals we probably use Python code in standalone Jupyter notebooks or (multiple) Python notebooks in a common Jupyterlab environment. All within a browser.
The Jupyterlab interface resembles a typical IDE. Its structure and code are more complicated than those of pure Jupyter Notebooks. Jupyterlab comes with more configuration options, but also with more SW-problems. But Jupyterlab has some notable advantages. Once you turned to it you probably won’t go back to isolated Jupyter notebooks.
In this post series I want to discuss how you can create, update and organize multiple dynamic plots with Jupyterlab 4 (4.0.8 in a Python 3.9 environment), Python 3 and Matplotlib. There are various options and – depending on which way you want to go – one must also overcome some obstacles. I will describe options only for a Linux KDE environment. But things should work on Gnome and other Linux GUIs, too. I do not care about Microbesoft.
In this first post I show you how to get an overview over Matplotlib’s relevant graphics backends. Further posts will then describe whether and how the backends “Qt5Agg”, “TkAgg”, “Gtk3Agg”, “WebAgg” and “ipympl” work. Only two of these backends automatically and completely update plot figures after each of a series of multiple plot commands in a notebook cell. The Gtk3-backend poses a specific problem – but this gives me a welcome opportunity to discuss a method to trigger periodic plot updates with the help of user-controlled asynchronous background tasks.
Addendum and major changes, 11/25/23 and 11/28/23: This post was updated and partially rewritten according to new insights. I have also added an objective regarding plot updates from background jobs.
Objectives – updates of multiple plot frames with live data
- We will organize multiple plot figures at some central place on our desktop screen – within Jupyterlab or outside Jupyterlab on the Linux desktop GUI.
- We will perform updates of multiple plots with changed data – first one after the other with code in separate and independent Jupyter cells (and with the help of respective functions).
- We will perform live updates of multiple plots in parallel and continuously by one common loop that gathers new data and updates related figures.
- We will perform continuous updates of our plot figures from background jobs.