These days I started again to work with ResNets and images. To improve accuracy e.g. for classification projects the test on evaluation datasets is the only thing that counts regarding accuracy. One should use some form of data augmentation, best statistically during training, to prevent overfitting of a model. In the past I have often used Keras’ ImageDataGenerator.
ImageDatagenerator is “deprecated” in Keras 3
DataImageGenerator was an easy to use tool. With Keras 3 it is now classified “deprecated”. I do not regard this as a wrong step of the Keras team. Using ImageDataGenerator has never been an efficient way to do augmentation. An alternative is using “preprocessing layers” of your Keras based model – and do the preprocessing during training on the GPU.
Alternative: Keras and Keras CV Preprocessing Layers
Keras 3 offers a variety of layers for image preprocessing. See the Keras 3 API documentation on “Image augmentation layers” for more information. While these layers are certainly helpful one may have a requirement which is not fulfilled by these layers directly:
- We may not want to perform augmentation (with some random parameters) on every image, but only on some images. I.e. we want to apply augmentation only with a certain rate.
- We may want to build a pipeline with a certain number of selected Keras (or Keras CV) preprocessing layers.
Normally we would have to do some programming using the call-interface of layers on our own to fulfill these requirements. However, thanks to the Keras CV team, there is an alternative. See the documentation at “Keras 3 API documentation / KerasCV / Layers / Augmentation layers” and also have a view into the source code.
The pseudo-layer “RandAugment layer” offers a parameterized application of an augmentation pipeline were a full spectrum of augmentation transformations is applied to your images with a certain rate and repetition factor per image. I call it pseudo-layer because it actually build a defined sequence of layers for the most often used augmentation tasks. An alternative is given by another class “RandomAugmentationPipeline layer“. This class allows you to control which of the available transformations are applied by a sequence of layers to input image tensors of a model.
Both pseudo layers accept a parameter “rate” which defines the probability that augmentation is applied to an image of your batches. (You have to regard an additional parameter “augmentations_per_image” to calculate the eventual probability that an image is augmented.) See the classes code for details. See here for an example how to use “RandomAugmentationPipeline”.
For those of you who just want to construct your own augmented image datasets the very simple layer class “Augmenter” may be interesting, too. See here and here for usage and details.
A general introduction into image preprocessing and augmentation with the help of Keras layers is given at https://www.tensorflow.org/ tutorials/ images/ data augmentation.
Note that you can add layers directly to your model – or apply preprocessing to the model data asynchronously on the CPU during training. An application of the latter technique with Keras CV tools is described at https://keras.io/guides for Keras CV. See
https://keras.io/guides/ keras_cv/ cut_mix mix_up and rand_augment/.
Layers for a variety of typical image preprocessing and augmentation tasks ..
Together with the standard Keras 3 layers for augmentation there is a multitude of image preprocessing operations available you can cover by a pipeline. The list of operations include: resizing; grayscaling; zoom; changes to saturation, brightness and contrast; solarization; shearing; cropping; rotation; flipping; translation (= horizontal/vertical shifts).
Summary
Keras 3 strongly “encourages” you to perform augmentation tasks with the help of specialized Keras model layers. The Keras CV team has provided some complex layers which allow you to create a sequence of augmentation layers at the your model. Besides selecting specific augmentation layers you can also define a statistical rate by which image augmentation shall be done. In forthcoming posts on ResNets I will show code fragments of an application of Keras CV tools for image augmentation.