Skip to content

eremo

AdamW for a ResNet56v2 – II – Adam with weight decay vs. AdamW, linear LR-schedule and L2-regularization

This series is about a ResNetv56v2 tested on the CIFAR10 dataset. In the last post AdamW for a ResNet56v2 – I – a detailed look at results based on the Adam optimizer we investigated a piecewise constant reduction schedule for the Learning Rate [LR] over 200 epochs. We found that we could reproduce results of R. Atienza, who had claimed validation accuracy values of with the Adam optimizer. We saw a dependency on the batch size [BS] – and concluded that BS=64 was a good… Read More »AdamW for a ResNet56v2 – II – Adam with weight decay vs. AdamW, linear LR-schedule and L2-regularization

AdamW for a ResNet56v2 – I – a detailed look at results based on the Adam optimizer

This post requires Javascript to display formulas! The last days I started to work on ResNets again. The first thing I did was to use a ResNet code which Rowel Atienza has published in his very instructive book “Advanced Deep Learning with Tensorflow2 and Keras” [1]. I used the code on the CIFAR10 dataset. Atienza’s approach for this test example is to use image augmentation in addition to L2-regularization with the good old Adam optimizer and a piecewise constant Learning Rate schedule. For a ResNet56v2… Read More »AdamW for a ResNet56v2 – I – a detailed look at results based on the Adam optimizer

TF 2.16, Keras 3, alternatives for deprecated ImageDataGenerator

These days I started again to work with ResNets and images. To improve accuracy e.g. for classification projects the test on evaluation datasets is the only thing that counts regarding accuracy. One should use some form of data augmentation, best statistically during training, to prevent overfitting of a model. In the past I have often used Keras’ ImageDataGenerator. ImageDatagenerator is “deprecated” in Keras 3 DataImageGenerator was an easy to use tool. With Keras 3 it is now classified “deprecated”. I do not regard this as… Read More »TF 2.16, Keras 3, alternatives for deprecated ImageDataGenerator

Prompt based image generation with Stable Diffusion on a TI 4060 ?!

Midjourney and OpenAI give you access to image generators which create images based on key words in a text prompt. To access Dall-E2 you need to pay money. The tools of Midjourney are no longer free because they have been misused. OpenArt offers you a free service – but the images are public domain. So, some days ago I asked myself whether one can perform prompt based image creation on a Linux PC with a consumer board and a low price graphics card. I should… Read More »Prompt based image generation with Stable Diffusion on a TI 4060 ?!