generative models · deep learning · model evaluation
Diffusion models for image generation
Study and implementation of diffusion models applied to car synthesis with the Stanford Cars dataset. It includes tuning an existing architecture and computing FID to compare versions.
- Role
- Author
- Period
- January - February 2024
- Status
- shipped
- Visibility
- public repository
The interest
Diffusion models generate images by learning to reverse a noise process. The idea is elegant: an image is destroyed by adding noise step by step, and a network is trained to undo each step. Generating means starting from pure noise and walking the path backwards.
I wanted to understand that mechanism from the inside, not use a library that did it for me.
What I did
I started from an existing implementation and modified it on four fronts: architecture layers, training algorithm, hyperparameters and loss functions. Each modification got its own notebook, so the comparison would be honest and not an average of mixed changes.
The most useful part was implementing FID, a metric that compares the distribution of generated images with that of real ones. Without it, evaluating a generative model reduces to looking at samples and deciding which one seems better, which is exactly what you are not supposed to do.
What I took away
That evaluating is harder than generating. Building the model is following a paper; knowing whether version 3 is better than version 2 demands a metric, and the metric has assumptions of its own that have to be understood.
And that at 32×32 pixels you learn the mechanism but produce nothing presentable. It is a decision I made because of compute budget and would make again, as long as the statement makes clear what the exercise is for.