We introduce SODA, a self-supervised diffusion model, designed for representation learning. The model incorporates an image encoder, which distills a source view into a compact representation, that, in turn, guides the generation of related novel views. We show that by imposing a tight bottleneck between the encoder and a denoising decoder, and leveraging novel view synthesis as a self-supervised objective, we can turn diffusion models into strong representation learners, capable of capturing visual semantics in an unsupervised manner. To the best of our knowledge, SODA is the first diffusion model to succeed at ImageNet linear-probe classification, and, at the same time, it accomplishes reconstruction, editing and synthesis tasks across a wide range of datasets. Further investigation reveals the disentangled nature of its emergent latent space, that serves as an effective interface to control and manipulate the model's produced images. All in all, we aim to shed light on the exciting and promising potential of diffusion models, not only for image generation, but also for learning rich and robust representations.
@article{hudson2023soda,
title={SODA: Bottleneck Diffusion Models for Representation Learning},
author={Hudson, Drew A and Zoran, Daniel and Malinowski, Mateusz and Lampinen, Andrew K and Jaegle, Andrew and McClelland, James L and Matthey, Loic and Hill, Felix and Lerchner, Alexander},
journal={arXiv preprint arXiv:2311.17901},
year={2023}
}
Latent Interpolation
Style & Content Separation
Novel View Synthesis (from 1 view)
Novel View Synthesis (from 2 views)
Unsupervised Discovery of Semantic Attributes
Blossoming:
Petals Number:
Golden Core:
Blue Color:
Red Color:
Yellow Color:
Maturity:
Reconstructions (Original Input & Reconstructred Output):
