Load Diffusion Local Time XL in a dedicated window →

Diffusion Local Time, extra large

Diffusion Local Time is an art piece that displays the time in surrealistic landscapes. At CVPR, I displayed two versions, Diffusion Local Time XL and Diffusion Local Time XS.

Extra large is precomputed on a desktop computer with tweening from minute to minute like the Dali Clock! 09:44 on Diffusion Local Time XL is displayed as

09:44 on a clock face

The full code to generate these images is available.

Diffusion Local Time extra small is realtime and interactive on a Raspberry Pi! A simple client-server setup on a Pi allows users to change the prompt on demand and the pi will render a new minute in the new prompt.

0944, xl and xs

For ultra-high image quality, Diffusion Local Time XL uses 50-step Stable Diffusion XL, with the monster labs QR code controlnet. SDXL is exceptionally high quality compared to SD1.5, and 50 steps of an undistilled model produces ultra-realistic imagery compared to a few steps of a distilled model.

The tweening is a smoothstep across a fixed number of identically-spaced interpolations between each control image. Future directions can include interpolating the glyph shapes like the dali clock.

Diffusion Local Time, extra small

For realtime interactivity on a Raspberry Pi, the diffusion model must be more lightweight, and we can use Stable Diffusion XS. There are only around 300M parameters for SDXS, roughly a tenth of SDXL, and the diffusion model is distilled to only render using one step. The quality is lower than SD1.5, especially using a different ControlNet: the one control net for SDXS is trained on sketches of images, instead of luminosity. For successful imagery, it is useful to pick prompts that evoke objects that be assembled, like (as above) ferns in redwoods, or spices on a countertop. We can generate 1024x512 imagery in about 45 seconds on a Raspberry Pi 5, which is realtime for an unanimated workload.

Sample prompts that CVPR-goers entered

  • Kittens poking their heads out of blankets (installed in an urban residence)
  • Something in space
  • A messy room