Stable Diffusion Clock
Three clock faces! Open source! Check it out!
How this was built
QR Code Controlnet + Stable Diffusion, a hyperlegible font from the Braille institute, a g5.xlarge that generates an image in 5.7 seconds, and some prompts.
QR Code Controlnet + Stable Diffusion
A few months ago nhciao posted several pieces of art that scanned as QR codes.
Recently, Angry Penguin made a Huggingface space that implemented that workflow for arbitrary prompts and images, with fully available source.
The luminance controlnet model and the upscaler are both publicly available on Huggingface, so it is possible to run those locally with only the cost of keeping the box running.
Typography from the Braille Institute
The numbers of the clock face need to be legible, and typographers have investigated this exact problem! This is the Atkinson font which was initially designed for letterform visibility, and its numbers are very legible as well. Even at the 1024x1024 final resolution the forms are legible, filtered through the various prompts.
g5.xlarge
The relationship that one has with a clock depends on how fast it goes. Running this controlnet on a Mac Studio in cpu mode (I could not get MPS to be stable) runs in a few minutes (at residential energy prices). Running on a g5.xlarge on AWS runs in 5.78 seconds in float16 precision (at $1/hr). This latency is low enough to generate several types of images for a few instances of each clock face per minute.
Prompts
Bosch
detail of a new hieronymus bosch painting
Hieronymus Bosch artwork can be blobby and weird with light figures on a dark background, and the images often fit the control image and are similarly spooky at first glance.
Train
beautiful detail of a train map
The train maps can be fascinating to look at, sometimes installed in an actual train, sometimes a low depth-of-field close up, always nonsensical after thorough observation. Need to work on this more. (Very open to suggestions!)
Urban
watercolor of a leafy pedestrian mall at golden hour with multiracial genderqueer joggers and bicyclists and wheelchair users talking and laughing
This comes from the sort of city I want to live in: leafy, close-knit, welcoming, without danger from wheeled vehicles.
Inspirations from other artists
There are current living artists who are making art in these styles and they would love you to support their work and their vision! (Send me more suggestions!)
Open source
All of the pieces come together in https://github.com/lsb/stable-diffusion-clock. The viewer could probably be improved from a static HTML page that refreshes itself every few seconds. The images are mostly deterministic: their seed comes from the epoch time at generation.
Apologies to people using this as a clock who are not also in Pacific time.
Let me know what you think!
Some open questions I’ve been thinking of:
- Are there better prompts? I’m no Staff Prompt Engineer, but the frequency of new images per minute and gives enough variation to occasionally produce gems. The 0857 imagery at the top of this post was taken from the clock while running.
- What is the latency on a new Raspberry Pi 5? A Jetson board from NVIDIA? Can this run on a little SBC, displaying the time like a beautiful clock, requiring no ongoing resources except power? (Most clocks do not cost $25/day to operate, the retail cost of a
g5.xlarge.) - Control over training data? I’m unsure of what the sources of the training data for the upscaler and especially the QR Code controlnet were.
Tell me what you think! leebutterman@gmail.com