Help me bring stable-diffusion.cpp to AO Supercomputer !

[aymenx17]

Hello there great community, I know this help request is a bit unusual as it is related to blockchain. Basically AO Supercomputer which runs on top of Arweave blockchain has limited compute (4GB of RAM) and it's perfect for a small text-to-image model like the sd-nano (128x128). I would be very pleased if we could ported it there !

[stduhpf]

I think sd-nano is a fine tune of sd2? It should already be supported, no?

[aymenx17]

Yes exactly, a finetune of sd2 and I was able to run that on my local cpu. But what I'm trying to get help with is implementing something like this repo: https://github.com/PeterFarber/AO-Llama/tree/main
AO-Llama allow for the use of Llama.cpp in AO. But I thought we could bring text-to-image models into AO as well.

[Green-Sky]

So, you want a wasm build that fits into 4gig during inference? hmm
bin/sd -m ../models/sd_turbo-f16-q8_0.gguf --cfg-scale 1 --steps 4 --diffusion-fa --vae-tiling -p "lovely cat"
This stays below 1.7gigs edit: 1.7gig non-shared, so +2gig for the model makes 3.7gig for me.
It generates a 512x512, but his is not using wasm, just normal cpu with avx2...

(sd_turbo is based on sd2)

[Green-Sky]

You can find the model here https://huggingface.co/Green-Sky/SD-Turbo-GGUF/blob/main/sd_turbo-f16-q8_0.gguf

[aymenx17]

yes building an AO wasm. The cat is lovely indeed and lower resolution can even be faster, I found this that works already: https://huggingface.co/NikolayKozloff/stable-diffusion-nano-2-1-ckpt/resolve/main/stable-diffusion-nano-2-1.ckpt

[aymenx17]

Yes the hard part is building is eventually building the WASM. And there are a few steps to that:

• Compiling into a static library
• Writing cpp code to use that library. Writing the bindings and then the lua code. Theoretically only 3 code modules.
• Build the wasm64

I tried myself and I ended up building a wasm64 (sd.wasm, sd.js) but then I've been told it wasn't correct.
The build should be like: https://github.com/PeterFarber/AO-Llama/blob/main/build.sh

A brief intro to AO Supercomputer.

Basically it's a decentralised compute with a decentralised hardrive.
You can create autonomous processes that could run forever. They have a model that fits with that future where digital entities are self sovereign, it makes another level of gaming and virtual spaces.

https://cookbook_ao.g8way.io/welcome/getting-started.html