Unable to inference using Segmind Tiny SD model

[Rehaman1429]

Hi there,

I am trying to run Segmind's Distilled Diffusion model (segmind/tiny-sd) from Hugging Face on my machine with the following specifications:

• Processor: 12th Gen Intel(R) Core(TM) i5-1235U 1.30 GHz
• Installed RAM: 16.0 GB (15.7 GB usable)
• System Type: 64-bit operating system, x64-based processor

I successfully converted the model to a single checkpoint file (.safetensors, .ckpt) as shown below:

Then, I used the convert function in stable-diffusion.cpp to convert the .safetensors file to the GGUF format. While the conversion completed without any errors, the resulting GGUF file is significantly smaller than the original model, as shown here:

When I attempt to run inference with the converted GGUF file, I encounter the following error:

It appears that the issue may lie in the conversion process to GGUF, possibly due to the fact that the model in question is a distilled version (Tiny-SD). I am wondering if anyone has worked with distilled models in this context and found a fix for this issue.

Any insights or suggestions would be greatly appreciated.

Thank you for your time and assistance!

[stduhpf]

You don't need to convert the .safetensors to gguf, sdcpp supports loading safetensors files directly. You could try loading it and see if you get the same error? (I'm pretty sure you would)

I think there are two problems here:

• the model file doesn't contain the text encoder, which would explain why It's unable to detect the version. If it was included, I believe it would be detected as a sd1.x model.
• Even if the text encoder was included, there would be another problem causing a crash when loading, because It would try loading it as a standard sd1.x model, even though the architecture is actually fairly different.

So to "fix" this you would first need to make sure the text_encoder is included in the model file, and then you'd still have to add support for this model (I think this would require making changes to model.h, model.cpp, and unet .cpp mostly).

[Rehaman1429]

Thanks for your reply @stduhpf ,

The tiny-sd model(https://huggingface.co/segmind/tiny-sd/tree/main) has text encoder:

And hence the error is not because of the absence of text encoder, it would be something else.

[stduhpf]

Yes, there is a text encoder on the hf repo, I already knew that (and that's how I knew it should be detected as a sd1.x model), I was only saying the text encoder might be missing from the safetensors file. But it's also possible it has been included under a different name than the one expected.
Anyways this model has an unsupported architecture, so it won't work regardless.