Skip to content

Files

Latest commit

udaij12udaij12
Updating examples for security tags (pytorch#3224)
Jul 3, 2024
589b351 · Jul 3, 2024

History

History
This branch is 6 commits ahead of, 8 commits behind pytorch/serve:master.

diffusers

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
Download_model.py
Download_model.py
Jan 18, 2023
Readme.md
Readme.md
Jul 3, 2024
config.properties
config.properties
Oct 2, 2023
query.py
query.py
Nov 3, 2022
requirements.txt
requirements.txt
Nov 3, 2022
sample.txt
sample.txt
Nov 3, 2022
sample_v1.json
sample_v1.json
Nov 3, 2022
sample_v2.json
sample_v2.json
Nov 3, 2022
stable_diffusion_handler.py
stable_diffusion_handler.py
Aug 30, 2023

Readme.md

Running Stable diffusion model using Huggingface Diffusers in Torchserve.

Step 1: Download model

Set access token generated form Huggingface in Download_model.py file

Install dependencies

pip install -r requirements.txt
python Download_model.py

Step 2: Compress downloaded model

NOTE: Install Zip cli tool

Navigate back to model directory.

cd Diffusion_model
zip -r ../model.zip *

Step 3: Generate MAR file

Navigate up one level to diffusers directory.

torch-model-archiver --model-name stable-diffusion --version 1.0 --handler stable_diffusion_handler.py --extra-files model.zip -r requirements.txt

Step 4: Start torchserve

Update config.properties and start torchserve

torchserve --start --ts-config config.properties --disable-token-auth  --enable-model-api

Step 5: Run inference

python query.py --url "http://localhost:8080/predictions/stable-diffusion" --prompt "a photo of an astronaut riding a horse on mars"

The image generated will be written to a file output-20221027213010.jpg.

NOTE: For KServe implementation use the below inputs for v1 and v2 protocols. Kserve v1 protocol - sample_v1.json Kserve v2 protocol - sample_v2.json