Any tips for full-finetuning SD3.5 Medium? #1333
Replies: 1 comment 8 replies
-
|
it is sd35m's fault tbh it is too few parameters to be easily finetuned. needs a cluster of gpus |
Beta Was this translation helpful? Give feedback.
-
|
Ok, I’ll look into PixArt. What would you say about SD3.5 Large Turbo? Also keep in mind my entire dataset is pretty much pixel art characters in a UV format. |
Beta Was this translation helpful? Give feedback.
-
|
pixel art has its own challenges to train and i'm not sure a 16ch VAE even fixes them you will definitely struggle with the 4ch VAE though in general having clean pixel groups that don't change randomly in size is not. really doable. with latent diffusion models (LDMs) |
Beta Was this translation helpful? Give feedback.
-
|
and Turbo / Lightning models are not tunable, unfortunately. |
Beta Was this translation helpful? Give feedback.
-
|
Ok, I usually do just downscale the image by 16x “nearest” to make it pixel perfect. At the moment I’m trying to make Minecraft skins so all I really want is just a real fast model that I can get looking and producing how I want. I’ll try pixart, is it supported by simpletuner? lol. |
Beta Was this translation helpful? Give feedback.
-
|
yes: https://github.com/bghira/SimpleTuner/blob/main/documentation/quickstart/SIGMA.md |
Beta Was this translation helpful? Give feedback.
-
I recently tried to train SD3.5 Medium on a dataset that worked for SDXL (I didn't train SDXL with SimpleTuner) but haven't gotten as great results. The adherence and accuracy while trained on the same dataset are fairly lacking, I'm not sure if it's my validation settings or the model just doesn't get it. I tried to keep my settings similar to the previous training hoping that maybe that would help it out however it didn't. Keep in mind I'm also not very knowledgeable when it comes to fine-tuning and SimpleTuner so maybe it's just me doing something wrong. I have my config.json and multidatabackend.json ready if needed though just some simple guidance and tips would be much appreciated if someone has gotten something good from training this model. I've probably started over 60 trainings and still none are great. Thank you in advance!
Edit: I'm currently testing all combinations of Samplers x Schedulars x cfg x SD3Guidance in comfyui right now to see if it's just a perfect setting that'll do the trick lol. I never really could find an optimal combination for the base model let alone my fine-tuned one so it could potentially just be an issue with my validations.
Beta Was this translation helpful? Give feedback.
All reactions