ComfyUI-B-LoRA
A ComfyUI custom node that loads and applies B-LoRA models.
What is B-LoRA?
B-LoRA: By implicitly decomposing a single image into its style and content representation captured by B-LoRA, we can perform high quality style-content mixing and even swapping the style and content between two stylized images.
-
š Website: https://b-lora.github.io/B-LoRA/
-
Code: https://github.com/yardenfren1996/B-LoRA/
-
Currently B-LoRA models only works with SDXL (
sdxl_base_1.0
). (Compatible but not guaranteed with SDXL-based fine-tuned models.)
Advantages of B-LoRA
-
Can apply
Style
orContent
, or both. -
Much smaller model files. (~100M for SDXL B-LoRAs)
-
One B-LoRA only needs one image as training dataset and 15 minutes to train. (on a single RTX 4090)
Please share your B-LoRA models on Civit.ai or HuggingFace!
Node
Load B-LoRA
<img src="nodes/load_b_lora.png" width="400" />-
lora_name
: Choose the B-LoRA model you want to load. By default, it'll searches in themodels/loras/
folder for available models. -
load_style
: Do you want the style of that B-LoRA? -
load_content
: Do you want to content of that B-LoRA? -
strength
: How strong do you want that B-LoRA to affect the model?
Workflow Examples
A Single Load B-LoRA node
š <s>
is the training prompt for one B-Lora colorful-squirrel
A B-LoRA for Style, and another for Content
š <s>
is the training prompt for one B-Lora colorful-squirrel
, and <p>
is the training prompt for the other pencil-boy
.
B-LoRA models used in the workflows can be downloaded here:
https://huggingface.co/sida/B-LoRA-examples/tree/main
More pretrained B-LoRAs to try out:
https://huggingface.co/lora-library?sort_models=downloads#models
Train Your B-LoRAs (WIP):
I'm building a docker image for training. Please check train to see current progress.
Credit goes to:
-
https://github.com/yardenfren1996/B-LoRA
-
https://github.com/huggingface/diffusers/blob/main/scripts/convert_diffusers_sdxl_lora_to_webui.py
-
https://github.com/yardenfren1996/B-LoRA/issues/7
-
https://github.com/comfyanonymous/ComfyUI/issues/3674
Citation
If you use B-LoRA in your research, please cite the authors' paper:
@misc{frenkel2024implicit,
title={Implicit Style-Content Separation using B-LoRA},
author={Yarden Frenkel and Yael Vinker and Ariel Shamir and Daniel Cohen-Or},
year={2024},
eprint={2403.14572},
archivePrefix={arXiv},
primaryClass={cs.CV}
}