ComfyOnline
ComfyUI Flux Accelerator

🍭 ComfyUI Flux Accelerator

Note 日本語のREADMEはこちらです。

ComfyUI Flux Accelerator is a custom node for ComfyUI that accelerates Flux.1 image generation, just by using this node.

<p align="center"> <img src="./assets/sample.gif" width=40%> </p>

How does ComfyUI Flux Accelerator work?

ComfyUI Flux Accelerator accelerates the generation of images by:

  1. Using TAEF1.

    TAEF1 is a fast and efficient AutoEncoder that can encode and decode pixels in a very short time, in exchange for a little bit of quality.

  2. Quantization and Compilation.

    ComfyUI Flux Accelerator utilizes torchao and torch.compile() to optimize the model and make it faster.

  3. Skipping redundant DiT blocks.

    ComfyUI Flux Accelerator offers an option to skip redundant DiT blocks, which directly affects the speed of the generation.

    You can choose the number of blocks to skip in the node (default is 3, 12 of MMDiT blocks).

How much faster is ComfyUI Flux Accelerator?

ComfyUI Flux Accelerator can generate images up to 37.25% faster than the default settings.

Here are some examples (tested on RTX 4090):

512x512 4steps: 0.51s → 0.32s (37.25% faster)

<p align="center"> <img src="./assets/512x512_4steps.png" width=80%> </p>

1024x1024 4steps: 1.94s → 1.24s (36.08% faster)

<p align="center"> <img src="./assets/1024x1024_4steps.png" width=80%> </p>

1024x1024 20steps: 8.77s → 5.74s (34.55% faster)

<p align="center"> <img src="./assets/1024x1024_20steps.png" width=80%> </p>

How to install ComfyUI Flux Accelerator?

  1. Clone this repository and place it in the custom_nodes folder of ComfyUI

    git clone https://github.com/discus0434/comfyui-flux-accelerator.git
mv comfyui-flux-accelerator custom_nodes/

  2. Install PyTorch and xFormers

    ## Copied and modified https://github.com/facebookresearch/xformers/blob/main/README.md

# cuda 11.8 version
pip3 install -U torch torchvision torchao triton xformers --index-url https://download.pytorch.org/whl/cu118
# cuda 12.1 version
pip3 install -U torch torchvision torchao triton xformers --index-url https://download.pytorch.org/whl/cu121
# cuda 12.4 version
pip3 install -U torch torchvision torchao triton xformers --index-url https://download.pytorch.org/whl/cu124

  3. Download TAEF1 with the following command

    cd custom_nodes/comfyui-flux-accelerator
chmod +x scripts/download_taef1.sh
./scripts/download_taef1.sh

  4. Launch ComfyUI

    Launch command may vary depending on your environment.

    a. If you have H100, L40 or more newer GPU

    python main.py --fast --highvram --disable-cuda-malloc

    b. If you have RTX 4090

    python main.py --fast --highvram

    c. Otherwise

    python main.py

  5. Load the workflow in the workflow folder

    You can load the workflow by clicking the Load button in the ComfyUI.

  6. Enjoy!

How to use ComfyUI Flux Accelerator?

Just use the FluxAccelerator node in the workflow, and you're good to go!

If your GPU has less than 24GB VRAM, you may encounter frequent Out Of Memory errors when changing parameters. But simply ignore them and run again and it will work!

What are the limitations of ComfyUI Flux Accelerator?

ComfyUI Flux Accelerator has the following limitations:

  1. Image Quality

    ComfyUI Flux Accelerator sacrifices a little bit of quality for speed by using TAEF1 and skipping redundant DiT layers. If you need high-quality images, you may want to use the default settings.

  2. Compilation Time

    ComfyUI Flux Accelerator may take 30-60 seconds to compile the model for the first time. This is because it uses torch.compile() to optimize the model.

  3. Compatibility

    ComfyUI Flux Accelerator is now only compatible with Linux.

License

ComfyUI Flux Accelerator is licensed under the MIT License. See LICENSE for more information.