ComfyUI Node: Quantization Config Node

Authored by Big-Idea-Technology

Created 7 months ago

Updated 3 months ago

55 stars

Category

LLM

Inputs

quantization_mode

• none
• load_in_8bit
• load_in_4bit

llm_int8_threshold FLOAT

llm_int8_skip_modules STRING

llm_int8_enable_fp32_cpu_offload BOOLEAN

llm_int8_has_fp16_weight BOOLEAN

bnb_4bit_compute_dtype STRING

bnb_4bit_quant_type STRING

bnb_4bit_use_double_quant BOOLEAN

bnb_4bit_quant_storage STRING

Outputs

QUANTIZATIONCONFIG

Extension: LLM Node for ComfyUI

The LLM_Node enhances ComfyUI by integrating advanced language model capabilities, enabling a wide range of NLP tasks such as text generation, content summarization, question answering, and more. This flexibility is powered by various transformer model architectures from the transformers library, allowing for the deployment of models like T5, GPT-2, and others based on your project's needs.

Authored by Big-Idea-Technology