ComfyUI Node: Load LLM Model Advanced

Authored by daniel-lewis-ab

Created 10 months ago

Updated 4 months ago

40 stars

Category

LLM

Inputs

Model

n_gpu_layers INT

split_mode

• LLAMA_SPLIT_NONE
• LLAMA_SPLIT_LAYER
• LLAMA_SPLIT_ROW

main_gpu INT

tensor_split FLOAT

vocab_only BOOLEAN

use_mmap BOOLEAN

use_mlock BOOLEAN

seed INT

n_ctx INT

n_batch INT

n_threads INT

n_threads_batch INT

rope_scaling_type

• LLAMA_ROPE_SCALING_UNSPECIFIED
• LLAMA_ROPE_SCALING_NONE
• LLAMA_ROPE_SCALING_LINEAR
• LLAMA_ROPE_SCALING_YARN

rope_freq_base FLOAT

rope_freq_scale FLOAT

yarn_ext_factor FLOAT

yarn_attn_factor FLOAT

yarn_beta_fast FLOAT

yarn_beta_slow FLOAT

yarn_orig_ctx INT

mul_mat_q INT

logits_all BOOLEAN

embedding BOOLEAN

offload_kqv BOOLEAN

last_n_tokens_size INT

lora_base STRING

lora_scale FLOAT

lora_path STRING

numa BOOLEAN

chat_format STRING

verbose BOOLEAN

Outputs

LLM

Extension: ComfyUI-Llama

This is a set of nodes to interact with llama-cpp-python

Authored by daniel-lewis-ab