Hi Folks, which config approach is best for majority of use cases OR which approach we should try the first or in what order

[lirilkumar]

Hi Team,
Thanks for amazing package!.

Q1: in case we need to start using PEFT, which approach (Lora, prefix etc) is suitable or works most of the time and what will be order of these approaches according to their performance (all 7-8 approaches order i mean)

Q2: are we planning to add merge_and_unload method for prompt tuning approaches like 'prefix tuning'.

Thanks in advance.

[BenjaminBossan]

1. There really is no answer to that question, it depends on too many variables. If trying out multiple methods is too costly for your task, I'd suggest doing a literature review and check for tasks that are as close to the one you're trying to solve as possible.
2. I don't think it's technically possible to implement merge_and_unload for prefix tuning.

[lirilkumar]

Thanks @BenjaminBossan for quick help.

in case we want to merge prefix_tuning adapter weights with base model weights like it's supported in LORA, how can we achieve this (as we want to save this as a single model (merged weights) and not separately as model,adapter)

goal is to host tuned model pipeline via VLLM

[BenjaminBossan]

Honestly, I wouldn't know how to merge prefix tuning weights, as it essentially extends the input prompt by adding new vectors to the start of the sequence of encoded tokens. I'm not aware of any method that would allow to merge those into the existing weights and be equivalent. If you have any suggestion, let me know.