bk99.de entertain the web since 1997

bk99.de
HF · TrainingCurated

Merging LoRA adapters

Summary

PEFT adds methods for combining several trained adapters. This allows specialisations to be mixed without training a complete model from scratch each time. PEFT supports several merge methods for LoRA adapters.

Ideas

  • Adapter merging treats learned changes as building blocks that can be combined.
  • Weighting several adapters is an optimisation problem in itself.

Insights

  • Scalable training needs reproducible transitions between code, data, devices and checkpoints.

Facts

  • Combinations can be created without a new complete fine-tuning run.

Critique

  • Merged abilities can interfere with each other or reinforce each other unexpectedly.

Recommendations

  • Evaluate each adapter on its own and the mixture with conflict and safety tests.

References

Read the original article on Hugging Face

Search the Web Archive

HF · 2024

Switching between DeepSpeed and FSDP

Accelerate unifies configurations for DeepSpeed and PyTorch FSDP so that training code stays largely stable. The article compares the concepts and migration paths of both sharding systems. DeepSpeed ZeRO and FSDP distribute model states across several devices.

HF · 2023

Training LLaMA with RLHF

StackLLaMA combines supervised fine-tuning, a reward model and PPO into a complete RLHF pipeline. The guide shows how PEFT and quantisation lower the hardware requirements. The pipeline first uses instruction data for supervised fine-tuning.

HF · 2023

Parameter-efficient fine-tuning with PEFT

PEFT bundles LoRA and related methods that train only a small number of additional parameters. This makes large base models adaptable on limited hardware. PEFT supports several parameter-efficient adaptation methods.