bk99.de entertain the web since 1997

bk99.de
HF · TrainingCurated

Parameter-efficient fine-tuning with PEFT

Summary

PEFT bundles LoRA and related methods that train only a small number of additional parameters. This makes large base models adaptable on limited hardware. PEFT supports several parameter-efficient adaptation methods.

Ideas

  • Many tasks do not require changing all of a model’s weights.
  • Separate adapters allow several specialisations on the same base model.

Insights

  • Scalable training needs reproducible transitions between code, data, devices and checkpoints.

Facts

  • With LoRA, the base model weights stay frozen.

Critique

  • Fewer trained parameters guarantee neither a small amount of data nor robust generalisation.

Recommendations

  • Compare adapter training with prompting and full fine-tuning on the same test data.

References

Read the original article on Hugging Face

Search the Web Archive

HF · 2023

Training LLaMA with RLHF

StackLLaMA combines supervised fine-tuning, a reward model and PPO into a complete RLHF pipeline. The guide shows how PEFT and quantisation lower the hardware requirements. The pipeline first uses instruction data for supervised fine-tuning.

HF · 2021

Simplifying distributed training with Accelerate

Accelerate encapsulates device selection, mixed precision and distributed training behind a small interface. Existing PyTorch code is meant to scale without a complete framework rebuild. Accelerate supports CPU, single and multiple GPUs as well as TPU configurations.

HF · 2024

Switching between DeepSpeed and FSDP

Accelerate unifies configurations for DeepSpeed and PyTorch FSDP so that training code stays largely stable. The article compares the concepts and migration paths of both sharding systems. DeepSpeed ZeRO and FSDP distribute model states across several devices.