bk99.de entertain the web since 1997

bk99.de
HF · ToolsCurated

Combining models with Gradio

Summary

Gradio 2.0 makes models interactively usable with a few lines of Python and allows chained demos. This lowers the hurdle for checking model behaviour visibly with real inputs. Gradio generates web interfaces directly from Python functions.

Ideas

  • A good demo shortens the path between model code and human feedback.
  • Chained models make interface errors between individual processing steps visible.

Insights

  • Low barriers to access speed up feedback but must not pretend operational maturity.

Facts

  • Several interfaces can be combined in series or in parallel.

Critique

  • An accessible interface can lead people to overestimate the maturity and reliability of an experimental model.

Recommendations

  • Only publish demos with input limits, error handling and clear labelling of their limits.

References

Read the original article on Hugging Face

Search the Web Archive

HF · 2024

Serving thirty LoRA models with one server

Text Generation Inference loads many LoRA adapters on top of a shared base model and serves them together. This avoids complete model copies and improves utilisation. The demonstration serves 30 LoRA models through one TGI instance.

HF · 2023

Adapting Stable Diffusion efficiently with LoRA

LoRA trains small additional matrices instead of all the weights of a diffusion model. This lowers memory requirements and produces compact adapters that can be distributed separately. LoRA freezes the original model weights.

HF · 2022

Loading very large models with Accelerate

Accelerate distributes model parts across GPU, CPU and disk instead of first building the whole model in memory. Empty initialisation and automatic device maps avoid unnecessary peaks. init_empty_weights first creates the model structure without parameter data.