Pipe Viewer makes invisible data streams visible
Summary
Peteris Krumins shows how pv displays progress, throughput and remaining time of any Unix pipeline without changing its data flow. It reads from standard input and writes to standard output. The program displays time, rate, amount and progress.
Ideas
- pv sits as a transparent process between source and destination.
- The tool counts transferred bytes while passing them on.
- Known total sizes allow a percentage and an estimated remaining time.
- Throughput displays reveal slow sections of a pipeline.
- Rate limits throttle data streams for tests or weak destinations.
- Several pv instances locate bottlenecks within long pipelines.
Insights
- Observability is especially valuable for simple, composed tools.
- A transparent measuring point improves diagnosis without changing the application.
- Progress displays are operational information, not mere convenience.
- Unix pipelines stay extensible because programs use standardised streams.
Facts
- pv stands for Pipe Viewer.
- A known input size improves the remaining time estimate.
Recommendations
- Use pv for long backup, compression and copy operations.
- Compare measuring points before and after compute-intensive filters.
- Do not rely on visible progress alone when restoring.
References
Links to the original source and the Web Archive open in a new tab.