bk99.de entertain the web since 1997

bk99.de
Own project · Email & self-hostingOwn post

120,314 emails from 26 years: My mailbox moves into its own archive

Summary

In September 2026 I merged Outlook, Thunderbird, Evolution and Cyrus mail from 26 years into a local mail archive running Stalwart and Roundcube. In the end there were 120,314 searchable emails, including more than 11,000 that had long been considered deleted. This post shows why Message-IDs do not detect duplicates, what Thunderbird really does when you delete something and how an archive can stay safe without any cloud.

A quarter of a century in old files

For decades my email was scattered: Outlook files in PST, OST and NST format, several Thunderbird profiles, an Evolution mailbox and Maildir data from an old Cyrus server. Together that was 2,221 original files with 23.3 GB. Each of them needed its own program.

My goal was a single archive that I could search in the browser like a normal mailbox. It should only run at home, without a cloud and without an external search service.

The setup: two containers and clear boundaries

The archive consists of two services on my desktop: Stalwart as a mail server with built-in full-text search, and Roundcube as the web interface. Both sit in an internal Docker network without their own ports to the outside. Stalwart has no SMTP listener, and the archive user is not allowed to send mail. Roundcube hides all sending functions.

As a result, the archive can neither receive nor send mail. It is a pure reading room, and that is exactly what an archive should be.

What you can take away: An archive is not a mailbox. Switch off everything it does not need, above all receiving and sending. What does not listen cannot be attacked.

Why the Message-ID does not find duplicates

The same email often existed several times, for example in the Outlook inbox and in an old Thunderbird copy at the same time. The obvious idea is to detect duplicates by their Message-ID. That is unreliable, though: some programs assign the same ID twice, some emails have none at all, and identical IDs can carry different content.

So I calculated a checksum over the entire email content. Before that, line endings are normalised and local status lines that every program writes differently are removed. In a single delivery, 22,868 email occurrences turned into 15,746 distinct emails this way; the rest were copies.

What you can take away: Detect duplicates by their content, not by an identifier some program assigned. And normalise first whatever changes between programs without the email itself changing.

Deleted does not mean deleted in Thunderbird

The biggest surprise was in the Thunderbird files. Thunderbird stores a folder as one large text file. When you delete an email, it is only marked as deleted there. It is really gone only once the folder is compacted.

My old profiles contained 11,588 emails that I had long thought were deleted. I imported them, but kept them separately in a folder called “Recovered”, so they would not get mixed into my actual filing.

What you can take away: If you pass on an old Thunderbird profile, you may also pass on deleted emails. Compact the folders first. And if you are looking for old emails: an uncompacted profile is a small treasure trove.

When the source itself has gaps

Not every file kept its promise. In an old Outlook PST, 69 of 70 messages were stored without text and without a date. In the archive they first appeared with the import date. Only a look at the internal Outlook timestamps with a second tool revealed the real dates from 2002 and 2003. The missing texts cannot be recovered from this file.

What you can take away: Keep the original files, even after the import is done. An import date is not a historical date, and no tool can later invent what is missing from the source.

Safe, but not in the cloud

The archive is not supposed to leave my house. The backup therefore goes to an encrypted drive on the same computer. It contains emails, attachments, folders, flags, configuration and the tools needed to rebuild everything. I restored every backup into an empty directory and checked it there: number of emails, folders, search and byte-identical samples.

By the way, a later delivery with 18,161 new emails was imported in 38 seconds. The real work was never in copying, but in checking.

What you can take away: A backup you have never restored is a hope. Test the restore before you need it. Also keep in mind: a backup on the same computer protects against mistakes, not against fire or theft.

Numbers at a glance

  • The archive contains 120,314 emails from the years 2000 to 2026.
  • The original sources consisted of 2,221 files with a total of 23.3 GB.
  • 11,588 of the imported emails were marked as deleted in Thunderbird but were still present in the folder files.
  • Stalwart and Roundcube run without an SMTP listener, without sending rights and without any cloud service.

References

Remarks

  • The contents of individual emails were not analysed for this post; all numbers come from import and verification reports.

Visit Stalwart

CCC · 2026

Self-hosting email: DNS, spam filters and reputation

davd delivers a reality check on running your own mail server and explains why DNS, deliverability, spam defence and reputation are harder than the actual server software. With email, self-hosting fails more often because of social trust than because of SMTP. Decentralisation only survives if small operators can achieve deliverability in practice.

Own project · 2026

6,594 emails from my servers: What 20 years of robot mail tell

While building my mail archive, I found a folder with 6,594 emails that servers I was responsible for sent me between 2004 and 2025. Logwatch reports, cron output and warnings show how my server operations changed over twenty years. This post shows what this robot mail reveals, when it became a burden and how I would approach monitoring by email today.

Own project · 2026

Mail from the ISS: SSTV images with a Raspberry Pi and a magnetic-base antenna

From 2 to 6 October 2026, the International Space Station is transmitting twelve images via SSTV on 437.550 MHz as part of ARISS Series 33. I received them in Meerane with a Raspberry Pi, an RTL-SDR stick, a simple magnetic-base antenna and OpenWebRX+, and on the first usable pass two images came through cleanly. The deciding factor was not the antenna, but a small script that tracks the frequency during the pass and saves the images itself.