Maildir

Maildir
Original uploader was GVdSteen at nl.wikipedia (Original text : GVdSteen) · CC BY-SA 3.0 · source
Name	Maildir
Introduced	1995
Developer	D. J. Bernstein
Type	Email mailbox format
Container for	Email messages

Maildir Maildir is a filesystem-based email mailbox format designed to store each message in a separate file within a structured directory layout. It was introduced to provide atomic delivery and reliable concurrent access on Unix-like systems, addressing issues encountered with mbox-style files used by projects such as Sendmail, Postfix, and Exim. The format influenced numerous mail delivery agents and clients, and it remains relevant in server deployments, integration projects, and archival workflows involving software like Dovecot, Courier Mail Server, Qmail, OpenSMTPD, and Procmail.

History

Maildir was published by Daniel J. Bernstein in the mid-1990s as part of work on qmail and related mail-handling tools. The design responded to concurrency and corruption problems experienced in popular systems such as Sendmail and MH (mail handler), and it drew attention from administrators of University of California, Berkeley-based systems, commercial providers like Yahoo!, and open-source communities including Debian, Red Hat, and Gentoo. Adoption accelerated as projects like Courier, Dovecot, and Qmail incorporated Maildir support. Over time, standardization efforts and community discussions involving organizations such as the Internet Engineering Task Force and contributors on lists like Linux Kernel Mailing List shaped best practices for portability, locking, and namespace conventions.

Format and components

The Maildir layout uses a top-level directory per mailbox containing three subdirectories: new, cur, and tmp. This arrangement echoes earlier mailbox structures used in systems like Plan 9 and tools from GNU Project, but it is distinct in its per-message file approach. Messages in new represent newly delivered mail; cur contains read or processed messages with optional status information; tmp holds files during delivery to ensure atomicity. Filenames typically encode unique identifiers derived from process IDs, timestamps, hostnames, and delivery agent information; implementations may borrow techniques from POSIX file semantics, Unix inode behaviour, and locking strategies used by Berkeley DB or SQLite-backed systems. Metadata such as flags (seen, replied-to, flagged) are represented in filename suffixes or companion state files, enabling clients like Mutt, Alpine, Thunderbird, and Evolution to present per-message status without centralized databases.

Delivery and access mechanisms

Mail delivery agents (MDAs) such as Procmail, Dovecot's doveadm, Postfix local delivery agent, and Exim write messages into the tmp directory and atomically move them into new. This leverages atomic operations provided by filesystems like ext4, XFS, ZFS, and Btrfs. Mail user agents (MUAs) and IMAP servers such as Dovecot and Courier IMAP scan new and cur to populate mailboxes for clients that connect via protocols like IMAP and POP3. Concurrency control is achieved without global locks by using filesystem semantics; some deployments enhance safety with locking helpers from libc-based APIs, flock utilities, or by delegating access to a single authoritative daemon as in Dovecot or Courier. Integration with system services such as systemd or container platforms like Docker may influence delivery timing, file permission handling, and resource limits.

Variants and extensions

Multiple extensions and conventions have arisen to accommodate features not specified in the original design. The Maildir++ extension introduced conventions for nested mailboxes and quota support used by projects like Dovecot and hosting providers including FastMail and Rackspace. Some deployments adopt modified filename encodings to carry Message-ID or UID information, similar to approaches in Cyrus IMAP and proprietary systems from vendors such as Microsoft in certain migration tooling. Third-party adaptations exist for distributed filesystems (for example, deployments over NFS, Ceph, or GlusterFS), and for cloud storage integrations common to providers like Amazon Web Services and Google Cloud Platform; these often require additional consistency layers or daemonized mediators to preserve atomicity.

Advantages and limitations

Advantages of the Maildir model include per-message atomicity, resilience to corruption relative to single-file formats used by mbox and early UW-IMAP setups, and straightforward backup and restore operations compatible with staging tools from rsync and tar. The format permits parallel delivery without complex locking schemes and supports easy inspection by system administrators using tools such as grep, awk, and sed. Limitations include performance overhead when handling large numbers of small files on filesystems with poor directory scaling (issues observed on older ext3 and some network filesystems like NFSv3), metadata inconsistency when using non-POSIX storage backends, and complications for features like server-side threading, message indexing, and global flag synchronization found in systems like Exchange Server or Cyrus IMAP. Operational concerns include inode exhaustion, backup chattiness, and the need for additional tooling for quotas, trash folders, and per-user mail aggregation.

Implementations and adoption

Maildir support is widespread across open-source and commercial software. Servers and agents with native support include qmail, Postfix, Exim, Dovecot, Courier, OpenSMTPD, and Zimbra (in certain configurations). Clients and utilities interoperable with Maildir include Mutt, Alpine, Thunderbird (via plugins), Evolution, and indexing tools like Notmuch. Hosting providers, academic institutions such as MIT, and Linux distributions including Debian and Ubuntu have recommended or provided Maildir-based setups for scalable mail services. Migration tools and converters exist between Maildir and mbox or proprietary stores, and projects in mail archiving, forensics, and compliance leverage Maildir’s per-message files for simplified ingestion by systems like Archivematica and Violet-style archival pipelines.

Category:Email