The idea: RAM is fast, the disk survives
A file system keeps two copies of everything. The kernel works on a copy in RAM (the page cache and the inode and dentry caches), because the disk is thousands of times slower. The copy on disk is the only one that survives a power cut. Most of what makes file systems interesting is the gap between the two: when RAM changes reach the disk, in what order, and what a crash in the middle leaves behind.
The canvas models a small ext4 file system: one directory /app with config.json (inode 13, data block 200, content "v1"). Pick a system call and press Do It, arm a crash inside the next journal commit with the crash point select, or press Crash Now. Every crash is followed by a reboot that runs journal recovery.
Path lookup: names, directory entries and inodes
A file has no name inside it. Its inode holds the type, size, link count, times and the numbers of its data blocks. Names live in directories, which are just files of entries name → inode number. To open /app/config.json the kernel starts at the root inode (2), reads its directory block, finds app → 12, reads inode 12 and its block, finds config.json → 13, and reads inode 13. The dentry cache remembers each answer (even "no such name"), and the inode cache keeps the inodes, so the next lookup needs no disk at all (Demo 1: 4 disk reads the first time, since the root directory is cached at mount, then a page cache hit on the second read).
write() and the page cache
write() copies the bytes into a page of the page cache, marks it dirty and returns. Nothing has reached the disk. Later the flusher thread writes dirty pages that are older than dirty_expire_centisecs (30 s by default), or sooner under memory pressure. A crash before that loses the write, even though the program saw it succeed (Demo 2). Wait 30 s and the flusher does write it (Demo 8), but nothing tells the program when that happened.
How read() copies out of the page cache while mmap() maps the same page into a process is on The Page Cache: read() vs mmap().
fsync()
fsync(fd) does not return until the file's data and its metadata are on stable storage: it writes the dirty pages, commits the journal transaction that holds the inode, and flushes the drive's cache. After it returns, a crash cannot lose that data (Demo 3). It is the only promise POSIX gives you. Databases call it on every commit (see WAL and crash recovery), which is why fast fsync matters so much.
How the dirty pages travel from the page cache through the bio, blk-mq queues and the NVMe driver to the disk is on The Linux Block Layer: How a write() Reaches the Disk.
The journal
One change, such as creating a file, touches several blocks: the new inode, the directory, the free-space bitmaps. If a crash wrote some of them and not others, the file system would be corrupt. ext4's journal (jbd2) makes metadata changes all-or-nothing. Changes collect in a running transaction; every 5 s, or on fsync, it is committed:
- (ordered mode) the data blocks those inodes point to are written to their home location;
- a descriptor block and a copy of every changed metadata block are written to the journal;
- after a flush, a commit block is written. Only now does the transaction count.
Later a checkpoint copies the metadata to its home location and frees the journal space. On mount after a crash, recovery replays every committed transaction and ignores one without a commit block.
Demo 4 crashes after the journal blocks but before the commit block. The transaction is dropped, so the inode still says the file is 2 bytes long. But in ordered mode the data is not journaled: block 200 was already overwritten in place with "version 2". After the reboot the file reads "ve", the old size over the new bytes. The metadata is consistent; the content is torn. ext4's data=journal mode would journal the data too, at the cost of writing it twice.
Delayed allocation and the rename trap
To replace a file atomically, programs write a new file and rename() it over the old one: readers see the old file or the new one, never half. But rename only says something about names. With delayed allocation the new file has no data block until writeback. If the journal commits the rename first and the machine crashes before writeback, the name points at an inode with size 0: the old content is gone and the new one never arrived (Demo 5). The safe sequence is:
fd = open("config.json.tmp", O_WRONLY|O_CREAT|O_TRUNC)
write(fd, data); fsync(fd); close(fd) // the data is on disk
rename("config.json.tmp", "config.json")
dfd = open("/app", O_DIRECTORY); fsync(dfd) // the new name is on disk
(Demo 6.) ext4 has a heuristic (auto_da_alloc) that forces the data out when a rename replaces an existing file, added after this bit many programs in 2009. The page models it switched off, as other file systems and POSIX do not promise it.
Hard links, symlinks and unlink
| hard link | symbolic link | |
|---|---|---|
| what it is | another directory entry for the same inode | its own inode whose content is a path |
| link count | raises nlink | does not change the target |
| target deleted | the data stays while any name is left | dangling: open() gives ENOENT |
| across file systems / to directories | no | yes |
unlink() removes a name and lowers the link count. The inode is freed only when nlink is 0 and no process has it open. An open file with no names left is put on the orphan list; it stays readable through its fd and is freed at close(), or by recovery after a crash (Demo 7). That is why deleting a big log file that a server still writes does not free any space.
See also System calls for how read and write enter the kernel, DMA for how a block reaches RAM, and Virtual memory: the page cache is made of the same pages. What open() Does follows the same lookup through the VFS name by name, with permission checks, mount points and the struct file that open() returns. How the kernel drops clean page-cache pages and writes back dirty ones under memory pressure: Linux Out of Memory.
Below the block device, How a Hard Disk Reads and Writes Data shows the drive's write cache and why fsync() ends with FLUSH CACHE or FUA, and How an SSD Reads and Writes Data follows each block write into the flash translation layer and shows why TRIM (fstrim, the discard mount option) makes garbage collection cheaper.
What the page leaves out
Extents and block bitmaps (a file here has one block), multi-block files and partial pages, O_DIRECT and O_SYNC, fdatasync, the drive's own write cache and FUA, data=writeback and data=journal modes, the dentry and inode cache eviction, permissions and timestamps, copy-on-write file systems (btrfs, ZFS) that never overwrite in place, and fsck.