Block I/O, Buffer Cache, and Writeback

By the end of this lesson, you will be able to...

• Trace a file write from byte offset to cached block, dirty state, and device submission.

• Distinguish a write that is visible to readers from one that is durable after a crash.

• Diagnose unsafe eviction, stale reads, and an incomplete writeback path.

Idea in one sentence: A buffer cache gives the kernel one shared, mutable copy of a disk block; dirty state records that the copy is newer than storage until writeback makes it durable.

logger already has an open-file object for /var/log/app.log from lesson 010. It calls:

write(fd, "OK\n", 3);

Suppose its current file offset is 1026 and the teaching kernel uses 1024-byte blocks. The bytes do not go directly from the user buffer to “the file.” First the VFS routes the request to the inode. The filesystem maps the byte range to a logical block within that file, then to a physical block on the device. The buffer cache finds or creates the in-memory copy for that physical block. Only then can the kernel modify three bytes.

The difficult question arrives after write returns: if another process reads the same file now, should it see OK\n? Usually yes. If power fails now, must the bytes survive reboot? Not necessarily. A cache can make a write visible before it is durable. That gap is useful for performance, but it is also a contract the kernel must state and protect.

Core Insight

The small state machine for a cached block is more useful than the word “cache”:

not cached
  -> present and clean
  -> present and dirty
  -> writeback in flight
  -> present and clean

Plain meaning:

The buffer cache is the kernel's shared notebook for one disk block. A dirty page in that notebook contains the latest agreed edit, while the disk may still have yesterday's page.

In this scenario:

The three bytes at offsets 1026..1028 are part of file logical block 1, at in-block offsets 2..4. The cache buffer for whichever device block backs logical block 1 becomes the authoritative in-memory version for later readers.

Technical name:

This is a buffer cache or block cache. A block with modifications not yet installed on persistent media is dirty. Moving dirty contents to the device is writeback. A completed writeback is a durability event only when the storage stack's ordering and flush guarantees say it is; a mere CPU copy is not persistence.

Why a Cache Is Also a Correctness Boundary

The naive design lets each reader or writer allocate its own temporary block buffer:

writer A reads sector 42 into private buffer A
writer B reads sector 42 into private buffer B
A changes bytes 2..4 and writes A
B changes bytes 500..503 and writes B

If B writes last, its older private copy can erase A's new bytes. Reads can also disagree: a process that rereads through a new buffer may see an older device copy even though another process just received a successful write result.

A buffer cache maintains a central invariant:

For each device and block number, there is at most one cached buffer representing its current in-memory contents.

The exact implementation can use a hash table plus an eviction list, or a simpler fixed table in a teaching kernel. The important part is not the data structure. It is that lookup, buffer ownership, dirty marking, and eviction all respect the “one current copy” invariant. The locks and reference rules from lesson 009 protect that state; the ownership rules from lesson 005 explain why an in-use or transaction-pinned buffer cannot simply be reused.

The Moving Parts

The cache does not replace the filesystem's mapping. It stores device blocks, while the inode says which blocks belong to which byte range in a file. Lesson 010 separated the open-file offset from the inode; this lesson separates logical file layout from one current copy of a device block.

Worked Trace: a write becomes dirty, then durable

Assume fd is writable, its current offset is 1026, block size is 1024, and the inode maps logical file block 1 to device block 42. The process writes OK\n.

Input: write(fd, user_buf, 3) with bytes O, K, newline.

1. The VFS validates the open and finds the byte range

The syscall handler validates the user range before copying it, as in lesson 008. The VFS checks fd and the open-file mode, then asks the inode layer to write at offset 1026.

file offset = 1026
logical block = floor(1026 / 1024) = 1
in-block offset = 1026 mod 1024 = 2
range in block 1 = bytes 2, 3, 4

The inode mapping returns device block 42. If the file must grow, the mapping may allocate a new data block and update metadata; those metadata changes have their own ordering requirements.

2. The cache returns the one buffer for block 42

The kernel looks up (device0, 42). On a cache miss it reserves a reusable clean and unpinned buffer, labels it as block 42, and asks the driver to read it before modifying unrelated bytes. On a hit it waits or locks as needed so that no second caller edits a competing copy.

Intermediate state: one buffer represents block 42; it is locked or referenced by this operation. It is clean if its bytes match storage, or already dirty if an earlier writer changed it.

3. The write changes the cache before storage

The inode layer copies OK\n into positions 2..4 of the buffer. It records the data-length or inode metadata change that the write requires, marks the relevant buffers dirty, and advances this open-file object's offset from 1026 to 1029 after the accepted byte count is known.

buffer 42: clean -> dirty
cache bytes 2..4: old -> "OK\n"
device block 42: still old contents

At this point a later reader that uses the same cache can observe the new bytes. The process may receive return value 3 if the kernel's write contract accepts buffered writes. That return value says the kernel accepted the bytes into its managed state; it does not automatically say the bytes survive a power loss.

4. Writeback submits the dirty buffer

A synchronous request, memory pressure, a background policy, or an explicit durability operation may select buffer 42 for writeback. The cache keeps it referenced or marks it in flight, and the block layer gives the driver a request for device block 42.

The driver programs the device and later receives a completion signal. On successful completion, the kernel can mark buffer 42 clean only if no later writer dirtied it again while the earlier write was in flight. If a second write occurred, it remains dirty and needs another pass.

Output/decision: after the required data and metadata writes complete in the promised order, the kernel may report durability for an explicit sync-style request. If the device reports an error, the buffer cannot be quietly treated as clean; the error must reach the policy and, where applicable, the caller.

5. Crash contrast: visible is not durable

If power fails after step 3 but before the relevant writes complete, the cache disappears. After reboot, device block 42 can contain its old bytes. If the write also extended a file, data and metadata may disagree unless the filesystem uses a disciplined ordering or a journal/transaction protocol.

Naive contrast: writing each caller's private temporary buffer directly to the device loses the single-current-copy invariant. Marking a buffer clean immediately after issuing I/O can claim durability before completion. Evicting a dirty buffer without flushing it discards accepted data.

So far, we have seen a complete path: user bytes become a range, the range maps to a logical and then device block, one cache buffer becomes dirty, and writeback plus device completion determines when the storage copy catches up. The cache is fast because it avoids unnecessary I/O, but it is correct only because later readers and writers agree on the same buffer state.

Visibility, Durability, and Ordering

Three statements that sound similar are different:

A small kernel can choose simple synchronous writes: do not return until the device confirms the block. That narrows the visibility/durability gap but makes each write slow. A write-back cache batches work and improves throughput, but exposes a window in which a crash loses recent changes. The trade-off is explicit: buffering reduces device waits and enables coalescing, while it costs memory, eviction policy, error handling, and a clear durability contract.

Ordering adds another limit. Updating a file can touch both a data block and an inode or directory block. If the system writes metadata that exposes a new file length before its data is safe, a crash can leave a length that points at uninitialized or old content. A logging or journaling design records groups of metadata changes so recovery can decide whether to replay a complete transaction. xv6's log pins modified cache buffers until commit and uses a commit point before installing them at their final locations; that is one small example of turning writeback order into a crash-consistency protocol.

This lesson does not prove a production filesystem recovery algorithm. It gives the boundary: dirty cache state alone provides coherence in memory, not atomic crash recovery.

Common Confusions

Confusion: “A cache is only a performance optimization.”

Why it is tempting:

The first visible benefit is fewer device reads.

Better model:

One shared buffer per block also makes concurrent reads and writes observe a coherent current copy. Removing it requires another protocol that preserves the same invariant.

Confusion: “write returning success means the data is on disk.”

Why it is tempting:

The program gave bytes to an operation called write.

Better model:

Success may mean buffered acceptance. Durability depends on the API's sync guarantees, writeback completion, device semantics, and metadata ordering.

Confusion: “A dirty buffer may be evicted because it is already in memory.”

Why it is tempting:

Eviction is often described as making room in cache.

Better model:

Dirty means memory is the only current copy. The buffer must be retained, written back, or incorporated into a protected transaction before reuse.

Confusion: “A device completion makes every earlier filesystem change safe.”

Why it is tempting:

One request completed successfully.

Better model:

Durability may require several data and metadata writes in a specific order. One block completion does not establish a whole-file atomic update.

Check: Buffer 42 is dirty and has no active callers. Memory pressure needs a reusable cache buffer. What must happen before buffer 42 can represent block 99?

Think first, then reveal.

Answer: The kernel must preserve or flush the dirty contents and wait for the required completion/error outcome before recycling the buffer. Reusing it immediately loses the only current copy of block 42.

Check: A read immediately after a successful buffered write sees the new bytes. A power loss one millisecond later loses them. Is this necessarily a cache-coherence bug?

Think first, then reveal.

Answer: No. The cache behaved coherently: readers saw the one dirty in-memory copy. The missing property is durability, which requires a completed and correctly ordered persistence path.

Practice: audit a write path

Review this design claim: “On write, copy into a cached block, set dirty = true, return success, and let the eviction code discard the block whenever cache space is low.”

Write a short review containing:

1. the state transition that is correct;
2. the invariant the design violates;
3. the minimum safe eviction/writeback transition;
4. what write can honestly promise; and
5. one additional requirement if the write changes both file data and inode metadata.

Use this rubric:

Resources

• [BOOK] MIT 6.1810 xv6 book, File system and logging — Focus: follow buffer-cache ownership, logging, commit, and recovery as one small implementation path.
• [COURSE] MIT 6.1810 xv6 filesystem reading — Focus: inspect bio.c, fs.c, file.c, and sysfile.c to separate mapping, cache, and VFS responsibilities.
• [COURSE] MIT 6.1810 xv6 logging reading — Focus: trace how modified cache blocks enter the log and reach the commit point.

Key Takeaways

1. A buffer cache owns the coherent in-memory copy of each device block, not merely a convenient duplicate.
2. Dirty state means the cache is newer than persistent storage; it must not be silently recycled.
3. Accepted, visible, and durable writes are distinct milestones with different evidence.
4. Writeback needs completion, error, and ordering rules, especially when one operation changes both data and metadata.