Journaling file system
|
|
A journaling file system is a file system that logs changes to a journal (usually a circular log in a specially-allocated area) before actually writing them to the main file system.
Rationale
File systems tend to be very large data structures; updating them to reflect changes to files and directories usually requires many separate write operations. This introduces a race condition, in which an interruption (like a power failure or system crash) can leave data structures in an invalid intermediate state.
For example, deleting a file on a Unix file system involves two steps:
- removing its directory entry
- marking the file's inode as free space in the free space map
If step 1 occurs just before a crash, there will be an orphaned inode and hence a storage leak. On the other hand, if step 2 is performed first before the crash, the not-yet-deleted inode will be marked free and possibly be overwritten by something else.
One way to recover is to do a complete walk of the file system's structures when it is next mounted to detect and correct any inconsistencies. This can be very slow for large file systems, and is likely to become slower yet, given that the ratio of storage capacity to I/O bandwidth on modern computer systems is rising.
Another way to recover is to for the file system can keep a journal of the changes it intends to make, ahead of time. Recovery then simply involves a forward replay of changes until the file system is consistent again. In this sense, the changes are said to be atomic (or indivisible) in that they will have either:
- have succeeded originally
- be replayed completely during recovery
- not be replayed at all
Log-structured file systems are those which the journal is itself the entire filesystem. As of 2005, none of the most popular general-purpose filesystems are log-structured, although log-structured file system concepts influenced the development of WAFL and Reiser4.
Databases use more rigorous versions of the same journaling techniques to ensure data integrity.
Metadata-only journaling
Journaling can have a severe impact on performance because it requires that all data be written twice. Metadata-only journaling is a compromise between reliability and performance that stores only changes to file metadata (which is usually relatively small and hence less of a drain on performance) in the journal. This still ensures that the file system can recover quickly on next mount, but cannot guarantee complete consistency because unjournaled file data already committed to disk may out of sync with the metadata.
For example, when appending to a file on a Unix file system typically requires three steps:
- increasing the size of the file in its inode
- allocating space for the extension in the free space map
- actually writing the appended data to the newly-allocated space
In a metadata-only journal, it is not clear whether step 3 has happened, because it was not logged.
To avoid this out-of-order write hazard, file data must committed to storage before writing its associated metadata to the journal. This has generally tricky to implement because it requires coordination within the operating system kernel between the file system and memory cache, which traditionally uses elevator sorting (or some similar discipline) to maximize write throughput.
Soft updates take a variation of this approach by dispensing with a journal but imposing an order on all writes to ensure that the file system never becomes inconsistent to begin with, or that the only inconsistency that ever happens is a storage leak. Recovery then simply becomes a matter of running a background walk of the file system to garbage collect orphaned metadata.
See also
- Be File System
- WAFL file system
- Comparison of file systems
cs:Žurnálování de:Journaling-Dateisystem fr:Journal (système de fichier) ja:ジャーナリングファイルシステム pl:Journalling