DragonFly BSD
|
|
DragonFlyBSD-Logo.png
In computing, the DragonFly BSD operating system is a fork of FreeBSD. Matt Dillon, a long-time FreeBSD and Amiga developer, started work on DragonFly BSD in June 2003 and announced it on the FreeBSD mailing lists on 16 July 2003.
Dillon started DragonFly in the belief that the methods and techniques being adopted for threading and SMP in FreeBSD 5 would lead to a poorly performing system that would be very difficult to maintain. He sought to correct these suspected problems within the FreeBSD project. Others in the project did not think highly of his ideas, which is among the reasons his ability to directly change the FreeBSD code was revoked. Despite this, the DragonFly BSD and FreeBSD projects still work together contributing bug fixes, driver updates and other system improvements to each other.
Intended to be "the logical continuation of the FreeBSD 4.x series", DragonFly is being developed in an entirely different direction from FreeBSD 5, including a new Light Weight Kernel Threads implementation and a light weight ports/messaging system. Many concepts planned for DragonFly were inspired by the AmigaOS.
| Contents |
Kernel architecture
Like most modern kernels, DragonFly is a hybrid, containing features of both monolithic and microkernels, attempting to make the best use of both technologies, such as the message passing capability of microkernels enabling larger portions of the OS to benefit from protected memory, as well as retaining the speed of monolithic kernels for certain critical tasks. The messaging subsystem being developed is similar to those found in microkernels such as Mach, though it is less complex by design. DragonFly's messaging subsystem has the ability to act either in a synchronous, or an asynchronous fashion, and uses this capability to attempt to achieve the best performance possible in any given situation.
There is progress being made to provide both device input/output (I/O) and virtual file system (VFS) messaging capabilities that will allow the remainder of the project goals to be met. The new infrastructure will allow many parts of the kernel to be migrated out into userland, whereby they will be more easily debugged as they will be smaller, isolated programs, instead of being small parts entwined in a larger chunk of code. The migration of select kernel code into userspace has the additional benefit of making the system more robust; if a userspace driver crashes, it will not crash the kernel.
System calls are being split into userland and kernel versions, as well as being encapsulated into messages. This will help reduce the size and complexity of the kernel by moving variants of standard system calls into a userland compatibility layer, as well as help maintain forwards and backwards compatibility between DragonFly versions. Linux and other Unix-like OS compatibility code is being migrated out similarly. Multiple instances of the 'native' userland compatibility layer created in jails could give DragonFly functionality similar to that found in User Mode Linux (UML). Unlike UML (which is essentially a port of Linux to itself as if the host kernel was a different hardware platform), DragonFly's virtualization will not require special drivers to communicate with the real hardware on the computer.
Scalability improvements
Compartmentalization
In DragonFly, threads are locked to CPUs by design, and each processor has its own LWKT scheduler. Threads are never preemptively switched from one processor to another; they are only migrated by the passing of an "Interprocessor Interrupt" (IPI) message between the CPUs involved. Interprocessor thread scheduling is also accomplished by sending asynchronous IPI messages. One advantage to this clean compartmentalization of the threading subsystem is that the processors' on-board caches in SMP systems do not contain duplicated data, allowing for higher performance by giving each processor in the system the ability to use its own cache to store different things to work on.
Protecting shared resources
In order to run safely on multiprocessor machines, access to shared resources (files, data structures etc.) must be serialized so that threads or processes do not attempt to modify the same resource at the same time. Atomic operations, spinlocks, critical sections, mutexes and serializing tokens are all possible methods that can be used to prevent concurrent access. Whereas both Linux and FreeBSD 5 employ fine-grained mutex models to achieve higher performance on multiprocessor systems, DragonFly does not. In order to prevent multiple threads from accessing or modifying a shared resource simultaneously, DragonFly employs SPLs, critical sections, and serializing tokens to prevent concurrent access.
Much of the system's core including the LWKT subsystem, the IPI messaging subsystem and the new kernel memory allocator among other things are lockless, meaning that they work without using mutexes. Critical sections are used to protect against local interrupts and operate on a per-CPU basis, guaranteeing that a thread currently being executed will not be preempted.
Serializing tokens are used to prevent concurrent accesses from other CPUs and may be held simultaneously by multiple threads, ensuring that only one of those threads is running at any given time. Blocked or sleeping threads therefore do not prevent other threads from accessing the shared resource unlike a thread that is holding a mutex. Among other things, the use of serializing tokens prevents many of the situations that could result in deadlocks and priority inversions when using mutexes, as well as greatly simplifying the design and implementation of a many-step procedure that would require a shared resource among multiple threads. The serializing token code is evolving into something quite similar to the "Read-copy-update" feature now available in Linux. Unlike Linux's current RCU implementation, DragonFly's is being implemented such that only processors competing for the same token are affected rather than all processors in the computer.
Additional features
Early on in its development, DragonFly acquired a slab allocator, which replaced the aging FreeBSD 4 kernel memory allocator. The new slab allocator requires neither mutexes nor blocking operations for memory assignment tasks, and unlike the code it replaced, it is multiprocessor safe.
Amiga-style 'resident' application support has been added which takes a snapshot of a large, dynamically linked program's virtual memory space after loading, allowing future instances of the program to start much more quickly than it otherwise would have. This replaces the prelinking capability that was being worked on earlier in the project's history, as the resident support is much more efficient. Large programs like those found in KDE with many shared libraries will benefit the most from this support.
DragonFly uses SFBUFs (Super-Fast BUFfers) and MSFBUFs (Multi-SFBUFs). An SFBUF is used to manage ephemeral single-page mappings and cache them when appropriate. They are used for retrieving a reference to data that is held by a single VM page. This simple, yet powerful abstraction gives a broad number of abilities, such as zero-copy achieved in the sendfile(2) system call.
SFBUFs are used in numerous parts of the kernel, such as the Vnode Object Pager and the PIPE subsystems (indirectly via XIOs) for supporting high-bandwidth transfers. An SFBUF can only be used for a *single* VM page; MSFBUFs are used for managing ephemeral mappings of multiple-pages.
The SFBUF concept was devised by David Greenman of the FreeBSD Project when he wrote the sendfile(2) system call; it was later revised by Dr. Alan L. Cox and Matthew Dillon. MSFBUFs were designed by Hiten Pandya and Matthew Dillon.
Other features new to BSD in general include variant symlinks and application checkpointing support.
Development and distribution
DragonFly forked from FreeBSD 4.8 and imports features and bug fixes from FreeBSD 4 and 5 where appropriate, such as ACPI and a new ATA driver framework from FreeBSD 4. Physical Address Extension (PAE) support may also soon be added. As the number of DragonFly developers is currently small, with most of them focused on implementing basic functionality, device drivers are being kept mostly in sync with FreeBSD 5.x, the branch of FreeBSD where all new drivers are being written. The DragonFly developers are slowly moving toward using the "busdma" APIs, which will help to make the system easier to port to new architectures, but it is not a major focus at this time.
The BSD operating systems are actively replacing "K&R" style C code with modern, ANSI equivalents. Also like OpenBSD, DragonFly's version of the GNU Compiler Collection has an enhancement called the "Stack-Smashing Protector" (formerly known as "ProPolice") enabled by default, providing some additional protection against buffer overflow based attacks.
Being a derivative of FreeBSD, DragonFly has inherited an easy to use integrated build system that can rebuild the entire base system from source with only a few commands. Like people from the other BSD projects, the DragonFly developers use a version control system called CVS to manage changes to the DragonFly source code. Unlike its parent FreeBSD, DragonFly will have both stable and unstable releases in a single source tree, due to a smaller developer base.
There is now a live CD distribution that boots into a complete DragonFly system. In the future, this will be usable as a recovery CD, an installation CD and as a demo CD. It includes the base system and a complete set of manual pages and will include source code and useful packages in future versions. Daily snapshots are available from Simon 'corecode' Schubert via FTP (ftp://chlamydia.fs.ei.tum.de/pub/DragonFly/snapshots/i386/ISO-IMAGES/) and HTTP (http://chlamydia.fs.ei.tum.de/pub/DragonFly/snapshots/i386/ISO-IMAGES/) for those who want to install the most recent versions of DragonFly without needing to build from source. An installer was added just before the first release, eliminating the need to install DragonFly manually.
Like the other free, open source BSDs, DragonFly is distributed under the terms of the modern version of the BSD license.
Releases
- Version 1.0 Release Candidate 1 was announced on 28 June, 2004.
- Version 1.0 Release Candidate 2 was released on 5 July, 2004.
- Version 1.0 was released on 12 July, 2004.
- Version 1.2 Release Candidate 1 was released on 7 April, 2005 [1] (http://www.osnews.com/comment.php?news_id=10229).
- Version 1.2 was released on 8 April, 2005 [2] (http://www.dragonflybsd.org/main/release1_2.cgi)
Future directions
DragonFly BSD 1.0 was meant to be a "technology showcase" rather than an integrated production release. It is hoped that the messaging infrastructure will be fully in place, and the VFS subsystem is to be capable of hosting filesystem drivers in userland. For the time being, DragonFly is to continue using FreeBSD's packaged third party software, and DragonFly's own native VFS based package management system is not expected to be completed for some time.
Although both system calls and device I/O have been largely converted to DragonFly's threaded messaging interface, both still operate synchronously. Ultimately, the entire messaging system is to be capable of both synchronous and asynchronous operation.
Userland VFS - the ability to migrate filesystem drivers into userspace, will take a lot of work to accomplish. Some of this work is already complete, though there is still much to do. The namecache code has to be extracted from and made independent of the VFS code, and this is Dillon's next major focus. Converting the VFS code to DragonFly's threaded messaging interface is more difficult than converting the device I/O and system calls was, due to the fact that the VFS system inherited from FreeBSD uses a massively reentrant model. Once the namecache code has been separated out, the remaining VFS code will have to be largely, if not completely rewritten.
The userland VFS system is a prerequisite of a number of desired features to be incorporated into DragonFly. Dillon envisions a new package management system based at least in part, on "VFS environments" which give the packages the environment they expect to be in, independent of the larger filesystem environment and its quirks. In addition to system call message filtering, VFS environments are also to play a role in future security mechanisms, by restricting users or processes to their own isolated environments.
Currently, DragonFly runs on x86 (Intel and AMD) based computers, both single processor and SMP models. A port to the Athlon 64 (x86-64) has been started, but doesn't exist yet. A port to the PowerPC processor has been speculated about sometime following the eventual x86-64 port.
Ultimately, Dillon wants DragonFly to natively enable "secure anonymous system clustering over the Internet," and the light weight ports/messaging system will help to provide this capability. Security settings aside, there is technically no difference between messages created locally or on another computer over a network. Achieving this "single-system image" capability transparently will be a big job, and will take quite some time to properly implement, even with the new foundation fully in place. While some of the short term goals of the project will be completed in months, other features may take years to complete.
External links
General information
- The DragonFly BSD Project home page (http://www.DragonFlyBSD.org/)
- Original announcement (http://lists.freebsd.org/pipermail/freebsd-current/2003-July/006889.html) on the FreeBSD -CURRENT mailing list
- DragonFly Packages (http://gobsd.com/packages/) from GoBSD.com
- DragonFly Installer (http://www.bsdinstaller.com)
- Original Announcement (http://groups.google.com/groups?hl=en&lr=&th=e096e605ffedd05b)
Wiki
- the DragonFly BSD Wiki (http://wiki.dragonflybsd.org)
News and interviews
- The DragonFly BSD Digest (http://www.shiningsilence.com/dbsdlog/)
- 2003 Year-End summary (http://www.dragonflybsd.org/status/report-2003.cgi)
- Matt Dillon IRC Interview (http://www.slashnet.org/forums/DragonflyBSD-20031009.html) from http://www.slashnet.org/
- OSNews interview (http://www.osnews.com/story.php?news_id=6338) (13 March 2004)
- Behind DragonFly BSD (http://www.onlamp.com/pub/a/bsd/2004/07/08/dragonfly_bsd_interview.html) interview with DFly developers that was published on O'Reilly OnLamp.com (http://www.onlamp.com) (8 July 2004)
- DragonFlyBSD 1.0A: A strong start (http://www.newsforge.com/article.pl?sid=04/08/02/1531233)
Technology references
- The DragonFlyBSD Operating System (http://people.freebsd.org/~hsu/papers/dragonflybsd.asiabsdcon04.pdf)
- The Slab Allocator (Part 1) -- Bonwick94 (http://citeseer.ist.psu.edu/bonwick94slab.html) and The Slab Allocator (Part 2) -- Bonwick2001 (http://www.usenix.org/event/usenix01/bonwick.html)
- Improving the FreeBSD SMP implementation (http://www.lemis.com/~grog/SMPng/USENIX/paper.ascii) (DragonFly does things differently)
- Intel's MultiProcessor Specification (http://developer.intel.com/design/pentium/datashts/24201606.pdf) (PDF)
- Read-Copy Update Mutual Exclusion (http://lse.sourceforge.net/locking/rcupdate.html)
- The stack-smashing protector (http://www.trl.ibm.com/projects/security/ssp/)
- The Grid Engine Project (http://gridengine.sunsource.net/)
- FreeBSD Jails (http://docs.freebsd.org/44doc/papers/jail/jail.html)
- The Design and Implementation of the 4.4BSD Operating System (http://www.freebsd.org/doc/en_US.ISO8859-1/books/design-44bsd/)
Documentation
- DragonFly handbook (http://leaf.dragonflybsd.org/~justin/handbook/)
- Printable manual pages and supplementary documentation (Postscript and PDF) (http://yoyodyne.ath.cx/paper/dflyman)
Howtos (From The DragonFly BSD Digest)
- Installing DragonFly BSD from source (http://www.shiningsilence.com/dbsdlog/archives/000009.html) (Upgrading from FreeBSD 4.x)
- How to enable profiling to help catch bugs (http://www.shiningsilence.com/dbsdlog/archives/000071.html)
- Getting a kernel crash dump (http://www.shiningsilence.com/dbsdlog/archives/000336.html)
- Set up a boot splash screen (http://www.shiningsilence.com/dbsdlog/archives/000047.html)
- How to do a diff (http://www.shiningsilence.com/dbsdlog/archives/000059.html)de:DragonFly BSD
fr:DragonFly BSD ja:DragonFly BSD pl:DragonFly BSD sv:DragonFly BSD