Devfs (Device File System) FAQLinux Devfs (Device File System) FAQRichard Gooch20-AUG-2002Document languages:-----------------------------------------------------------------------------NOTE: the master copy of this document is available online at:http://www.atnf.csiro.au/~rgooch/linux/docs/devfs.htmland looks much better than the text version distributed with thekernel sources. A mirror site is available at:http://www.ras.ucalgary.ca/~rgooch/linux/docs/devfs.htmlThere is also an optional daemon that may be used with devfs. You canfind out more about it at:http://www.atnf.csiro.au/~rgooch/linux/A mailing list is available which you may subscribe to. Sendemailto majordomo@oss.sgi.com with the following line in thebody of the message:subscribe devfsTo unsubscribe, send the message body:unsubscribe devfsinstead. The list is archived athttp://oss.sgi.com/projects/devfs/archive/.-----------------------------------------------------------------------------ContentsWhat is it?Why do it?Who else does it?How it worksOperational issues (essential reading)Instructions for the impatientPermissions persistence across rebootsDealing with drivers without devfs supportAll the way with DevfsOther IssuesKernel Naming SchemeDevfsd Naming SchemeOld Compatibility NamesSCSI Host Probing IssuesDevice drivers currently portedAllocation of Device NumbersQuestions and AnswersMaking things workAlternatives to devfsWhat I don't like about devfsHow to report bugsStrange kernel messagesCompilation problems with devfsdOther resourcesTranslations of this document-----------------------------------------------------------------------------What is it?Devfs is an alternative to "real" character and block special deviceson your root filesystem. Kernel device drivers can register devices byname rather than major and minor numbers. These devices will appear indevfs automatically, with whatever default ownership andprotection the driver specified. A daemon (devfsd) can be used tooverride these defaults. Devfs has been in the kernel since 2.3.46.NOTE that devfs is entirely optional. If you prefer the olddisc-based device nodes, then simply leave CONFIG_DEVFS_FS=n (thedefault). In this case, nothing will change.  ALSO NOTE that if you doenable devfs, the defaults are such that full compatibility ismaintained with the old devices names.There are two aspects to devfs: one is the underlying devicenamespace, which is a namespace just like any mounted filesystem. Theother aspect is the filesystem code which provides a view of thedevice namespace. The reason I make a distinction is because devfscan be mounted many times, with each mount showing the same devicenamespace. Changes made are global to all mounted devfs filesystems.Also, because the devfs namespace exists without any devfs mounts, youcan easily mount the root filesystem by referring to an entry in thedevfs namespace.The cost of devfs is a small increase in kernel code size and memoryusage. About 7 pages of code (some of that in __init sections) and 72bytes for each entry in the namespace. A modest system has only acouple of hundred device entries, so this costs a few morepages. Compare this with the suggestion to put /dev on a <ahref="#why-faq-ramdisc">ramdisc.On a typical machine, the cost is under 0.2 percent. On a modestsystem with 64 MBytes of RAM, the cost is under 0.1 percent.  Theaccusations of "bloatware" levelled at devfs are not justified.-----------------------------------------------------------------------------Why do it?There are several problems that devfs addresses. Some of theseproblems are more serious than others (depending on your point ofview), and some can be solved without devfs. However, the totality ofthese problems really calls out for devfs.The choice is a patchwork of inefficient user space solutions, whichare complex and likely to be fragile, or to use a simple and efficientdevfs which is robust.There have been many counter-proposals to devfs, all seeking toprovide some of the benefits without actually implementing devfs. Sofar there has been an absence of code and no proposed alternative hasbeen able to provide all the features that devfs does. Further,alternative proposals require far more complexity in user-space (andstill deliver less functionality than devfs). Some people have themantra of reducing "kernel bloat", but don't consider the effects onuser-space.A good solution limits the total complexity of kernel-space anduser-space.Major&minor allocationThe existing scheme requires the allocation of major and minor devicenumbers for each and every device. This means that a centralco-ordinating authority is required to issue these device numbers(unless you're developing a "private" device driver), in order topreserve uniqueness. Devfs shifts the burden to a namespace. This maynot seem like a huge benefit, but actually it is. Since driver authorswill naturally choose a device name which reflects the functionalityof the device, there is far less potential for namespace conflict.Solving this requires a kernel change./dev managementBecause you currently access devices through device nodes, these mustbe created by the system administrator. For standard devices you canusually find a MAKEDEV programme which creates all these (hundreds!)of nodes. This means that changes in the kernel must be reflected bychanges in the MAKEDEV programme, or else the system administratorcreates device nodes by hand.The basic problem is that there are two separate databases ofmajor and minor numbers. One is in the kernel and one is in /dev (orin a MAKEDEV programme, if you want to look at it that way). This isduplication of information, which is not good practice.Solving this requires a kernel change./dev growthA typical /dev has over 1200 nodes! Most of these devices simply don'texist because the hardware is not available. A huge /dev increases thetime to access devices (I'm just referring to the dentry lookup timesand the time taken to read inodes off disc: the next subsection showssome more horrors).An example of how big /dev can grow is if we consider SCSI devices:host           6  bits  (say up to 64 hosts on a really big machine)channel        4  bits  (say up to 16 SCSI buses per host)id             4  bitslun            3  bitspartition      6  bitsTOTAL          23 bitsThis requires 8 Mega (1024*1024) inodes if we want to store allpossible device nodes. Even if we scrap everything but id,partitionand assume a single host adapter with a single SCSI bus and only onelogical unit per SCSI target (id), that's still 10 bits or 1024inodes. Each VFS inode takes around 256 bytes (kernel 2.1.78), sothat's 256 kBytes of inode storage on disc (assuming real inodes takea similar amount of space as VFS inodes). This is actually not so bad,because disc is cheap these days. Embedded systems would care about256 kBytes of /dev inodes, but you could argue that embedded systemswould have hand-tuned /dev directories. I've had to do just that on myembedded systems, but I would rather just leave it to devfs.Another issue is the time taken to lookup an inode when firstreferenced. Not only does this take time in scanning through a list inmemory, but also the seek times to read the inodes off disc.This could be solved in user-space using a clever programme whichscanned the kernel logs and deleted /dev entries which are notavailable and created them when they were available. This programmewould need to be run every time a new module was loaded, which wouldslow things down a lot.There is an existing programme called scsidev which will automaticallycreate device nodes for SCSI devices. It can do this by scanning filesin /proc/scsi. Unfortunately, to extend this idea to other devicenodes would require significant modifications to existing drivers (sothey too would provide information in /proc). This is a non-trivialchange (I should know: devfs has had to do something similar). Onceyou go to this much effort, you may as well use devfs itself (whichalso provides this information).  Furthermore, such a system wouldlikely be implemented in an ad-hoc fashion, as different drivers willprovide their information in different ways.Devfs is much cleaner, because it (naturally) has a uniform mechanismto provide this information: the device nodes themselves!Node to driver file_operations translationThere is an important difference between the way disc-based characterand block nodes and devfs entries make the connection between an entryin /dev and the actual device driver.With the current 8 bit major and minor numbers the connection betweendisc-based c&b nodes and per-major drivers is done through afixed-length table of 128 entries. The various filesystem types setthe inode operations for c&b nodes to {chr,blk}dev_inode_operations,so when a device is opened a few quick levels of indirection bring usto the driver file_operations.For miscellaneous character devices a second step is required: thereis a scan for the driver entry with the same minor number as the filethat was opened, and the appropriate minor open method is called. Thisscanning is done *every time* you open a device node. Potentially, youmay be searching through dozens of misc. entries before you find youropen method. While not an enormous performance overhead, this doesseem pointless.Linux *must* move beyond the 8 bit major and minor barrier,somehow. If we simply increase each to 16 bits, then the indexingscheme used for major driver lookup becomes untenable, because themajor tables (one each for character and block devices) would need tobe 64 k entries long (512 kBytes on x86, 1 MByte for 64 bitsystems). So we would have to use a scheme like that used formiscellaneous character devices, which means the search time goes uplinearly with the average number of major device drivers on yoursystem. Not all "devices" are hardware, some are higher-level driverslike KGI, so you can get more "devices" without adding hardwareYou can improve this by creating an ordered (balanced:-)binary tree, in which case your search time becomes log(N).Alternatively, you can use hashing to speed up the search.But why do that search at all if you don't have to? Once again, itseems pointless.Note that devfs doesn't use the major&minor system. For devfsentries, the connection is done when you lookup the /dev entry. Whendevfs_register() is called, an internal table is appended which hasthe entry name and the file_operations. If the dentry cache doesn'thave the /dev entry already, this internal table is scanned to get thefile_operations, and an inode is created. If the dentry cache alreadyhas the entry, there is *no lookup time* (other than the dentry scanitself, but we can't avoid that anyway, and besides Linux dentriescream other OS's which don't have them:-). Furthermore, the number ofnode entries in a devfs is only the number of available deviceentries, not the number of *conceivable* entries. Even if you removeunnecessary entries in a disc-based /dev, the number of conceivableentries remains the same: you just limit yourself in order to savespace.Devfs provides a fast connection between a VFS node and the devicedriver, in a scalable way./dev as a system administration toolRight now /dev contains a list of conceivable devices, most of which Idon't have. Devfs only shows those devices available on mysystem. This means that listing /dev is a handy way of checking whatdevices are available.Major&minor sizeExisting major and minor numbers are limited to 8 bits each. This isnow a limiting factor for some drivers, particularly the SCSI discdriver, which consumes a single major number. Only 16 discs aresupported, and each disc may have only 15 partitions. Maybe this isn'ta problem for you, but some of us are building huge Linux systems withdisc arrays. With devfs an arbitrary pointer can be associated witheach device entry, which can be used to give an effective 32 bitdevice identifier (i.e. that's like having a 32 bit minornumber). Since this is private to the kernel, there are no C librarycompatibility issues which you would have with increasing major andminor number sizes. See the section on "Allocation of Device Numbers"for details on maintaining compatibility with userspace.Solving this requires a kernel change.Since writing this, the kernel has been modified so that the SCSI discdriver has more major numbers allocated to it and now supports up to128 discs. Since these major numbers are non-contiguous (a result ofunplanned expansion), the implementation is a little more cumbersomethan originally.Just like the changes to IPv4 to fix impending limitations in theaddress space, people find ways around the limitations. In the longrun, however, solutions like IPv6 or devfs can't be put off forever.Read-only root filesystemHaving your device nodes on the root filesystem means that you can'toperate properly with a read-only root filesystem. This is because youwant to change ownerships and protections of tty devices. Existingpractice prevents you using a CD-ROM as your root filesystem for a*real* system. Sure, you can boot off a CD-ROM, but you can't changetty ownerships, so it's only good for installing.Also, you can't use a shared NFS root filesystem for a cluster ofdiscless Linux machines (having tty ownerships changed on a common/dev is not good). Nor can you embed your root filesystem in aROM-FS.You can get around this by creating a RAMDISC at boot time, makingan ext2 filesystem in it, mounting it somewhere and copying thecontents of /dev into it, then unmounting it and mounting it over/dev.A devfs is a cleaner way of solving this.Non-Unix root filesystemNon-Unix filesystems (such as NTFS) can't be used for a rootfilesystem because they variously don't support character and blockspecial files or symbolic links. You can't have a separate disc-basedor RAMDISC-based filesystem mounted on /dev because you need devicenodes before you can mount these. Devfs can be mounted without anydevice nodes. Devlinks won't work because symlinks aren't supported.An alternative solution is to use initrd to mount a RAMDISC initialroot filesystem (which is populated with a minimal set of devicenodes), and then construct a new /dev in another RAMDISC, and finallyswitch to your non-Unix root filesystem. This requires clever bootscripts and a fragile and conceptually complex boot procedure.Devfs solves this in a robust and conceptually simple way.PTY security