Develop your own filesystem with FUSE

A filesystem is a method for storing and organizing computer files and directories and the data they contain, making it easy to find and access them. If you are using a computer, you are most likely using more than one kind of filesystem. A filesystem can provided extended capabilities. It can be written as a wrapper over an underlying filesystem to manage its data and provide an enhanced, feature-rich filesystem (such as cvsfs-fuse, which provides a filesystem interface for CVS, or a Wayback filesystem, which provides a backup mechanism to keep old copies of data).

Before the advent of user space filesystems, filesystem development was the job of the kernel developer. Creating filesystems required knowledge of kernel programming and the kernel technologies (like vfs). And debugging required C and C++ expertise. But other developers needed to manipulate a filesystem -- to add personalized features (such as adding history or forward-caching) and enhancements.

Introducing FUSE

FUSE lets you develop a fully functional filesystem that has a simple API library, can be accessed by non-privileged users, and provides a secure implementation. And, to top it all off, FUSE has a proven track record of stability.

With FUSE, you can develop a filesystem as executable binaries that are linked to the FUSE libraries -- in other words, this filesystem framework doesn't require you to learn filesystem internals or kernel module programming.

When it comes to filesystems, the user space filesystem is not a new design. A sampling of commercial and academic implementations of user space filesystems include:

• LUFS is a hybrid user space filesystem framework that supports an indefinite number of filesystems transparently for any application. It consists of a kernel module and a user space daemon. Basically it delegates most of the VFS calls to a specialized daemon that handles them.
• UserFS allows user processes to be mounted as a normal filesystem. This proof-of-concept prototype provides ftpfs, which allows anonymous FTP with a filesystem interface.
• The Ufo Project is a global filesystem for Solaris that allows users to treat remote files exactly as if they were local.
• OpenAFS is an open source version of the Andrew FileSystem.
• CIFS is the Common Internet FileSystem.

Unlike these commercial and academic examples, FUSE brings the capabilities of this filesystem design to Linux. Because FUSE uses executables (instead of, say, shared objects as LUFS uses), it makes debugging and developing easier. FUSE works with both kernels (2.4.x and 2.6.x) and now supports Java™ binding, so you aren't limited to programming the filesystem in C and C++. (See  for more userland filesystems that use FUSE.)

To create a filesystem in FUSE, you need to install a FUSE kernel module and then use the FUSE library and API set to create your filesystem.

Unpack FUSE

To develop a filesystem, first download the FUSE source code (see ) and unpack the package: tar -zxvf fuse-2.2.tar.gz. This creates a FUSE directory with the source code. The contents of the fuse-2.2 directory are:

• ./doc contains FUSE-related documentation. At this point, there is only one file, how-fuse-works.
• ./kernel contains the FUSE kernel module source code (which, of course, you don't need to know for developing a filesystem with FUSE).
• ./include contains the FUSE API headers, which you need to create a filesystem. The only one you need now is fuse.h.
• ./lib holds the source code to create the FUSE libraries that you will be linking with your binaries to create a filesystem.
• ./util has the source code for the FUSE utility library.
• ./example, of course, contains samples for your reference, like the fusexmp.null and hello filesystems.

Build and install FUSE

You can do the above steps in just one step if you want. From the fuse-2.2 directory, run ./configure; make; make install;. 

Important: When making FUSE, you need to have the kernel headers or source code in place. To keep things simple, make sure that the kernel source code is in the /usr/src/ directory.

Customize the filesystem

Now let's create a filesystem so you can access your AFS space on a Linux box with latest kernel, using an older Linux kernel. You will have two processes: one server process running on a older Linux kernel, and the FUSE client process running on a Linux box with the latest kernel. Whenever a request comes to your FUSE client process, it contacts the remote server process. For communication purposes, this filesystem uses RX RPC code that is part of AFS, so you need to build OpenAFS. (Figure 1 gives an overview of this AFS filesystem.)

Figure 1. Overview of AFS-FUSE filesystem

Resources

Learn

• For an excellent example of FUSE in action, see , which details and dissects Google Gmail (Richard Jones, Computerworld, September 2006).
• Read this introductory article on .
• The "" (developerWorks) is a series of articles that can help you become a filesystem expert.
• "" (developerWorks, September 2005) shows how to write better userspace applications.
• In the , find more resources for Linux developers.
• Stay current with .

Get products and technologies

• Download the .
• The  holds downloads, a FAQ and wiki, mailing lists galore, a , and a list of OSes and systems that support FUSE.
•  and comes with a "proof-of-concept" ZIP filesystem that seems to be pretty stable.
• With , available for download directly from developerWorks, build your next development project on Linux.

Discuss

• Check out  and get involved in the .

Comments