In Memory FUSE filesystem
Asked Answered
D

2

15

Write a simple FUSE filesystem that is stored in memory. The filesystem has to support the following commands:

ls, mkdir, cp

This question was recently asked in an interview, I wasn't able answer it. So I have decided to learn it.

Did some searching and found some guides on building my own FUSE filesystem. I am really clueless on how to implement the filesystem in memory.

My questions are

  • Am I heading in the right direction ?
  • What else should I read up on ?
  • What is the solution ?

Links that I am reading:

In the last link the there is a mention on in-memory caching with PyFileSystem. I am not sure on how that might help.

PS : This was a written interview Question, So the answer has to simple enough to write on paper within 10-15 mins.

Daladier answered 25/1, 2012 at 13:16 Comment(5)
I understand this might be circumventing the question, but why not use tmpfs instead of rolling your own filesystem through FUSE?Azurite
@FrédéricHamidi : tmpfs is a good alternative thanks. But sadly that does not answer the question as you mentioned .Daladier
I guess stored in memory means that you have to allocate some kind of buffer and use that buffer as fs backend?Tehuantepec
just download the tar.gz, start implementing the callbacks and you'll be done. when you are done coding the callbacks you will see clearly what you didn't understandColorfast
@FrédéricHamidi because it is one way to reduce the dependence of an unprivileged user on the sysadmin?Conk
J
6

I had taken a course where we had to build an in-memory distributed file system similar in design to Frangipani. The course was heavily inspired by MIT's Distributed Systems course. Doing their first few lab assignments would be a good exercise.

This tutorial is also quite helpful.

Jordonjorey answered 27/1, 2012 at 0:7 Comment(0)
B
5

You didn't specify a programming language, though FUSE is native C++ there are native Golang bindings, implemented at bazil.org/fuse.

I'd say that the primary parts of the answer need to include the following:

  1. A data structure to handle the file system tree in memory
  2. Descriptions of Nodes and their relationship to iNodes
  3. Hooks to capture FUSE server requests to handle the cli commands
  4. A description of mounting a folder with the FUSE server.

I recently wrote an in-memory file system using this adapter: github.com/bbengfort/memfs. My writeup about its performance is here: In-Memory File System with FUSE. Quickly, a few choices I made:

The in memory data structure contains 2 primary structs, dir and file that are both nodes:

type Node struct {
    ID uint64 
    Name string 
    Attrs fuse.Attr 
    Parent *Dir 
}

type Dir struct {
    Node
    Children map[string]Node
}

type File struct {
    Node
    Data []byte 
}

As you can see, this is a simple tree that is traversable up and down through the Children and Parent links. The Data attribute of the File holds all the contents of files. The file system therefore just needs to create a "root" directory called "\" at the mount point, and then on mkdir a Dir is added to its children, and on cp a File is added. In Go, this is as simple as:

type FS struct {
    root *Dir 
}

func Mount(path string) error {

    // Unmount the FS in case it was mounted with errors.
    fuse.Unmount(path)

    // Mount the FS with the specified options
    conn, err := fuse.Mount(path)
    if err != nil {
        return err
    }

    // Ensure that the file system is shutdown
    defer conn.Close()

    // Create the root dir and file system 
    memfs := FS{
        root: &Dir{
            ID: 1, 
            Name: "\", 
            Parent: nil, 
        },
    }

    // Serve the file system
    if err := fs.Serve(conn, memfs); err != nil {
        return err
    }
}

Now you need hooks to implement the various FUSE requests and calls. Here is an example for mkdir:

func (d *Dir) Mkdir(ctx context.Context, req *fuse.MkdirRequest) (fs.Node, error) {
    // Update the directory Atime
    d.Attrs.Atime = time.Now()

    // Create the child directory
    c := new(Dir)
    c.Init(req.Name, req.Mode, d)

    // Set the directory's UID and GID to that of the caller
    c.Attrs.Uid = req.Header.Uid
    c.Attrs.Gid = req.Header.Gid

    // Add the directory to the directory
    d.Children[c.Name] = c

    // Update the directory Mtime
    d.Attrs.Mtime = time.Now()

    return c, nil
}

Finally, close the interview question with a discussion about how to compile and run the server, mounting to a path and perhaps how FUSE intercepts kernel calls and passes them to your process in user space.

Bevins answered 5/2, 2017 at 14:42 Comment(2)
The writeup link died: web.archive.org/web/20170219015022/https://bbengfort.github.io/…Meunier
I've updated the writeup link to the new blog.Bevins

© 2022 - 2024 — McMap. All rights reserved.