How can I simulate ext3 filesystem corruption?

Question

I would like to simulate filesystem corruption for the purpose of testing how our embedded systems react to it and ultimately have them fail as gracefully as possible. We use different kinds of block device emulated flash storage for data which is modified often and unsuitable for storage in NAND/NOR.

Since I have a pretty good idea of how often data is modified in different parts of the file tree and where sensitive data is stored. I would like to inject errors in specific areas and not just randomly.

In cases of emergency we use fsck -y as a sort of last resort in order to attempt to bring the system up and report that is in a very bad state. I would very much like to cause errors which would trigger fsck to attempt repairs in order to study the effect on the systems capability to come back up.

dd if=/dev/random is not precise enough for my purpose since it can't easily be used to inject controlled errors. Are there any other tools or methods which fit my needs better or do I have to invent my own?

Beige

Great question, but I'd tag this fault-tolerance instead of graceful-degradation. – Wrinkly

Answer 1

If you already know what to modify, dd can read a file containing the bytes you want to write, and you tell it where to write them.

To figure out where to write, debugfs from the e2fsprogs package could help you.

Endophyte

Answer 2

It should be possible to use a Linux filesystem image fuzzer to cause corruption:

• e2fuzz from e2fsprogs
• http://people.redhat.com/sgrubb/files/fsfuzzer-0.7.tar.gz
• http://projects.info-pull.com/mokb/fsfuzzer-0.6.tgz
• http://www.cccmz.de/~snakebyte/fsfuzzer-0.6-lmh2.tar.bz2
• http://thread.gmane.org/gmane.comp.file-systems.ext4/32167 (uses zzuf)

or one of the Linux disk fault injection techniques (e.g. dm-flakey in corruption mode) described in https://unix.stackexchange.com/a/144200 .

Lieberman