How to check a ubifs filesystem?

Question

ubifs has no fsck program, so how do you check the filesystem integrity when using ubifs?

My target system is ARM, Linux 3.2.58.

Viceroy

Answer 1

From what I have found so far on UBIFS' web page:

integrity - UBIFS (as well as UBI) checksums everything it writes to the flash media to guarantee data integrity, UBIFS does not leave data or meta-data corruptions unnoticed (JFFS2 is doing the same); By default, UBIFS checks only meta-data CRC when it reads from the media, but not the data CRC; however, you may force CRC checking for data using one of the UBIFS mount options - see here.

If you need to check filesystem for corruption

If your UBIFS filesystem is mounted with chk_data_crc option, then simple cat $FILES > /dev/null should be sufficient. If not, you can only detect and recover metadata corruption. The corruption of file body will pass unnoticed.

I used something like find / -type f -print -exec cat {} + > /dev/null

If you need to recover broken files

Again from the overview section:

to make it more clear, imaging you have wiped out the FAT table on your FAT file system; for FAT FS this would be fatal; but if you similarly wipe out UBIFS index, you still may re-construct it, although a special user-space tool would be required to do this (this utility is not implemented at the moment, though)

While theoretically possible, you're on your own.

Back up the flash contents, arm yourself with UBIFS data structures (maybe sources) and a hex editor, and good luck.

edit: As I figured, Linux's MTD driver already applies ECC (error correction code) to MTD devices.

I believe, the criterion for a data loss if there are more than /sys/class/mtd/mtd*/ecc_strength errors per /sys/class/mtd/mtd*/ecc_step_size flash block. mtd_read() (it's a MTD API one level below UBIFS) will return EUCLEAN in this case. Don't know if there exists a tool that uses it to check for errors.

The "bit flipped" warnings we get do not mean that there was a data loss yet. You can write to /sys/class/mtd/mtd*/bitflip_threshold to control the amount of warnings you get.

Safe

Answer 2

You can just read all files, which essentially causes ubifs to check them. Cfr. the advise given on the mailing list. The ubifs implementation will recover if it is possible. There is however no guarantee that this will capture all corruptions.

In theory, ubifs should never be corrupted, but in practice bugs in ubifs or NAND drivers can still cause corruptions.

Retsina

ubifs should never be corrupted - there is no such theory, and the original email doesn't say that. Flash does wear out. ubifs only can recover metadata, but not data: linux-mtd.infradead.org/doc/ubifs.html#L_overview – Safe

UBI is supposed to protect against flash wear. At least wear due to write cycles. It does not yet protect against retention time failures (i.e. not accessing a block at all for several years), but with uibihealthd that will also be covered. – Retsina

Sorry, "is supposed"? What supports this statement? I, for instance, witnessed a ubifs corruption yesterday. – Safe

"Is supposed" means: it is intended to. But there can be bugs, or types of flash corruption that the UBI developers haven't thought of. What really, really is NOT supposed to happen (i.e. what always is a bug) is that after a power cycle the filesystem doesn't mount any more. Note, however, that the bug is very often not in UBI/UBIFS, but rather in the NAND driver. – Retsina

a) Nobody talked about a power-cycle in this question. 2) once again, what do you have to support this statement? So far, in ubifs overview, we read: "recoverability - UBIFS may be fully recovered if the indexing information gets corrupted [...] integrity - UBIFS [...] checksums everything it writes [...] does not leave data or meta-data corruptions unnoticed". This is much weaker statement than "ubifs should never be corrupted". What is the source of your information? – Safe