How does Linux determine the order of module init calls?

Question

I have a device with SPI flash storage I'd like to use an UBIFS filesystem on that flash device as my rootfs. The problem I'm facing is that the UBI module initializes before the SPI module initializes. Because of this, when UBI loads, it cannot attach to the UBI device that I've told it to (via the kernel command line), so there is no rootfs. The console output below illustrates this.

I've been diving into the source enough to see that init/main.c has a do_initcalls() function that simply calls a list of function pointers. Those function pointers point to the all the module_init() functions of the modules that are built-in to the kernel. Those function pointers are placed in a special section in the kernel binary, so this order is chosen at compile-time. However, I haven't yet figured out how that order is determined.

    [    0.482500] UBI error: ubi_init: UBI error: cannot initialize UBI, error -19
    [    0.492500] atmel_spi atmel_spi.0: Using dma0chan0 (tx) and  dma0chan1 (rx) for DMA transfers
    [    0.500000] atmel_spi atmel_spi.0: Atmel SPI Controller at 0xf0000000 (irq 13)
    [    0.507500] m25p80 spi0.1: mx25l25635e (32768 Kbytes)
    [    0.512500] Creating 7 MTD partitions on "jedec_flash":
    [    0.520000] 0x000000000000-0x000000020000 : "loader"
    [    0.527500] 0x000000020000-0x000000060000 : "u-boot"
    [    0.537500] 0x000000060000-0x000000080000 : "u-boot-env"
    [    0.547500] 0x000000080000-0x000000280000 : "kernel0"
    [    0.557500] 0x000000280000-0x000000480000 : "kernel1"
    [    0.567500] 0x000000480000-0x000001240000 : "fs"
    [    0.575000] 0x000001240000-0x000002000000 : "play"
    [    0.590000] AT91SAM9 Watchdog enabled (heartbeat=15 sec, nowayout=0)
    [    0.607500] TCP cubic registered
    [    0.615000] VFS: Cannot open root device "ubi0:root0" or unknown-block(0,0)
    [    0.622500] Please append a correct "root=" boot option; here are the available partitions:
    [    0.630000] 1f00             128 mtdblock0  (driver?)
    [    0.635000] 1f01             256 mtdblock1  (driver?)
    [    0.640000] 1f02             128 mtdblock2  (driver?)
    [    0.645000] 1f03            2048 mtdblock3  (driver?)
    [    0.650000] 1f04            2048 mtdblock4  (driver?)
    [    0.655000] 1f05           14080 mtdblock5  (driver?)
    [    0.660000] 1f06           14080 mtdblock6  (driver?)
    [    0.665000] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)

Ninetta

I've flagged for migration to Stack Overflow, because even though this may not require source code modification, it does require at least Kbuild trickery. Also Stack Overflow is where the experts are. I suspect that you're not supposed to rely on a particular order and that the UBI driver should not try to access any device until it's told to mount something. – Halfwitted

possible duplicate of LINUX: order of statically linked module loading which suggests you need to make sure the modules occur in the (tree of) makefiles in the desired initialization order. – Etzel

There may be an alternate solution to the problem using EPROBE_DEFER; I'll be checking into that. – Ninetta

Answer 1

The init routines for a module that is initialized by the kernel (when they are statically linked into the kernel) are wrapped in an initcall() macro that indicates when in the startup sequence they should be run.

See the include file: include/linux/init.h for a list of the macros and their ordering.

The order specified there is:

• early_initcall
• pure_initcall
• core_initcall
• postcore_initcall
• arch_initcall
• subsys_initcall
• fs_initcall
• rootfs_initcall
• device_initcall
• late_initcall

Most of these have an "initcall_sync() phase, which is used to wait for completion of all module initialization routines within that phase. The macros are used to build a table of function pointers for each phase, which are called in sequence by do_initcalls().

If "module_init()" is used to wrap the initialization function, then by default initcall() puts the call in the "device" phase of initialization. Within that phase, the items are ordered by link order. This means that the table is created by the order of the functions as they are encountered by the linker.

You may be able to move an initialization to an earlier phase, by changing which initcall macro wraps the modules initialization function, but be careful because there are order dependencies between the various modules. Another method of changing the initialization order (within a phase) would be to adjust the link order for the module in the kernel.

Reverential

Answer 2

@Tim Bird has already answered it - I would like to show how to change the order for a module:-

pure_initcall(fn)         --> Loaded first
core_initcall(fn)         
core_initcall_sync(fn)    
postcore_initcall(fn)     
postcore_initcall_sync(fn)
arch_initcall(fn)         
arch_initcall_sync(fn)    
subsys_initcall(fn)       
subsys_initcall_sync(fn)  
fs_initcall(fn)           
fs_initcall_sync(fn)      
rootfs_initcall(fn)       
device_initcall(fn)       
device_initcall_sync(fn)  
late_initcall(fn)         
late_initcall_sync(fn)    --> Loaded last

Usage - Replace fn by the module init function pointer, example for i2c core driver:
.......
postcore_initcall(i2c_init);  // To delay i2c core loading use subsys_initcall(i2c_init)                
module_exit(i2c_exit);
.......

Bartolemo

Answer 3

I might be wrong, so please check if this is correct.
Try compiling all the drivers you need as modules (M), and put the modules to be loaded into /etc/modules, in the right order, this could solve your problem. Just to be precise, because you are doing this before your rootfs is mounted, the above step shoudl be done in the initram disk. (I have a similar scenario and I need to load some modules in the right order, to be able to decrypt a fs)

Hope this helps
CiaoCiao
Sergio

Indorse

It will not work to compile them as separate modules because it would put the modules on the root filesystem. These modules I'm talking about are the ones for mounting the root filesystem. – Ninetta

@ShawnJ.Goff: I edited the post, as I realized I forget to mention an important detail. The initramfs part. :) – Indorse

An initramfs is not an option - it increases the size of my build by an unacceptable amount. – Ninetta

Too bad :( I'd have a look at the code of depmod at this point. It's not strictly your scenario, but the depmod tools is able to scan the kernel folder and calculate modules dependencies. What I do not know is what kind of info it uses/read to determine the modules dependencies. Maybe you will find some hint in there... – Indorse