Legacy BIOS bootloader to bootstrap real-mode code in second stage

Question

I am working on writing my own operating system. So far, my code exceeds 512 bytes, which is too large to fit in a simple boot sector.

I understand that I now have to write a bootloader that reads arbitrary code that may or may not be greater than a single 512-byte sector.

The bootloader would need to:

• Function as a boot record with disk signature 0xaa55.
• Read a second stage (the test code) start from LBA 1 (LBA 0 is boot sector) of arbitrary length starting at memory address 0x7E00.
• Transfer control to it using a FAR JMP to 0x0000:0x7E00.
• Be usable as a 1.44 MiB floppy disk image for use in emulators like QEMU, BOCHS, VirtualBox etc.
• Can be transferred and used on a USB stick to test on real hardware with the BIOS set to boot USB using Floppy Disk Drive (FDD) emulation. Note: Some bootloaders do not work well when placed on USB drives.
• Pass the boot drive to the second stage in DL.
• Zero out all the segment registers and set SS:SP to 0x0000:0x7C00 (grows down from just under the bootloader).

This would also serve as a good starting point for asking questions on Stack Overflow that involve OS development. Programmers often struggle to create a Minimal, Complete, and Verifiable Example. A common boilerplate/template would allow other Stack Overflow users wishing to help to test the code with a limited amount of fuss.

How would I go about building such a reusable bootloader?

Keener

Note: This question is being discussed on Meta. If you have an objection to its format, or suggestions on how to improve, please weigh in there. Limit discussion here to technical matters and clarifications about the question itself. – Trying

For an alternate implementation I've done something similar. Though it was done a while back as I was learning different parts of x86. github.com/sherrellbc/realmode-loader – Thrum

Answer 1

I have written such code as part of other answers but never had an opportunity to present a simple test harness that could be referenced from other Stackoverflow questions. What you are asking for is rather trivial. One can do this by writing a bootloader in NASM that includes a binary image of the assembled code you wish to test. This image would be read from disk starting at LBA 1 (first sector after the bootloader) using BIOS function Int 13/ah=2. Control would then be transferred to it via a FAR JMP to 0x0000:0x7e00.

The bootloader code would look like this:

bpb.inc:

global bpb_disk_info

    jmp short boot_continue
    nop

bpb_disk_info:

    ; Dos 4.0 EBPB 1.44MB floppy
    OEMname:           db    "mkfs.fat"  ; mkfs.fat is what OEMname mkdosfs uses
    bytesPerSector:    dw    512
    sectPerCluster:    db    1
    reservedSectors:   dw    1
    numFAT:            db    2
    numRootDirEntries: dw    224
    numSectors:        dw    2880
    mediaType:         db    0xf0
    numFATsectors:     dw    9
    sectorsPerTrack:   dw    18
    numHeads:          dw    2
    numHiddenSectors:  dd    0
    numSectorsHuge:    dd    0
    driveNum:          db    0
    reserved:          db    0
    signature:         db    0x29
    volumeID:          dd    0x2d7e5a1a
    volumeLabel:       db    "NO NAME    "
    fileSysType:       db    "FAT12   "

boot.asm:

STAGE2_ABS_ADDR  equ 0x07e00
STAGE2_RUN_SEG   equ 0x0000
STAGE2_RUN_OFS   equ STAGE2_ABS_ADDR
                                ; Run stage2 with segment of 0x0000 and offset of 0x7e00

STAGE2_LOAD_SEG  equ STAGE2_ABS_ADDR>>4
                                ; Segment to start reading Stage2 into
                                ;     right after bootloader

STAGE2_LBA_START equ 1          ; Logical Block Address(LBA) Stage2 starts on
                                ;     LBA 1 = sector after boot sector
STAGE2_LBA_END   equ STAGE2_LBA_START + NUM_STAGE2_SECTORS
                                ; Logical Block Address(LBA) Stage2 ends at
DISK_RETRIES     equ 3          ; Number of times to retry on disk error

bits 16
ORG 0x7c00

; Include a BPB (1.44MB floppy with FAT12) to be more compatible with USB floppy media
%ifdef WITH_BPB
%include "bpb.inc"
%endif

boot_continue:
    xor ax, ax                  ; DS=SS=0 for stage2 loading
    mov ds, ax
    mov ss, ax                  ; Stack at 0x0000:0x7c00
    mov sp, 0x7c00
    cld                         ; Set string instructions to use forward movement

    ; Read Stage2 1 sector at a time until stage2 is completely loaded
load_stage2:
    mov [bootDevice], dl        ; Save boot drive
    mov di, STAGE2_LOAD_SEG     ; DI = Current segment to read into
    mov si, STAGE2_LBA_START    ; SI = LBA that stage2 starts at
    jmp .chk_for_last_lba       ; Check to see if we are last sector in stage2

.read_sector_loop:
    mov bp, DISK_RETRIES        ; Set disk retry count

    call lba_to_chs             ; Convert current LBA to CHS
    mov es, di                  ; Set ES to current segment number to read into
    xor bx, bx                  ; Offset zero in segment

.retry:
    mov ax, 0x0201              ; Call function 0x02 of int 13h (read sectors)
                                ;     AL = 1 = Sectors to read
    int 0x13                    ; BIOS Disk interrupt call
    jc .disk_error              ; If CF set then disk error

.success:
    add di, 512>>4              ; Advance to next 512 byte segment (0x20*16=512)
    inc si                      ; Next LBA

.chk_for_last_lba:
    cmp si, STAGE2_LBA_END      ; Have we reached the last stage2 sector?
    jl .read_sector_loop        ;     If we haven't then read next sector

.stage2_loaded:
    mov ax, STAGE2_RUN_SEG      ; Set up the segments appropriate for Stage2 to run
    mov ds, ax
    mov es, ax

    ; FAR JMP to the Stage2 entry point at physical address 0x07e00
    xor ax, ax                  ; ES=FS=GS=0 (DS zeroed earlier)
    mov es, ax

    ; SS:SP is already at 0x0000:0x7c00, keep it that way
    ; DL still contains the boot drive number
    ; Far jump to second stage at 0x0000:0x7e00
    jmp STAGE2_RUN_SEG:STAGE2_RUN_OFS

.disk_error:
    xor ah, ah                  ; Int13h/AH=0 is drive reset
    int 0x13
    dec bp                      ; Decrease retry count
    jge .retry                  ; If retry count not exceeded then try again

error_end:
    ; Unrecoverable error; print drive error; enter infinite loop
    mov si, diskErrorMsg        ; Display disk error message
    call print_string
    cli
.error_loop:
    hlt
    jmp .error_loop

; Function: print_string
;           Display a string to the console on display page 0
;
; Inputs:   SI = Offset of address to print
; Clobbers: AX, BX, SI

print_string:
    mov ah, 0x0e                ; BIOS tty Print
    xor bx, bx                  ; Set display page to 0 (BL)
    jmp .getch
.repeat:
    int 0x10                    ; print character
.getch:
    lodsb                       ; Get character from string
    test al,al                  ; Have we reached end of string?
    jnz .repeat                 ;     if not process next character
.end:
    ret

;    Function: lba_to_chs
; Description: Translate Logical block address to CHS (Cylinder, Head, Sector).
;
;   Resources: http://www.ctyme.com/intr/rb-0607.htm
;              https://en.wikipedia.org/wiki/Logical_block_addressing#CHS_conversion
;              https://mcmap.net/q/741805/-why-isn-39-t-my-root-directory-being-loaded-fat12/3857942
;              Sector    = (LBA mod SPT) + 1
;              Head      = (LBA / SPT) mod HEADS
;              Cylinder  = (LBA / SPT) / HEADS
;
;      Inputs: SI = LBA
;     Outputs: DL = Boot Drive Number
;              DH = Head
;              CH = Cylinder (lower 8 bits of 10-bit cylinder)
;              CL = Sector/Cylinder
;                   Upper 2 bits of 10-bit Cylinders in upper 2 bits of CL
;                   Sector in lower 6 bits of CL
;
;       Notes: Output registers match expectation of Int 13h/AH=2 inputs
;
lba_to_chs:
    push ax                    ; Preserve AX
    mov ax, si                 ; Copy LBA to AX
    xor dx, dx                 ; Upper 16-bit of 32-bit value set to 0 for DIV
    div word [sectorsPerTrack] ; 32-bit by 16-bit DIV : LBA / SPT
    mov cl, dl                 ; CL = S = LBA mod SPT
    inc cl                     ; CL = S = (LBA mod SPT) + 1
    xor dx, dx                 ; Upper 16-bit of 32-bit value set to 0 for DIV
    div word [numHeads]        ; 32-bit by 16-bit DIV : (LBA / SPT) / HEADS
    mov dh, dl                 ; DH = H = (LBA / SPT) mod HEADS
    mov dl, [bootDevice]       ; boot device, not necessary to set but convenient
    mov ch, al                 ; CH = C(lower 8 bits) = (LBA / SPT) / HEADS
    shl ah, 6                  ; Store upper 2 bits of 10-bit Cylinder into
    or  cl, ah                 ;     upper 2 bits of Sector (CL)
    pop ax                     ; Restore scratch registers
    ret

; If not using a BPB (via bpb.inc) provide default Heads and SPT values
%ifndef WITH_BPB
numHeads:        dw 2          ; 1.44MB Floppy has 2 heads & 18 sector per track
sectorsPerTrack: dw 18
%endif

bootDevice:      db 0x00
diskErrorMsg:    db "Unrecoverable disk error!", 0

; Pad boot sector to 510 bytes and add 2 byte boot signature for 512 total bytes
TIMES 510-($-$$) db  0
dw 0xaa55

; Beginning of stage2. This is at 0x7E00 and will allow your stage2 to be 32.5KiB
; before running into problems. DL will be set to the drive number originally
; passed to us by the BIOS.

NUM_STAGE2_SECTORS equ (stage2_end-stage2_start+511) / 512
                                ; Number of 512 byte sectors stage2 uses.

stage2_start:
    ; Insert stage2 binary here. It is done this way since we
    ; can determine the size(and number of sectors) to load since
    ;     Size = stage2_end-stage2_start
    incbin "stage2.bin"

; End of stage2. Make sure this label is LAST in this file!
stage2_end:

; Fill out this file to produce a 1.44MB floppy image
TIMES 1024*1440-($-$$) db 0x00

To use this you would first generate a binary file called stage2.bin. After stage2.bin has been built you can build a 1.44MiB disk image without a BIOS Parameter Block (BPB) with this command:

nasm -f bin boot.asm -o disk.img

To build a 1.44MiB disk image with a BPB you can build it with this command:

nasm -DWITH_BPB -f bin boot.asm -o disk.img

The code in stage2.bin would have to be generated with the assumption that the ORG (origin point) is 0x07e00 in memory.

---

Sample Usage/Example

An example of code generated to a file called stage2.bin that can be loaded with this test harness:

testcode.asm:

ORG 0x7e00

start:
    mov si, testCodeStr
    call print_string

    cli
.end_loop:
    hlt
    jmp .end_loop

testCodeStr: db "Test harness loaded and is executing code in stage2!", 0

; Function: print_string
;           Display a string to the console on display page 0
;
; Inputs:   SI = Offset of address to print
; Clobbers: AX, BX, SI

print_string:
    mov ah, 0x0e                ; BIOS tty Print
    xor bx, bx                  ; Set display page to 0 (BL)
    jmp .getch
.repeat:
    int 0x10                    ; print character
.getch:
    lodsb                       ; Get character from string
    test al,al                  ; Have we reached end of string?
    jnz .repeat                 ;     if not process next character
.end:
    ret

Note: there is an ORG 0x7e00 at the top. This is important. To assemble this file into stage2.bin use:

nasm -f bin testcode.asm -o stage2.bin

Then create the 1.44MiB disk image with:

nasm -f bin boot.asm -o disk.img

The result should be a disk image exactly 1.44MiB in size, contains a copy of stage2.bin and has our test harness boot sector.

The file stage2.bin can be anything that has binary code written to be loaded and started at 0x0000:0x7e00. The language (C, assembly etc) used to create the code in stage2.bin doesn't matter. I use NASM for this example. When this test code is executed in QEMU using qemu-system-i386 -fda disk.img it would look similar to this:

---

Special Note: : Using -DWITH_BPB to enable a BPB is useful if you are booting from USB using FDD emulation. Some BIOSes that boot USB as a floppy will assume a BPB is is present and overwrite the area with drive geometry before transferring control to it at physical address 0x07c00.

Keener

I've seen the "mov sp, 0x7c00" in other bootloaders, in fact on some random USB flash drive I inspected. At first it looks very clever, but are there any robustness issues with this? In particular, what if an interrupt comes in one or two cycles after sp is updated? Won't the push all of the interrupt handling trash the load_stage2 code? Or if in the worst case we get a couple nested interrupts? Would cli help here? Why economize on RAM this way that leaves a window open for a race condition? Or are there guarantees about no interrupts at this stage? Inquiring minds want to know! – Thun

@Thun : a side effect of updating the SS segment register is that the CPU turns off interrupts until after the instruction following the SS update. On a single core this has the effect of making the update of SS and SP atomic. This only works if the update to SS is immediately proceeded by the SP update. The only known situation this would fail is on a defective 8088 processor from the 1970s and 1980s (see pcjs.org/documents/manuals/intel/8086) – Keener

@Thun After SS:SP have been updated the stack grows down from 0x0000:0x7c00. When you do a push (manually or by an interrupt) the SP register is decremented first before data is placed on the stack. The stack doesn't grow up to clobber any data at addresses 0x7c00 or above. – Keener

You beat me to it. I remembered that stack grows down just now and ran to delete my newbie comment, but you've answered (thanks!) so I'll leave it. Yes I knew about the 1-instruction interrupts delay, but I was speaking to the stack growing into the code, which it obviously doesn't :). – Thun

Answer 2

I modified my own boot sector loader to add a new protocol. It makes it set es = ds = ss = 0 and load the whole load file to address 07E00h, jumping to that at 0000h:7E00h. However, sp is left pointing somewhat below 7C00h.

And there's the big difference to the requirements in the question: This loader uses the (FAT12 or FAT16) filesystem to load the next stage. It loads from a file named KERNEL7E.BIN if found. The file name, like the entire load protocol, can be adjusted by editing the source file or passing defines on the NASM command line.

A limitation due to the code size is that only single-character error messages are output when an error occurs: R means disk Read error, M means the file to be loaded is too large (out of Memory). Another limitation is that the RPL (Remote Program Loader) protocol isn't used as it needs some more bytes.

To lessen the space pressure, the loader can be built with -D_CHS=0 -D_QUERY_GEOMETRY=0 (if to load via ROM-BIOS's LBA interface) or -D_LBA=0 (if to load via CHS interface).

To build the loader, clone the lmacros and ldosboot repositories, and put them next to each other. The loader is to be built from the ldosboot directory with NASM this way for FAT12:

$ nasm -I ../lmacros/ boot.asm -l boot7e12.lst -D_MAP=boot7e12.map -o boot7e12.bin -D_COMPAT_KERNEL7E

Or this way for FAT16:

$ nasm -I ../lmacros/ boot.asm -l boot7e16.lst -D_MAP=boot7e16.map -o boot7e16.bin -D_FAT16 -D_COMPAT_KERNEL7E

Here's how to install the loader into an existing already-formatted FAT12 or FAT16 file system image:

dd if=boot7e12.bin of=floppy.img bs=1 count=11 conv=notrunc
dd if=boot7e12.bin of=floppy.img bs=1 count=$((512 - 0x3e)) seek=$((0x3e)) skip=$((0x3e)) conv=notrunc

Instead of using an existing image, an entire image can be created by NASM. I wrote such a program at https://hg.ulukai.org/ecm/bootimg It builds like this:

nasm -I ../lmacros/ -D_BOOTFILE="'../ldosboot/boot12.bin'" -D_MULTIPAYLOADFILE="'../ldebug/bin/ldebug.com','../ldebug/bin/lddebug.com'" bootimg.asm -o bootimg.img

Note how the long def has double quotes around single-quoted list entries. Each list entry is stripped to the basename (after the last slash or backslash), has its content added to the data area, and has a directory entry added to the root directory. Filenames are ASCII and in allcaps.

The ldosboot repo contains a two-sector FAT32 loader too, but I didn't modify it to support this protocol yet. With relocation, the FAT buffer should be at the top of memory already. That means the file can be loaded to 07E00h. However, ss will be at a high segment instead of zero. Other than that difference, the protocol can be specified with switches. The command to build this is nasm -I ../lmacros/ boot32.asm -l boot7e32.lst -D_MAP=boot7e32.map -o boot7e32.bin -D_RELOCATE -D_MEMORY_CONTINUE=0 -D_ZERO_DS -D_ZERO_ES -D_SET_BL_UNIT=0 -D_SET_DL_UNIT=1 -D_LOAD_ADR=07E00h -D_EXEC_SEG_ADJ=-7E0h -D_EXEC_OFS=7E00h -D_OEM_NAME="'KERNEL7E'" -D_LOAD_NAME="'KERNEL7E'" -D_LOAD_EXT="'BIN'"

There's also the instsect program (in its own repo) for DOS, which is built with loader images and installs them to a DOS drive.

Subdiaconate

I only briefly summed it up as common boilerplate/template would allow other Stack Overflow users wishing to help to test the code with a limited amount of fuss. at the bottom of my question. The target wasn't really for OS design but a method to help people potentially create better minimal reproducible example when presenting their code on this site as many people doing OSDev neglect to show us their bootloaders. – Keener

I'm the upvote. I probably should add to my question the reasons why I had a specification that didn't involve using a file system. On Stackoverflow we get questions that span OSes (especially Windows and Linux). The spec was designed in such a way that simply having NASM could produce a floppy image that was usable without requiring more tools (to mount and unmount images - much more involved on Windows). Much easier in userspace on Linux with mtools. I'm actually partial to directing people to Alexey Frunze's (another SO user) BOOTPROG as well: github.com/alexfru/BootProg – Keener

@Michael Petch: I did actually work with alexfru's loader implementation too when I started developing the ldosboot collection. However, I eventually decided to use Chris Giese's. As for your requirements, I understand that the formatting and accessing of an image isn't always easy. Still I want to advertise my loader as an alternative. Thanks for the upvote! – Subdiaconate

Understood, there was nothing wrong with your answer which was why I was more than happy to upvote. I too have bootloaders that will read Fat12 etc. My first bootloader was in the late 1980s (I was doing work with Wendin DOS) and back then besides asking for help on places like Compuserv there wasn't a plethora of example bootloaders to draw inspiration from except to disassemble DOS ones. – Keener

@Michael Petch: I added a program (script ish) that uses NASM to build a file system image. I added the description to my answer. It lacks sophistication, but is certainly a sufficient alternative to your solution for anyone wanting to use only NASM. – Subdiaconate