segment limit check in AMD 64-bit mode

Question

I am writing my own OS for 64bit processors and I am stuck with the problem of general protection. My OS will not rely on page fault to implement user space protection mechanism, so I found there is a way to do it with segment limit checking:

This presentation from VMWare

http://download3.vmware.com/vmworld/2005/pac346.pdf

on page 20 says:

• Initial AMD64 architecture did not include segmentation in 64-bit mode

  • Segmentation also missing from EMT64T

  How do we protect the VMM ?

  • 64-bit guest support requires additional hardware assitance
  • Segment limit checks available in 64-bit mode on newer AMD processors

Now, I have the newer AMD processor model and my question is how do I achieve limit segment limit check on AMD processor in 64-bit (long) mode ? I have downloaded the Sep 2011 version (lastest) of developer's manual and I can't find how to do this in any place, please help.

Barimah

Answer 1

I think they're probably talking about the Long Mode Segment Limit Enable bit (LMSLE bit 13) in the "Extended Feature Enable Register" (EFER) in Volume 2 3.1.7 pg. 55. It is describe in a little more detail in "4.12.2 Data Limit Checks in 64-bit Mode" on page 114. Note that EFER is a model-specific register (more in "6.2.5 Accessing Model-Specific Register" pg. 156 also in volume 2).

Pedagogy

thanks. this is exactly the feature. but unfortunately I can't use it, because it only checks limits in first 4GB of memory. What I need is to check segment limits per process and it should work with addresses above 4GB. – Barimah

Unfortunately, as @PaulBetts answer also states, you're working against the machine (and you'll soon be raging) if you try to impose segmented memory access on your 64-bit OS. The limit check was probably added in response to VMWare and others wanting it for their purposes, not for OS writers. – Pedagogy

Answer 2

Segmentation is an old and very slow way to implement memory protection. Even when it came out, nobody used it because it was too slow - Intel invented this but didn't actually talk to OS vendors to see what they wanted first. You really need to use page faulting like other modern operating systems.

Seaborne

I think using paging will give much more performance penalty when page walk is done. It is better to have a segment and check for its limits, that can be done in 1 cycle. A page walk will incur in a DRAM access which is much worse and can evict data from cache – Barimah

@Nulik: Not all cycles are made alike. While the segment check might happen in "one cylce" in your use program that cycle might take a lot longer than 30 (or thousands) of other clock cycles that don't perform the same operation. Exactly like a mov eax, [addr] might only take N user clock cylces but millions of kernel cycles to page in. (It all comes down to trade offs. With segmenting you'd have to fix up pointers if you ever wanted to move code/data around). – Pedagogy

Nulik: The performance penalty happens when you have to constantly switch segments on every context switch, this is the slow part – Seaborne

@PaulBetts but as I understand, during context switch, the kernel has to save all the registers of the task, including DS. Or, does that mean that it only saves some registers? If so, what if my assembly program uses DS to store some auxiliary data, not as data pointer? The program would crash on first context switch. – Barimah

@Paul Betts some people on osdev forum pointed out that a context switch implies a TLB switch, but with segmentation you don't this, so I am doubting is paging is faster than segmentation. – Barimah

Yes, a context switch requires a TLB flush – Seaborne

Answer 3

Although this doesn't answer the question, this is related to removal of segment limit checks in 64 bit mode supposedly making protection of hypervisor trap handlers 'impossible' without hardware virtualisation, which is a discussion people may expect to see seeing this question title. 'breaking some existing implementations' I agree with, but not 'impossible'.

The initial version of x86-64 (AMD64) did not allow for a software-only full virtualization due to the lack of segmentation support in long mode, which made the protection of the hypervisor's memory impossible, in particular, the protection of the trap handler that runs in the guest kernel address space.

I see this all over but I'm not convinced. You don't need segmentation to protect hypervisor trap handlers or the IDT. You can do it with paging.

Make a certain virtual address range in the SPT always map the hypervisor trap handlers, and the IDT at the virtual address it is at on the host. The IDT requires 1 4KiB page. These pages in the SPT are made supervisor, meaning the guest kernel in ring 1 cannot write to them because it will cause a trap right into the IDT which is mapped into the guest. Now ring 0, the code can be executed. When the guest does read/write to those reserved virtual address ranges, it needs to be unaware that it is reserved, i.e. the hypervisor driver silently redirects the accesses with certain CR3s to a clean page. A read/write from a ring 1 guest kernel will cause a GPF, and the CR2 will contain the address that was attempted to be written to. Usually, the page is faulted in and then the RIP pushed is of the instruction to be reattempted, but it can't do this in this case. It needs to decode and perform the read/write itself at the RIP in the trap frame to a new host physical page by translating it using a special internal table purely for those reserved regions built when the guest modifies the PTEs, and then increase the RIP.

Lethargic