Accessing ring 0 mode from user applications ( and why Borland allows this )

Question

As the semester's deadlines approach, I decided to start working on a project in Operating Systems course at my college. The problem with the project assignment is that it requires students to develop a user application (exe) that will execute as a simple kernel ( basic process and thread management ).

First thing that popped to my mind was : How the hell am I supposed to execute privileged code in user application?

After consulting with other students ( who did the project on time ), I learned that they were able to execute privileged code without problems using Borland 3.1 compiler. However, none of them found that weird nor knew why that worked. Why ( better question here would be how ) does Borland do this? Doesn't this violate fundamental principles of OS security?

Note: I added C++ tag because the project is supposed to be written as a C++ application, with most of the privileged code executed as inline assembly.

Update My question was somewhat poorly phrased originally. Of course I was able to compile code with privileged instructions with any compiler - running the code was the problem.

Motorcade

If you enter the root password you can do whatever you want (if you don't have a tpm module of course). Why would that be surprising? The whole os consists of applications with more or less privileges. – Midship

@Midship Sorry for not mentioning it, but this is a Windows project. – Motorcade

@Solarflare: There is a difference between running as an administrator, and executing privileged instructions. The latter gives you way more privileges (e.g. reading process memory of service applications). – Shults

That applies to windows too. If you enter the administrator password you can do whatever you want (if you don't have a tpm module of course). – Midship

Your "problem" is: You're thinking sane - you're thinking in Protected Mode environment. Borland C++ 3.1 Compiler is not necessarily like that: No Protected Mode => No (or fewer) problems accessing privileged instructions. – Liar

@Solarflare: No, it doesn't. An administrator cannot access resources protected by the local system account, for example. And the local system account cannot write to LDT's or GDT's. Code running in ring 0, on the other hand, can. – Shults

Why was this marked as duplicate. I've read the linked question before posting this one and found no clear answer to why (and how) Borland allows this? Also, as you can see from the question, I am NOT writing a driver. – Motorcade

Exactly what privileged instructions does your Borland C++ program need to execute? For compatibility with older software, Windows allows some of these to succeed in a user mode program, will emulate others, and will disallow others still. Also the exact mix of what's allowed or not allowed depends on what version on Windows you're talking about and I think may also depend on whether it's an x64 version of Windows or not. – Brigade

Borland 3.1 generated 16-bit code. Using privileged instructions was entirely common, such code did not run on a protected mode operating system yet. It was in fact required, the INT instruction was necessary to invoke DOS and BIOS functions. Today you would run such a program on a DOS emulator. Which emulates the behavior of these kind of privileged instructions by handling the processor fault. You are not exactly learning practical skills that are useful today, not uncommon in school projects. That's okay, you are learning how to learn. – Programme

@tellerath: Borland doesn't allow that, because it cannot allow that. Borland cannot - under no circumstances - circumvent protected mode. No one can (except hardware hackers ;-) because preventing that is the purpose of protected mode. Why "Remy Lebeau" marked your question as a duplicate? Because he can! He is gold in the c++ tag and decided to do so. – Liar

@MichaelBurr Well for example i need to be able to set interrupt flag or override interrupt handlers, which people were able to do just by executing inline assembly. Could you provide me with some resource on this emulation techniques, since I really have no idea about it. It's a Windows 7 (x86). – Motorcade

Sounds like you are still missing the "learning how to learn" goal. Your teacher intentionally picked a project that you cannot get help with on a site like this because it is no longer relevant to computing today. His goal is to force you to learn how to learn by researching this yourself. You'll have lots and lots of use for that in your career after school. – Programme

You may want to Google for information on "Virtual 8086" mode, which is what is largely used on Windows to support 16-bit applications. Virtual 8086 mode is not supported on x64 versions of Windows, so what surprises you as working on your Windows machine will probably not work at all on a Win x64 machine. – Brigade

@HansPassant Well in many similar instances, I would say that this was caused by obsolete materials and subjects, but since the professor who gave us the assignment is highly respected and teaches some rather advanced and modern concepts as well, I will agree with you on that point. However, I might have put myself in an awkward position by not starting this project on time. I have learned some things from the comments and found a starting point so I hope I'll be able to fully answer my own question after I'm done with the research :) – Motorcade

I saw a comment saying you are using Windows 7 x86. The x86 (32-bit versions of Windows) still support NTVDM. NTVDM is a limited Virtual DOS environment that uses VM8086 mode on 386 processors. Your 16-bit C++ program when run on Windows is loading your program into an NTVDM session. Your program thinks it is running on real hardware (but it is virtualized).Privileged instructions (I assume things like CLI/STI) are trapped by the processor and simulated. – Systole

@Liar You can use priviledged code from borland even on nowadays OS without emulation. But you need an driver for that like DLPORTIO. This was commonly used as workaround for accessing things like LPT from Windows due to bug in the WinAPI making it impossible to access the way it should. – Smallman

@RossRidge The down-votes where clear in your favor... I deleted my answer – Smallman

Answer 1

Two things:

1. Back in the days of 8086 real mode there were no privilege levels. Borland 3.1 was a 16-bit compiler. If you're running code it produces on a 32-bit version of Windows NT, it will run in Virtual 8086 mode using the NTVDM, which also has no privilege levels.

2. Even when using a modern compiler / assembler, it generally won't complain about privileged instructions even in protected mode and long mode. This source code compiles just fine for me in MSVC 2015 but crashes whenever I run it because it tries to access a register that is off-limits to user-mode applications:

int  main()
{
    __asm
    {
        mov eax, cr0
    }
    return 0;
} 

Hujsak

Answer 2

The compiler allows it because the compiler's job is strictly to convert the input into compiled output. It's not designed to impose or enforce any system security rules. That's the job of the execution environment, typically the OS or emulator that executes the compiled code.

Simpleminded

Again, my question was somewhat poorly phrased. I was able to compile the code but not run it. – Motorcade

So what was your question then? – Simpleminded

I'm pretty sure the actual question was why Borland-produced 16bit executables can run privileged instructions. i.e. the question that Michael Petch answered in comments about NTVDM providing a virtual machine for 16bit executables. – Loess

@PeterCordes That's what I thought at first, but he said he wasn't able to run it. – Simpleminded

You mean the OP's comment on Govind's answer? He said he tried running code he built with MSVC, which will be 32 or 64bit. So he wasn't trying it from a 16bit executable, so no NTVDM. – Loess