Lightweight method to use Amd64 instructions under 32-bit Windows?

Question

For some CPU-bound code using 64-bit variables, it is beneficial to use the Amd64 instruction set rather than x86. How can it be done under 32-bit Windows (e.g. Windows XP SP3)? Of course I assume a modern, Amd64-enabled CPU. I'm excluding the working but heavyweight method: running a full-blown 64-bit OS as a virtual machine, e.g. Ubuntu for Amd64 under Virtualbox.

I understand some assembly is needed, and there will restrictions, in particular addressing more memory than 32-bit Windows manages. But I'm thinking of purely computational tasks needing only a moderate amount of memory and no call to external functions.

Selfrising

I think this won't be easy... you have to hook the scheduler in some way, to make make the CPU go in 64 bit mode when the scheduler gives the control of the CPU to your code, and then switch it back to 32 bit mode when the scheduler wants the CPU back. This surely can't be done in user mode (you need to circumvent the whole point of preemptive multitasking). – Merola

possible duplicate of Start x86_64 code on x86 (32bit) Linux, running on x86_64 CPU – Ingate

Answer 1

There is no way to use Amd64 instructions (Long mode) in 32-bit general-purpose OS (without kernel modification/special drivers/hypervisor).

This is because:

1) to use native 64-bit instructions you need to switch into long mode. This is privileged action. 32-bit OS kernel can't continue to work if the CPU is switched into 64-bit mode, so you should switch back before entering a kernel

2) But kernel are often called asynchronously, for timer (scheduler) and other hardware interrupts (drivers). It will not save 64-bit registers nor change mode from long into protected.

May be it is possible to write special driver, which will do the 64-bit tasks on 32-bit OS, but such driver is more like 64-bit kernel and dynamic patcher of kernel. I know no one such solution.

You can only use MMX, SSE, SSE2, SSE3, AVX for accessing 64-bit ALU and registers of your CPU when running in 32-bit OS.

I can say, that Linux, some BSD, Mac OS X have a mode, when 64-bit kernel is used, but user-space software is 32-bit. In such case it will be possible to run both 32-bit and 64-bit applications, because kernel knows about 64-bit mode and can access 64-bit registers to do a task switch. As far as I know, MS Windows have not such mode itself (the W7 emulates 32bit mode, but this is called my MS as simulator so I assume it is not an in-kernel feature).

Other possibility (this is better, is your CPU has support of hardware virtualization), is to use 64-bit hypervisor (VMware/Xen, other overpriced solutions) with 32-bit and 64-bit guest OSes. VirtualBox is other option of using hypervisor, and it is free to use.

Ingate

Note also that REX prefixes are interpreted as valid (INC or DEC) instructions in compatibility mode. – Epithelium

He can't use 64-bit instructions because the CPU is not in 64-bit mode (long mode). – Ingate

On the other hand, such "special driver" probably can be written: otherwise, how do VMWare & co. run 64-bit guests on 32-bit hosts running on 64-bit processors? ---EDIT--- VT-x/AMD-V seems to be the answer – Merola

Matteo Italia, the answer is 64 bit hypervisor. – Ingate

OK, kernel support is needed to enter long mode (I was not even sure of that). Does it exist natively on XP SP3 running on Amd64 CPUs, or do we need to add it?. Either way, Virtualbox runs a 64-bit OS on such 32-bit hosts, and thus includes that trick somewhere in its source. If all I want is run isolated Amd64 functions having zero interaction with hardware peripherals and only memory managed by the 32-bit OS, I guess some lighweight solution is possible. – Selfrising

fgrieu, 1) to use amd64-instructions you need to enter Long Mode 2) After entering Long mode, CPU changes assumptions about how memory is managed by OS (page table has other format). 3) 32-bit XP knows nothing about amd64-bit page table. 32-bit XP knows nothing about how drivers and interrupts will work in long mode. – Ingate

fgrieu, Virtual box can do this (host is 32-bit and guest is 64bit) ONLY IF: virtualbox.org/manual/ch03.html#intro-64bitguests "64-bit processor with hardware virtualization support" - so this IS THE hypervisor solution, described briefly in my answer. hypervisor in this case is the very small 64-bit OS, which is started by VBOX driver. It manages memory and routes/translates interrupts, schedule 64-bit and 32-bit tasks (do a long/protected mode switches), both privileged and unprivileged. Also this mode (see p.3 from link) "incurs additional overhead,". – Ingate

fgrieu, the source is VBox/VMM. I able to identify mode switcher here virtualbox.org/browser/trunk/src/VBox/VMM/VMMSwitcher/… VMM - is the source of hypervisor (VMM = Virtual Machine Monitor; term of intel's hardware virtualisation). – Ingate

osgx, many thanks. So I need a CPU with hardware support for visualization, and a simplified hypervisor; and you pointed an example. Lots of work, I'm surprised there is no out-of-the box solution. – Selfrising

fgrieu, if you cant afford a true 64-bit OS as primary OS on your PC - then yes. – Ingate

Answer 2

In general, running 64-bit code in a 32-bit OS kernel is going to be next to impossible, for the following reasons:

• The 32-bit OS is unaware of the additional 64-bit registers (and upper 32-bits of the existing registers) and will not save them across task switches
• The 32-bit OS is not prepared to enable 64-bit code execution. Enabling 64-bit code execution means switching to IA-32e paging (which requires an entirely different page table format) and setting CS.L = 1 and CS.D = 0 in the code segment descriptor in the GDT (or LDT). (See the IA-32 manuals, vol 3a/3b 5.2.1)

In principle, you may be able to workaround both problems by writing a new HAL for Windows, which operates in IA-32e mode, and switches to a 64-bit trampoline code segment to save and restore 64-bit registers. This is a rather complex task; take a look at the Windows DDK for details. You could also use an emulation approach, as VirtualBox and friends do, if your CPU supports VMX. But it would be much simpler to simply use a 64-bit OS from the start.

Epithelium

VMX is not an emulation, it is hardware-assited virtualization. – Ingate