Qemu-KVM: Translation of guest physical address to host virtual/host physical address

Question

I am working on a project where I need to translate qemu-guest physical addresses to host virtual/physical addresses.

I am using VMI (virtual machine introspection) to introspect into the qemu process (the KVM VM) and to read guest physical addresses stored in virtio ring buffer descriptors. Therefore, I am looking for a simple way to translate the qemu physical addresses to host virtual addresses at the host side. (i.e., to extract as less info as possible from the qemu process).

I read online that in previous versions, qemu stored the physical RAM base in the variable phys_ram_base, so that the host virtual address could be obtained as follows:

host_virtual = phys_ram_base + guest_physical_address

Is something like this possible in newer versions of qemu (e.g., how could I obtain the qemu-physical base address -- the former phys_ram_base?)

Kelleher

Did you manage to answer your question? If yes, would you be so kind to share it with the community? Thanks! – Yockey

Answer 1

I had to solve the same problem: translating a guest-virtual address to a host-physical address. My approach was the following:

Step 1 (native host): virtual address to physical address

• I wrote a program vaddr2paddr that takes a process ID (PID) and a virtual address, then returns the associated physical address.
• This program uses /proc/<pid>/pagemap to determine the physical address of the program
• I derived the code from the blogpost Translating Virtual Addresses to Physical Addresses in User Space and the dwks/pagemap tool.
• I tested this script thoroughly on my native host with a simple test program that allocates a buffer and prints the virtual address, then waits for an input and in the meanwhile I run my vaddr2paddr script and then check using devmem2 tool (or with xxd in /dev/mem by loading devmem-full-access kernel module) whether my previously written data is indeed there

Step 2 (VM): guest-virtual address to host-physical address

• I run the same test program that allocates a buffer and writes some data into it in my VM
• I execute vaddr2paddr with the guest-virtual address of the allocated buffer to get the guest-physical address (gpa)
• On the native host, I determine the largest memory region of the qemu process by parsing /dev/<pid>/maps which shows the allocated virtual memory regions of a process. In my case, the VM had 2 GB of memory and I found an area that was roughly 2 GB (all others were significantly smaller). I then take the start address of that area (vm_start_address)
  • For that, I have written a simple Python script but there is also existing code on the web, e.g., ouadev/proc_maps_parser
• Now we can compute the host-physical address of our allocated buffer by: hpa = vm_start_address + gpa
• Again, I verify using devmem2 tool whether the written data is at the calculated hpa

NOTE: This approach requires sudo to access /proc/<pid>/pagemap and also /proc/<pid>/maps.

Chimpanzee

Answer 2

I had to solve the same problem and I come up with the following solution.

When using QEMU with the -enable-kvm option, memory is allocated to the guest through the KVM_SET_USER_MEMORY_REGION ioctl. Basically, QEMU prepares a kvm_userspace_memory_regionstruct, where the physical addresses of the guest are associated to host virtual addresses, and then the ioctl is issued. Now, it turned out that the KVMSlot struct is (almost) 1:1 with the struct offered by the KVM API. QEMU stores all the information to perform the translation from guest physical to host virtual addresses there.

The KVMSlot struct is defined like this:

    typedef struct KVMSlot
    {
        hwaddr start_addr;
        ram_addr_t memory_size;
        void *ram;
        int slot;
        int flags;
        int old_flags;
        /* Dirty bitmap cache for the slot */
        unsigned long *dirty_bmap;
    } KVMSlot;

start_addr is the physical address corresponding to the begininnig of the considered slot, ram is its corresponding host virtual address and then memory_size is the size of the slot.

Now to perform the translation you have to:

1. Find the right slot. Many slots can be allocated and they are kept in a list of KVMSlot elements. The head of the list is stored in KVMMemoryListener. To find it, you can check if the guest physical address is in the range between start_addr and start_addr + memory_size.
2. Compute the offset of your guest physical address in the slot (offset = gpa - start_addr)
3. Compute the translated host virtual address like hva = ram + offset. The offset of course is the same both in the guest physical addresses and in the host virtual addresses, that's why you can use it.

Finally you can check that the translation was right using the function gpa2hva of the QEMU Monitor.

Heiduc

Aren't you supposed to find a slot first, then find kvm_userspace_memory_region in that slot to find GPA2HVA translation? Also BTW, is KVMSlot existing for emulating physical memory slot(like DIMM)? – Astral

Yes as I said at point 1, you have to find the right slot. This can be done only by using KVMSlot, as QEMU only keeps track of them and not of kvm_userspace_memory_region. You can read in the source code: to actually set a slot, the QEMU function kvm_set_userspace_memory_region takes as parameter a pointer to KVMSlot, which is used to initialize a local kvm_userspace_memory_region struct, then you can see the ioctl call. Furthermore, as far as I know, KVMSlots aren't emulating anything, they are only a memory mapping. – Heiduc

Also see this – Heiduc

Could anyone explain how I can access the previously created KVMSlots structs? – Chimpanzee

Answer 3

hpa = vm_start_address + gpa didn't work for me straight of the box. There was a 2GB offset for the guest physical address inside my VM. The method that I used was combination of above solutions

1. first use /proc//pagemap to determine the physical address inside the VM as mentioned above in a solution (https://shanetully.com/2014/12/translating-virtual-addresses-to-physcial-addresses-in-user-space/)

2. Later I used qemu-monitor command: "virsh qemu-monitor-command VM-name --hmp gpa2hva " to get the physical address on the host. gpa2hva command using qemu-monitor gave the actual address of 0x7F2F03345000 which was a 2GB offset below the calculated HPA. Calculated HPA is basically ( vm_start_address + gpa). in my case the physical address inside the VM was: 0x000000012f545000 and vm_start_address( virtual address on host for the qemu thread) was at: 0x7f2e53e00000. So, HPA (host physical address) should have been at: vm_start_address + gpa = 0x7F2F83345000. But in actual it was at 0x7F2F03345000 i.e. 2GB offset below the calculated value.

3. Once I knew the host physical address, I used ptedit_pmap call using PTEditor driver ( https://github.com/misc0110/PTEditor) to map the physical address to my virtual address and read the data. Indeed it was the right location. I could use some explanation to understand why the 2GB offset was present inside the VM. I am guessing this is kernel mapping the main memory at various different locations of the address space.

Declarative