PCI-e memory space access with mmap

Question

I'm using PCI-e port on Freescale MPC8308 processor (which is based on PowerPC architecture) and I have some problems when trying to use it. The endpoint PCI-e device has memory space equal to 256 MB. I can easily read and write configuration space of the endpoint device by using "pciutils" package.

After writing correct values in configuration registers and getting the permission to access the memory space; I tried to access memory space by using "mmap()" function in C and used the file descriptor located at :

"/sys/devices/pci0000:00/0000:00:00.0/resource0"

which was exactly 256 MB (equal to memory space of endpoint device) so it seems that I am using correct path for file descriptor. here you can find my code using "mmap()" as mentioned in https://github.com/billfarrow/pcimem:

https://github.com/billfarrow/pcimem/blob/master/pcimem.c

But unfortunately when I try to use memory space by using returned address of "mmap()" function; I cannot read the read-only registers of endpoint device correctly. Also, when I read addresses bigger than "0x7FFFFFC", the MPC8308 reboots. Considering above situation, do I miss any steps to initialize the PCI-e interface? Should I change anything in the Linux kernel image or U-Boot codes? Is there anything different for using PowerPC PCI-e with mmap()? Do you have any example code that can help me read PCI-e memory space?

Thanks

Tomkin

256MB of memory space for an endpoint seems to be too large. On my processor (i.MX6) I can't have endpoint larger than 16MB. Do you have an init error in Linux boot for PCIe ? What is your PCI-e memory device ? Is it an FPGA ? – Feverish

yes it also seemed too much to me, but I haven't any error in Linux booting and it has devoted 256 MB in memory for device as I can see in /proc/iomem. the end-point device isn't FPGA. Its an ASIC with PCI-e interface. – Tomkin

Another route would be by finding the device's base physical address and it's limit, and map in /dev/mem. (Honestly can't guarantee this will work. I'm curious if it does though.) – Ashlan

Answer 1

mmap() is a very useful but casual way to access PCIe devices from user space.

I notice that you pass 0 as the first argument to mmap. In my case of an FPGA card plugged into an x86 computer I make a call to lspci to get the physical address of the card in the pcie slot. Then I use that physical address as the first argument to mmap. I know you are writing the BAR's in config space of the device but maybe double check with lspci.

$ sudo lspci -s 02:00 -v
02:00.0 Memory controller: Xilinx Corporation Device 8028
    Subsystem: Xilinx Corporation Device 0007
    Flags: bus master, fast devsel, latency 0, IRQ 11
    Memory at f7e00000 (32-bit, non-prefetchable) [size=1M]
    Capabilities: [80] Power Management version 3
    Capabilities: [90] MSI: Enable- Count=1/1 Maskable- 64bit+
    Capabilities: [c0] Express Endpoint, MSI 00
    Capabilities: [100] Advanced Error Reporting

Belisle

The first argument to mmap determines the virtual address of the resulting mapping in the process that called mmap. Why would you need to set it to the physical address of the device's BAR? – Gink