get the physical address of a buffer under Linux

Question

I am running Linux kernel 3.3 on Xilinx's Microblaze with full MMU. the task that I am doing requires me to know the following: I need to create a text file (buffer) and locate the physical address of this buffer and I don't want the kernel to write this file into discontinuous regions of memory.

the reason I need this because I have a DMA engine that streams data from a preset physical memory address, so I need to force Linux to create the buffer file at that exact memory location, so that when I write data into this file its immediately transmitted by the DMA Engine to another Hardware core

More details:

my system has a 512 MB DDR3 RAM connected to the system via "Xilinx' multi port memory controller (MPMC). the base address of this memory controller is 0x90000000, all units in the system access memory through this controller, including MicroBlaze, The DMA unit that I have uses a special interface called Native Personality Interface (NPI) to communicate with memory at a very low level, thus resulting a very high speed performance.

This NPI DMA unit was originally designed to be utilized under a very basic embedded kernel called "xilkernel" which did not support virtual memory, neither MMU was part of MicroBlaze, so the programmer could see where the OS code will reside and select a physical memory address such as 0x91800000 as the source address which DMA will stream from, then the programmer can place a file in that exact address and run the system

when we needed to the migrate the project to use Linux instead of xilkernel we ran into this issue, I have files on an external storage device which I can access as block device from Linux and I need move each file to main memory (DDR3 RAM) and make the DMA stream the file. currently the DMA streams from a fixed address but I can make it generic if needed.

Bernardinabernardine

Take a look at this PDF of a chapter that explains how the Linux kernel manages virtual and physical memory. It gives examples of how to map and pin physical memory for various scenarios. lwn.net/images/pdf/LDD3/ch15.pdf – Cashman

@Bernardinabernardine Did you develop this device yourself or is it a 3rd party IP core that you can't modify? If you can modify it, then I highly recommend that you make the DMA address software configurable so that you can use the standard DMA functions which tian_yufen talks about. – Yatzeck

@BenjaminLeinweber, yes I have the source code for the DMA core and I can configure the address that DMA streams from and make it a generic parameter. – Bernardinabernardine

https://mcmap.net/q/357191/-how-to-find-the-physical-address-of-a-variable-from-user-space-in-linux – Rosauraroscius

Answer 1

To handle buffers to interface with DMA controller, there are specific functions. These functions take care of not only the address translation but also the cache coherency with memory such as cache flush(write data to memory before sending) and cache invalidate(invalidate the cache before receiving).

(1) To allocate a buffer, get both the virtual address and physical address:

void *dma_alloc_coherent(struct device *dev, size_t size,
                         dma_addr_t *dma_handle, gfp_t flag)

the return value of the function is the virtual address of the buffer allocated, while the dma_handle pointer saves the physical address of the buffer allocated.

(2) To pass one buffer allocated to device:

dma_addr_t dma_map_single(struct device *dev, void *ptr,
                          size_t size,
                          enum dma_data_direction dir)

The return value is physical address of the buffer, and parameter dir is DMA_TO_DEVICE, ptr is the virtual address of the buffer;

(3) To receive one buffer from device:

void dma_unmap_single(struct device *dev, dma_addr_t addr,
                            size_t size,
                            enum dma_data_direction dir)

The parameter dir is DMA_FROM_DEVICE.

Note: To use the three functions related to dma, one should register a device to one specific bus which has the dma_map_ops, or else these three functions can not be used.

Petrol

This does not answer the OP's question. The OP wants to create a dma buffer at a fixed location in memory. None of these APIs are able to do this. – Yatzeck

Answer 2

I need to force Linux to create the buffer file at that exact memory location

This is not possible. (Actually you have created an XY question.)

Since you have hardware that "streams data from a preset physical memory address", then you have to ensure that the Linux kernel does not use this memory region as part of its memory pool. You need to inform the kernel when it boots to not use this memory region. You will not be able to "reclaim" or allocate buffers in this specific physical memory region once it becomes part of the memory space controlled by the kernel.

The most generic method to exclude a memory region is to use the memmap= parameter on the kernel command line.

memmap=nn[KMG]$ss[KMG]
        [KNL,ACPI] Mark specific memory as reserved.
        Region of memory to be used, from ss to ss+nn.
        Example: Exclude memory from 0x18690000-0x1869ffff
                 memmap=64K$0x18690000
                 or
                 memmap=0x10000$0x18690000

Some architectures, such as ARM with its ATAGs, have other less visible and more secure methods of reserving regions of physical memory.

Somehow you then have to provide the address and size of this memory region to the device driver. This could be obtained by parsing the command line, or (thumbs down) hardcoded using #defines.

The driver should declare its use of the memory region by calling request_mem_region().
The driver can map this memory region into virtual address space by calling ioreamp().

Since the driver is provided or knows the physical address already, that's done. Since physical memory was allocated, the memory is therefore contiguous. You will have to configure the MMU to disable caching on this memory region. The memory region will then be "DMAable".

Telemachus

+1 for the direct answer to the question. Another similar method is to have the device use the top of memory and tell Linux that there is less memory than there actually is using the mem= boot parameter. For example, if you have 64 MB of memory, you can pass mem=63M to reserve a meg for your device starting from 0x3F00000. – Yatzeck

@Telemachus I did not understand how to "You will have to configure the MMU to disable caching on this memory region", I looked at MicroBlaze adjustable parameters and that option was not there. Is not setting memmap enough ? – Bernardinabernardine

@Bernardinabernardine - I included that as an afterthought for completeness. You probably do not have to do anything explicit. Hopefully the memmap= specification and the call to ioremap() for your arch/board/proc will setup the proper attributes for that memory region. – Telemachus

@Telemachus Thank you very much, I will attempt this solution and get back to this op and update it – Bernardinabernardine