Why is contiguous memory allocation is required in linux?

Question

GPUs and VPUs need contiguous memory.

CMA and Static memory allocation are the examples of contiguous memory.

Why is contiguous memory required here?

Angers

It depends on the specific device's driver, but probably has to do with performance reasons. – Enjambment

How it is related to performance? Can you please explain, how it depends on device driver. – Angers

Many devices work with a memory directly, ignoring MMU. Such devices use physical addresses for memory regions, thus require these regions to be physically contiguous. – Benzol

You're conflating contiguous virtual memory with contiguous physical memory. Memory allocations are always contiguous virtual memory. If your system has an IOMMU, you probably wouldn't be asking the question. – Hypnotism

I believe one of the reasons is DMA. The hardware requires a block of contiguous memory. Also, the graphics card use double or triple buffers. I imagine it would be very inefficient if a buffer could not be swapped by exchanging a pointer (as opposed to copying a fragmented buffer). – Tissue

@Tsyvarev, Why not physically discontinuous memory ? – Angers

@Hypnotism , Basic fundamental is virtually contiguous is not always equal to physically contiguous, in case of cma virtual contiguous == physical contiguous. so my question is why physically contiguous required ? what are the advantages or disadvantages ? – Angers

@jww, your answer is something that could be expected, but why hardware require contiguous memory ? – Angers

When configured, the device uses physical address of the start of the memory region, and the size of that memory region. These parameters adress only physically contiguous region. "why hardware require contiguous memory?" - Without specifying concrete type of the devices, this question would be too broad for Stack Overflow. – Benzol

@Tsyvarev, I know its too broad for stackoverflow, if possible i just want to know basics/fundamental behind that. – Angers

Actually, transforming from virtual addresses to physical ones (via MMU unit) is performed only for instructions, which are processed by CPU itself. GPU uses its own processing core for process GPU code, thus it simply cannot use MMU provided by normal CPU. No MMU - no virtual addresses. – Benzol

Answer 1

Contiguous memory allocation (CMA) is needed for I/O devices that can only work with contiguous ranges of physical memory. I/O devices that can only work with continuous ranges are built that way in order to simplify the design of the device.

On systems with an I/O memory management unit (IOMMU), this would not be an issue because a buffer that is contiguous in the device address space can be mapped by the IOMMU to non-contiguous regions of physical memory. Also some devices can do scatter/gather DMA (i.e., can read/write from/to multiple non-contiguous buffers). Ideally, all I/O devices should be designed to either work behind an IOMMU or should be capable of scatter/gather DMA. Unfortunately, this is not the case and there are devices that require physically contiguous buffers. There are two ways for a device driver to allocate a contiguous buffer:

• The device driver can allocate a chunk of physical memory at boot-time. This is reliable because most of the physical memory would be available at boot-time. However, if the I/O device is not used, then the allocated physical memory is just wasted.
• A chunk of physical memory can be allocated on demand, but it may be difficult to find a contiguous free range of the required size. The advantage, though, is that memory is only allocated when needed.

CMA solves this exact problem by providing the advantages of both of these approaches with none of their downsides. The basic idea is to make it possible to migrate allocated physical pages to create enough space for a contiguous buffer. More information on how CMA works can be found here.

Encumber

But why physically contiguous memory required ? – Angers

@PankajSuryawanshi The answer is right there in the first sentence. Which part of the answer is not clear? – Encumber

My question is why I/O devices that can only work with contiguous ranges of physical memory ? – Angers

@PankajSuryawanshi Because it leads to a simpler design where the device only has to deal with a single physical address instead of multiple physical addresses. – Encumber