Regarding Buffer interface in python

Question

I am quite confused with the term 'buffer Interface' in python. what does it mean to say that " A python object exposing its underlying memory structure' can someone explain with an example. Thanks in advance

Greensboro

I think It's important to understand that the word buffer doesn't imply the underlying memory structure is temporary or a part of a bigger chunk as in many other contexts. I guess it simply means a chunk of (underlying) memory – Nonce

Answer 1

"Underlying memory structure" refers to the sequence of octets that comprise the object in computer memory. For example, when you create the string "abc", Python must reserve at least 3 bytes of memory and store the letters a, b, and c. If the memory is contiguous (as is the case for strings), its address and size can be passed to any piece of C code that wants to inspect it without going through the overhead of the Python str type.

A useful example is the array type. An array is a sequence that works much like a Python list, with the difference that it contains single-typed elements - you can have an array of ints and an array of floats, but you can't mix the two. The benefit is that arrays pack data as efficiently as possible, storing them in a flat C array. This array is exposed through the buffer interface - it allows one to query the exact memory position and size of the underlying C array, and pass it to some C function that will efficiently initialize it or write it out to disk. The numpy numeric package is built around this kind of data sharing between Python and C (and even FORTRAN), for which they extended the buffer protocol, and some of these extensions made it into Python 3.

The mmap object, which provides a Python interface to OS-level memory mapping functionality, also provides a buffer interface. This enables C code that must efficiently access the memory, such as the re module, to also work with memory-mapped regions.

Hexa

Does it mean to say that every python object has an equivalent C representation which is used by python interpreter for processing. and we can access that C representation for objects which support buffer interface – Greensboro

@Greensboro Every Python object must be somehow represented in memory, and since CPython is implemented in C, the representation is ultimately based on C's data types. In some cases (arrays) it is possible and useful to expose this representation outside Python. In other cases this is either useless (as with lists, which are internally arrays of internal pointers to Python objects) or downright impossible (as with dicts, which are internally hash tables with fairly complex structures and as such cannot be represented as a single "buffer"). – Hexa

See also this explanation by Guido. – Hexa

Thanks a lot user4815162342 – Greensboro