I have read that converting a function pointer to a data pointer and vice versa works on most platforms but is not guaranteed to work. Why is this the case? Shouldn't both be simply addresses into main memory and therefore be compatible?
Why are function pointers and data pointers incompatible in C/C++?
Asked Answered
An architecture doesn't have to store code and data in the same memory. With a Harvard architecture, code and data are stored in completely different memory. Most architectures are Von Neumann architectures with code and data in the same memory but C doesn't limit itself to only certain types of architectures if at all possible.
Also, even if code and data are stored in the same place in physical hardware, software and memory access often prevent running data as code without operating system "approval". DEP and the like. –
Atrioventricular
At least as important as having different address spaces (maybe more important) is that function pointers may have a different representation than data pointers. –
Cogency
You don't even have to have a Harvard architecture to have code and data pointers using different address spaces - the old DOS "Small" memory model did this (near pointers with
CS != DS). –
Teplitz even modern processors would struggle with such mixture as the instruction and data cache are typically handled separately, even when the operating system allows you to write code somewhere. –
Megrims
Windows 8 specifically requires processors that provide code execution protection. It will not install without that CPU feature. That makes it impossible for user mode data to execute as code. –
Tipsy
@EricJ. Until you call
VirtualProtect, which allows you to mark regions of data as executable. –
Maletta @DietrichEpp: Apparently not quite that simple, but you're right, bypassing is apparently not that hard. Thanks for mentioning that. vulnfactory.org/blog/2011/09/21/… –
Tipsy
@Teplitz Not only the small memory model did have this, you also have the medium and compact models where one pointer is near and the other is far. On small model you're at least guaranteed to be able to cast it back to it's original type without loss of data. Then you have huge model where data pointers are huge but code pointers are only far - whether the compiler converts between these correctly I don't know (but if not done correctly you would end up pointing somewhere else). –
Hummel
Some computers have (had) separate address spaces for code and data. On such hardware it just doesn't work.
The language is designed not only for current desktop applications, but to allow it to be implemented on a large set of hardware.
It seems like the C language committee never intended void* to be a pointer to function, they just wanted a generic pointer to objects.
The C99 Rationale says:
6.3.2.3 Pointers
C has now been implemented on a wide range of architectures. While some of these architectures feature uniform pointers which are the size of some integer type, maximally portable code cannot assume any necessary correspondence between different pointer types and the integer types. On some implementations, pointers can even be wider than any integer type.The use of
void*(“pointer tovoid”) as a generic object pointer type is an invention of the C89 Committee. Adoption of this type was stimulated by the desire to specify function prototype arguments that either quietly convert arbitrary pointers (as infread) or complain if the argument type does not exactly match (as instrcmp). Nothing is said about pointers to functions, which may be incommensurate with object pointers and/or integers.
Note Nothing is said about pointers to functions in the last paragraph. They might be different from other pointers, and the committee is aware of that.
The standard could make them compatible without messing with this by simply making the data types the same size and guaranteeing that assigning to one and then back will result in the same value. They do this with void*, which is the only pointer type compatible with everything. –
Kalisz
@CrazyEddie You cannot assign a function pointer to a
void *. –
Dumont @Crazy Eddie: So,
void* is compatible with function pointers? I thought ALL non-function pointers were incompatible with function pointers, including void* –
Xanthous This does not really answer the question. It instead just says that it is not guaranteed to work which is something we already know. What about separate address spaces makes a difference? For instance with the old 8086 chip there were two kinds of pointers, near and far due to the segmented memory addressing used and one had to choose whether to how data and functions were accessed by selecting a memory model as the compiler target. So you could have incompatible function and data pointers. –
Underpainting
I could be wrong on void* accepting function pointers, but the point remains. Bits are bits. The standard could require that the size of the different types be able to accomodate the data from each other and the assignment would be guaranteed to work even if they are used in different memory segments. The reason this incompatibility exists is that this is NOT guaranteed by the standard and so data can be lost in the assignment. –
Kalisz
@CrazyEddie: Aside of all the platform specific issues, it discourages bad programming style. –
Shaman
But requiring
sizeof(void*) == sizeof( void(*)() ) would waste space in the case where function pointers and data pointers are different sizes. This was a common case in the 80's, when the first C standard was written. –
Softboiled @CrazyEddie it is not only not guaranteed but also not allowed by the C Standard. C says you can only convert function pointers to function pointers, not to object or
void pointers. –
Dumont @CrazyEddie: the standard could have required that function pointers be convertible to
void* and back, but doesn't. Presumably since there is pretty much no support in C for treating functions as objects, there was no reason for the committee to add potential overhead to object pointers in order to let them hold function pointers portably. –
Cogency Exactly Rob, hence my answer :P –
Kalisz
@RichardChambers: The different address spaces may also have different address widths, such as an Atmel AVR that uses 16 bits for instructions and 8 bits for data; in that case, it would be hard converting from data (8 bit) to function (16 bit) pointers and back again. C's supposed to be easy to implement; part of that ease comes from leaving data and instruction pointers incompatible with each other. –
Preachy
@JohnBode That round-trip is trivial, assuming no pointer-safety. But starting from the bigger type runs into trouble with compressibility. –
Stogy
For those who remember MS-DOS, Windows 3.1 and older the answer is quite easy. All of these used to support several different memory models, with varying combinations of characteristics for code and data pointers.
So for instance for the Compact model (small code, large data):
sizeof(void *) > sizeof(void(*)())
and conversely in the Medium model (large code, small data):
sizeof(void *) < sizeof(void(*)())
In this case you didn't have separate storage for code and date but still couldn't convert between the two pointers (short of using non-standard __near and __far modifiers).
Additionally there's no guarantee that even if the pointers are the same size, that they point to the same thing - in the DOS Small memory model, both code and data used near pointers, but they pointed to different segments. So converting a function pointer to a data pointer wouldn't give you a pointer that had any relationship to the function at all, and hence there was no use for such a conversion.
Re: "converting a function pointer to a data pointer wouldn't give you a pointer that had any relationship to the function at all, and hence there was no use for such a conversion": This doesn't entirely follow. Converting an
int* to a void* give you a pointer that you can't really do anything with, but it's still useful to be able to perform the conversion. (This is because void* can store any object pointer, so can be used for generic algorithms that don't need to know what type they hold. The same thing could be useful for function pointers as well, if it were allowed.) –
Tatiania @ruakh: In the case of converting the
int * to void *, the void * is guaranteed to at least point to the same object as the original int * did - so this is useful for generic algorithms that access the pointed-to object, like int n; memcpy(&n, src, sizeof n);. In the case where converting a function pointer to a void * doesn't yield a pointer pointing at the function, it isn't useful for such algorithms - the only thing you could do is convert the void * back to a function pointer again, so you might as well just use a union containing a void * and function pointer. –
Teplitz @caf: Fair enough. Thanks for pointing that out. And for that matter, even if the
void* did point to the function, I suppose it would be a bad idea for people to pass it to memcpy. :-P –
Tatiania Copied from above: Note what POSIX says in Data Types: §2.12.3 Pointer Types. All function pointer types shall have the same representation as the type pointer to
void. Conversion of a function pointer to void * shall not alter the representation. A void * value resulting from such a conversion can be converted back to the original function pointer type, using an explicit cast, without loss of information. Note: The ISO C standard does not require this, but it is required for POSIX conformance. –
Triode @Teplitz If it just should be passed through to some callback which knows the proper type, I'm only interested in round-trip safety, not any other relationship those converted values might possibly have. –
Stogy
Pointers to void are supposed to be able to accommodate a pointer to any kind of data -- but not necessarily a pointer to a function. Some systems have different requirements for pointers to functions than pointers to data (e.g, there are DSPs with different addressing for data vs. code, medium model on MS-DOS used 32-bit pointers for code but only 16-bit pointers for data).
but then should'nt the dlsym () function be returning something other than a void *. I mean, if the void * is not big enough for the function pointer, arn't we already fubared? –
Fribble
@Knickerkicker: Yes, probably. If memory serves, the return type from dlsym was discussed at length, probably 9 or 10 years ago, on the OpenGroup's email list. Offhand, I don't remember what (if anything) came of it though. –
Bullivant
you're right. This seems a fairly nice (although outdated) summary of your point. –
Fribble
@KnickerKicker: Yes, ideally there would be a separate
dlsym_function() for returning function symbols, or dlsym() would return a union {} that contains both a void * and a void (*)(). –
Teplitz @LegoStormtroopr: Interesting how 21 people agree with the idea of up-voting, but only about 3 have actually done so. :-) –
Bullivant
What is the "idea of up-voting"? –
Nerves
@pmor: The point was that many people had up-voted the comment saying "+1 for answering the question before it was asked", but only a few had actually up-voted the answer itself. –
Bullivant
In addition to what is already said here, POSIX requires pointers to functions with external linkage to be convertible to void* in the API of dlsym:
The ISO C standard does not require that pointers to functions can be cast back and forth to pointers to data. Indeed, the ISO C standard does not require that an object of type
void*can hold a pointer to a function. Implementations supporting the XSI extension, however, do require that an object of typevoid*can hold a pointer to a function. The result of converting a pointer to a function into a pointer to another data type (exceptvoid*) is still undefined, however. Note that compilers conforming to the ISO C standard are required to generate a warning if a conversion from avoid*pointer to a function pointer is attempted as in:
fptr = (int (*)(int))dlsym(handle, "my_function");
Due to the problem noted here, a future version may either add a new function to return function pointers, or the current interface may be deprecated in favor of two new functions: one that returns data pointers and the other that returns function pointers.
In other words:
- ISO C standard does not require that pointers to functions can be cast back and forth to pointers to data. Meaning that such casts are well-formed C code, but with non-portable and undefined effects, hence the requirement for compiler warning.
- POSIX requires that for
dlsym. - System V platform-specific ABIs (Mac OS, Linux, e.g. System V ABI for AMD64) in its ABI type
Pointerdefinition require that both function and data pointers have the same specific size and alignment.
does that mean that using dlsym to get the address of a function is currently unsafe? Is there currently a safe way to do it? –
Xanthous
It means that currently POSIX requires from a platform ABI that both function and data pointers can be safely cast to
void* and back . –
Snowblink @Xanthous It means that implementations that are POSIX compliant have made an extension to the language, giving an implementation-defined meaning to what is undefined behavior per the standard intself. It's even listed as one of the common extensions to the C99 standard, section J.5.7 Function pointer casts. –
Palette
@DavidHammen It is not an extension to the language, rather a new extra requirement. C doesn't require
void* to be compatible with a function pointer, whereas POSIX does. –
Snowblink C++11 has a solution to the long-standing mismatch between C/C++ and POSIX with regard to dlsym(). One can use reinterpret_cast to convert a function pointer to/from a data pointer so long as the implementation supports this feature.
From the standard, 5.2.10 para. 8:
converting a function pointer to an object pointer type or vice versa is conditionally-supported.
1.3.5 defines "conditionally-supported" as a:
program construct that an implementation is not required to support.
One can, but one shouldn’t. A conforming compiler must generate a warning for that (which in turn should trigger an error, cf.
-Werror). A better (and non-UB) solution is to retrieve a pointer to the object returned by dlsym (i.e. void**) and convert that to a pointer to function pointer. Still implementation-defined but no longer cause for a warning/error. –
Weinrich @KonradRudolph: Disagree. The "conditionally-supported" wording was specifically written to allow
dlsym and GetProcAddress to compile without warning. –
Pelagia @Pelagia What do you mean, “disagree”? Either I’m right or wrong. The dlsym documentation explicitly says that “compilers conforming to the ISO C standard are required to generate a warning if a conversion from a void * pointer to a function pointer is attempted”. This doesn’t leave much room for speculation. And GCC (with
-pedantic) does warn. Again, no speculation possible. –
Weinrich Follow-up: I think now I understand. It’s not UB. It’s implementation-defined. I’m still unsure whether the warning must be generated or not – probably not. Oh well. –
Weinrich
@KonradRudolph: I disagreed with your "shouldn't", which is an opinion. The answer specifically mentioned C++11, and I was a member of the C++ CWG at the time the issue was addressed. C99 indeed has different wording, conditionally-supported is a C++ invention. –
Pelagia
Depending on the target architecture, code and data may be stored in fundamentally incompatible, physically distinct areas of memory.
'physically distinct' I understand, but can you elaborate more on the 'fundamentally incompatible' distinction. As I said in the question, isn't a void pointer supposed to as large as any pointer type - or is that a wrong presumption on my part. –
Fribble
@KnickerKicker:
void * is large enough to hold any data pointer, but not necessarily any function pointer. –
Thorwald back to the future :P –
Blazon
undefined doesn't necessarily mean not allowed, it can mean that the compiler implementor has more freedom to do it how they want.
For instance it may not be possible on some architectures - undefined allows them to still have a conforming 'C' library even if you can't do this.
Another solution:
Assuming POSIX guarantees function and data pointers to have the same size and representation (I can't find the text for this, but the example OP cited suggests they at least intended to make this requirement), the following should work:
double (*cosine)(double);
void *tmp;
handle = dlopen("libm.so", RTLD_LAZY);
tmp = dlsym(handle, "cos");
memcpy(&cosine, &tmp, sizeof cosine);
This avoids violating the aliasing rules by going through the char [] representation, which is allowed to alias all types.
Yet another approach:
union {
double (*fptr)(double);
void *dptr;
} u;
u.dptr = dlsym(handle, "cos");
cosine = u.fptr;
But I would recommend the memcpy approach if you want absolutely 100% correct C.
They can be different types with different space requirements. Assigning to one can irreversibly slice the value of the pointer so that assigning back results in something different.
I believe they can be different types because the standard doesn't want to limit possible implementations that save space when it's not needed or when the size could cause the CPU to have to do extra crap to use it, etc...
The only truly portable solution is not to use dlsym for functions, and instead use dlsym to obtain a pointer to data that contains function pointers. For example, in your library:
struct module foo_module = {
.create = create_func,
.destroy = destroy_func,
.write = write_func,
/* ... */
};
and then in your application:
struct module *foo = dlsym(handle, "foo_module");
foo->create(/*...*/);
/* ... */
Incidentally, this is good design practice anyway, and makes it easy to support both dynamic loading via dlopen and static linking all modules on systems that don't support dynamic linking, or where the user/system integrator does not want to use dynamic linking.
Nice! While I agree this does seem more maintainable, it is still not obvious (to me) how I hammer on static linking on top of this. Can you elaborate? –
Fribble
If each module has its own
foo_module structure (with unique names), you can simply create an extra file with an array of struct { const char *module_name; const struct module *module_funcs; } and a simple function to search this table for the module you want to "load" and return the right pointer, then use this in place of dlopen and dlsym. –
Pappose @R.. True, but it adds maintenance cost by having to maintain the module structure. –
Fincher
A modern example of where function pointers can differ in size from data pointers: C++ class member function pointers
Directly quoted from https://blogs.msdn.microsoft.com/oldnewthing/20040209-00/?p=40713/
class Base1 { int b1; void Base1Method(); }; class Base2 { int b2; void Base2Method(); }; class Derived : public Base1, Base2 { int d; void DerivedMethod(); };There are now two possible
thispointers.A pointer to a member function of
Base1can be used as a pointer to a member function ofDerived, since they both use the samethispointer. But a pointer to a member function ofBase2cannot be used as-is as a pointer to a member function ofDerived, since thethispointer needs to be adjusted.There are many ways of solving this. Here's how the Visual Studio compiler decides to handle it:
A pointer to a member function of a multiply-inherited class is really a structure.
[Address of function] [Adjustor]The size of a pointer-to-member-function of a class that uses multiple inheritance is the size of a pointer plus the size of a
size_t.
tl;dr: When using multiple inheritance, a pointer to a member function may (depending on compiler, version, architecture, etc) actually be stored as
struct {
void * func;
size_t offset;
}
which is obviously larger than a void *.
On most architectures, pointers to all normal data types have the same representation, so casting between data pointer types is a no-op.
However, it's conceivable that function pointers might require a different representation, perhaps they're larger than other pointers. If void* could hold function pointers, this would mean that void*'s representation would have to be the larger size. And all casts of data pointers to/from void* would have to perform this extra copy.
As someone mentioned, if you need this you can achieve it using a union. But most uses of void* are just for data, so it would be onerous to increase all their memory use just in case a function pointer needs to be stored.
I know that this hasn't been commented on since 2012, but I thought it would be useful to add that I do know an architecture that has very incompatible pointers for data and functions since a call on that architecture checks privilege and carries extra information. No amount of casting will help. It's The Mill.
This answer is wrong. You can for example convert a function pointer to a data pointer and read from it (if you have permissions to read from that address, as usual). The result makes as much sense as it does e.g. on x86. –
Drum
© 2022 - 2024 — McMap. All rights reserved.
void. Conversion of a function pointer tovoid *shall not alter the representation. Avoid *value resulting from such a conversion can be converted back to the original function pointer type, using an explicit cast, without loss of information. Note: The ISO C standard does not require this, but it is required for POSIX conformance. – Triodevoid*conversions now applies only to values returned bydlsym(). – Beagledlsym()— note the end of the 'Application Usage' section where it says: Note that conversion from avoid *pointer to a function pointer as in:fptr = (int (*)(int))dlsym(handle, "my_function");is not defined by the ISO C standard. This standard requires this conversion to work correctly on conforming implementations. – Triode