What are the applications of the ## preprocessor operator and gotchas to consider?

Question

As mentioned in many of my previous questions, I'm working through K&R, and am currently into the preprocessor. One of the more interesting things — something I never knew before from any of my prior attempts to learn C — is the ## preprocessor operator. According to K&R:

The preprocessor operator ## provides a way to concatenate actual arguments during macro expansion. If a parameter in the replacement text is adjacent to a ##, the parameter is replaced by the actual argument, the ## and surrounding white space are removed, and the result is re-scanned. For example, the macro paste concatenates its two arguments:

#define paste(front, back) front ## back

so paste(name, 1) creates the token name1.

How and why would someone use this in the real world? What are practical examples of its use, and are there gotchas to consider?

Mendy

Answer 1

CrashRpt: Using ## to convert macro multi-byte strings to Unicode

An interesting usage in CrashRpt (crash reporting library) is the following:

#define WIDEN2(x) L ## x
#define WIDEN(x) WIDEN2(x)
//Note you need a WIDEN2 so that __DATE__ will evaluate first.

Here they want to use a two-byte string instead of a one-byte-per-char string. This probably looks like it is really pointless, but they do it for a good reason.

 std::wstring BuildDate = std::wstring(WIDEN(__DATE__)) + L" " + WIDEN(__TIME__);

They use it with another macro that returns a string with the date and time.

Putting L next to a __ DATE __ would give you a compiling error.

---

Windows: Using ## for generic Unicode or multi-byte strings

Windows uses something like the following:

#ifdef  _UNICODE
    #define _T(x)      L ## x
#else
    #define _T(x) x
#endif

And _T is used everywhere in code

---

Various libraries, using for clean accessor and modifier names:

I've also seen it used in code to define accessors and modifiers:

#define MYLIB_ACCESSOR(name) (Get##name)
#define MYLIB_MODIFIER(name) (Set##name)

Likewise you can use this same method for any other types of clever name creation.

---

Various libraries, using it to make several variable declarations at once:

#define CREATE_3_VARS(name) name##1, name##2, name##3
int CREATE_3_VARS(myInts);
myInts1 = 13;
myInts2 = 19;
myInts3 = 77;

Bing

Since you can concatenate string literals at compile time, you could reduce the BuildDate expression to std::wstring BuildDate = WIDEN(__DATE__) L" " WIDEN(__TIME__); and implicitly build the whole string at once. – Ovi

Answer 2

One thing to be aware of when you're using the token-paste ('##') or stringizing ('#') preprocessing operators is that you have to use an extra level of indirection for them to work properly in all cases.

If you don't do this and the items passed to the token-pasting operator are macros themselves, you'll get results that are probably not what you want:

#include <stdio.h>

#define STRINGIFY2( x) #x
#define STRINGIFY(x) STRINGIFY2(x)
#define PASTE2( a, b) a##b
#define PASTE( a, b) PASTE2( a, b)

#define BAD_PASTE(x,y) x##y
#define BAD_STRINGIFY(x) #x

#define SOME_MACRO function_name

int main() 
{
    printf( "buggy results:\n");
    printf( "%s\n", STRINGIFY( BAD_PASTE( SOME_MACRO, __LINE__)));
    printf( "%s\n", BAD_STRINGIFY( BAD_PASTE( SOME_MACRO, __LINE__)));
    printf( "%s\n", BAD_STRINGIFY( PASTE( SOME_MACRO, __LINE__)));

    printf( "\n" "desired result:\n");
    printf( "%s\n", STRINGIFY( PASTE( SOME_MACRO, __LINE__)));
}

The output:

buggy results:
SOME_MACRO__LINE__
BAD_PASTE( SOME_MACRO, __LINE__)
PASTE( SOME_MACRO, __LINE__)

desired result:
function_name21

Coleville

For an explanation of this preprocessor behavior, see #8232466 – Carlinecarling

@MichaelBurr i was reading your answer & i have a doubt. How come this LINE is printing the line number? – Weinshienk

@AbhimanyuAryan: I'm not sure if this is what you're asking, but __LINE__ is a special macro name that is replaced by the preprocessor with the current line number of the source file. – Coleville

It would be cool if language specifications could be cited/linked, as in here – Cardinale

Answer 3

Here's a gotcha that I ran into when upgrading to a new version of a compiler:

Unnecessary use of the token-pasting operator (##) is non-portable and may generate undesired whitespace, warnings, or errors.

When the result of the token-pasting operator is not a valid preprocessor token, the token-pasting operator is unnecessary and possibly harmful.

For example, one might try to build string literals at compile time using the token-pasting operator:

#define STRINGIFY(x) #x
#define PLUS(a, b) STRINGIFY(a##+##b)
#define NS(a, b) STRINGIFY(a##::##b)
printf("%s %s\n", PLUS(1,2), NS(std,vector));

On some compilers, this will output the expected result:

1+2 std::vector

On other compilers, this will include undesired whitespace:

1 + 2 std :: vector

Fairly modern versions of GCC (>=3.3 or so) will fail to compile this code:

foo.cpp:16:1: pasting "1" and "+" does not give a valid preprocessing token
foo.cpp:16:1: pasting "+" and "2" does not give a valid preprocessing token
foo.cpp:16:1: pasting "std" and "::" does not give a valid preprocessing token
foo.cpp:16:1: pasting "::" and "vector" does not give a valid preprocessing token

The solution is to omit the token-pasting operator when concatenating preprocessor tokens to C/C++ operators:

#define STRINGIFY(x) #x
#define PLUS(a, b) STRINGIFY(a+b)
#define NS(a, b) STRINGIFY(a::b)
printf("%s %s\n", PLUS(1,2), NS(std,vector));

The GCC CPP documentation chapter on concatenation has more useful information on the token-pasting operator.

Rephrase

Thanks - I wasn't aware of this (but then I don't use these preprocessing operators too much...). – Coleville

It's called the "token pasting" operator for a reason - the intent is to end up with a single token when you're done. Nice writeup. – Substandard

When the result of the token-pasting operator is not a valid preprocessor token, the behavior is undefined. – Methoxychlor

Language changes like hexadecimal floats, or (in C++) digit separators and user-defined literals, continually change what constitutes a "valid preprocessing token", so please never abuse it like that! If you have to separate (language proper) tokens, please spell them as two separate tokens, and don't rely on accidental interactions between the preprocessor grammar and the language proper. – Hoot

Answer 4

This is useful in all kinds of situations in order not to repeat yourself needlessly. The following is an example from the Emacs source code. We would like to load a number of functions from a library. The function "foo" should be assigned to fn_foo, and so on. We define the following macro:

#define LOAD_IMGLIB_FN(lib,func) {                                      \
    fn_##func = (void *) GetProcAddress (lib, #func);                   \
    if (!fn_##func) return 0;                                           \
  }

We can then use it:

LOAD_IMGLIB_FN (library, XpmFreeAttributes);
LOAD_IMGLIB_FN (library, XpmCreateImageFromBuffer);
LOAD_IMGLIB_FN (library, XpmReadFileToImage);
LOAD_IMGLIB_FN (library, XImageFree);

The benefit is not having to write both fn_XpmFreeAttributes and "XpmFreeAttributes" (and risk misspelling one of them).

Jock

Answer 5

A previous question on Stack Overflow asked for a smooth method of generating string representations for enumeration constants without a lot of error-prone retyping.

Link

My answer to that question showed how applying little preprocessor magic lets you define your enumeration like this (for example) ...;

ENUM_BEGIN( Color )
  ENUM(RED),
  ENUM(GREEN),
  ENUM(BLUE)
ENUM_END( Color )

... With the benefit that the macro expansion not only defines the enumeration (in a .h file), it also defines a matching array of strings (in a .c file);

const char *ColorStringTable[] =
{
  "RED",
  "GREEN",
  "BLUE"
};

The name of the string table comes from pasting the macro parameter (i.e. Color) to StringTable using the ## operator. Applications (tricks?) like this are where the # and ## operators are invaluable.

Ratha

Answer 6

You can use token pasting when you need to concatenate macro parameters with something else.

It can be used for templates:

#define LINKED_LIST(A) struct list##_##A {\
A value; \
struct list##_##A *next; \
};

In this case LINKED_LIST(int) would give you

struct list_int {
int value;
struct list_int *next;
};

Similarly you can write a function template for list traversal.

Clachan

Answer 7

The main use is when you have a naming convention and you want your macro to take advantage of that naming convention. Perhaps you have several families of methods: image_create(), image_activate(), and image_release() also file_create(), file_activate(), file_release(), and mobile_create(), mobile_activate() and mobile_release().

You could write a macro for handling object lifecycle:

#define LIFECYCLE(name, func) (struct name x = name##_create(); name##_activate(x); func(x); name##_release())

Of course, a sort of "minimal version of objects" is not the only sort of naming convention this applies to -- nearly the vast majority of naming conventions make use of a common sub-string to form the names. It could me function names (as above), or field names, variable names, or most anything else.

Selda

Answer 8

I use it in C programs to help correctly enforce the prototypes for a set of methods that must conform to some sort of calling convention. In a way, this can be used for poor man's object orientation in straight C:

SCREEN_HANDLER( activeCall )

expands to something like this:

STATUS activeCall_constructor( HANDLE *pInst )
STATUS activeCall_eventHandler( HANDLE *pInst, TOKEN *pEvent );
STATUS activeCall_destructor( HANDLE *pInst );

This enforces correct parameterization for all "derived" objects when you do:

SCREEN_HANDLER( activeCall )
SCREEN_HANDLER( ringingCall )
SCREEN_HANDLER( heldCall )

the above in your header files, etc. It is also useful for maintenance if you even happen to want to change the definitions and/or add methods to the "objects".

Ctenidium

Answer 9

SGlib uses ## to basically fudge templates in C. Because there's no function overloading, ## is used to glue the type name into the names of the generated functions. If I had a list type called list_t, then I would get functions named like sglib_list_t_concat, and so on.

Unpack

Answer 10

I use it for a home rolled assert on a non-standard C compiler for embedded:

#define ASSERT(exp) if(!(exp)){ \
                      print_to_rs232("Assert failed: " ## #exp );\
                      while(1){} //Let the watchdog kill us 

Kimball

I take it you mean by ‘non-standard’ that the compiler did not do string pasting but did do token pasting — or would it have worked even without ##? – Gregson

Answer 11

I use it for adding custom prefixes to variables defined by macros. So something like:

UNITTEST(test_name)

expands to:

void __testframework_test_name ()

Swingle

Answer 12

One important use in WinCE:

#define BITFMASK(bit_position) (((1U << (bit_position ## _WIDTH)) - 1) << (bit_position ## _LEFTSHIFT))

While defining register bit description we do following:

#define ADDR_LEFTSHIFT                          0

#define ADDR_WIDTH                              7

And while using BITFMASK, simply use:

BITFMASK(ADDR)

Dubonnet

Answer 13

It is very useful for logging. You can do:

#define LOG(msg) log_msg(__function__, ## msg)

Or, if your compiler doesn't support function and func:

#define LOG(msg) log_msg(__file__, __line__, ## msg)

The above "functions" logs message and shows exactly which function logged a message.

My C++ syntax might be not quite correct.

Cawthon

What were you trying to do with that? It would work just as well without the "##", since there is no need to token-paste "," to "msg". Were you trying to stringify msg? Also, FILE and LINE must be in uppercase, not lowercase. – Rephrase

You're right indeed. I need to find the original script to see how ## was used. Shame on me, no cookie today! – Cawthon