C++ preprocessor #define-ing a keyword. Is it standards conforming?

Question

Help settle the debate that's going on in the comments at this question about bool and 1:

Can a standards-conforming C++ preprocessor allow one to use #define to redefine a language keyword? If so, must a standards-conforming C++ preprocessor allow this?

If a C++ program redefines a language keyword, can that program itself be standards conforming?

Moyna

I'm quite sure I've seen the last point debated here, but in terms of C99. – Lynwoodlynx

Answer 1

In C++, the closest thing to forbidding #defineing a keyword is §17.4.3.1.1/2, which only disallows it in a translation unit that includes a standard library header:

A translation unit that includes a header shall not contain any macros that define names declared or defined in that header. Nor shall such a translation unit define macros for names lexically identical to keywords.

The second sentence of that paragraph has been changed in C++0x to outright forbid #defineing a keyword (C++0x FCD §17.6.3.3.1):

A translation unit shall not #define or #undef names lexically identical to keywords.

Edit: As pointed out by Ken Bloom in comments to his answer, the rules have not changed in C++0x; the text has just been rearranged to confuse people like me. :-)

Karlise

So technically, in C++03 it is legal to redefine keywords, as long as you do not include a single header? I think I'm going to interpret it as "redefining keywords is forbidden", like C++0x says. :) – Hinterland

This answers the question of whether a program that redefines the keywords is standards conforming, not the technical capability of the preprocessor. – Moyna

@Ken: Right. However, if a program violates that shall statement, then the program is ill-formed because the rule is diagnosable (i.e., it doesn't say "no diagnostic required" and doesn't say that violating it results in undefined behavior). Therefore, I think that a conforming preprocessor is at least required to warn that you are violating the rule. – Karlise

@jalf, and note that, like James said, "header" in the Standard only refers to Standard library headers. You may still happily include your own source files. And note that in C++03 some keywords can't be #define'ed. These are new and delete. Now that C++0x forbids all of them, i suspect open-std.org/jtc1/sc22/wg21/docs/cwg_active.html#369 has become resolved too. (though i'm not sure why they are treated specially in preprocessing phase). – Coprolite

@James C++0x's 17.6.3.3.1/2 says "A translation unit shall not #define or #undef names lexically identical to keywords.", thereby reserving those names. And 17.6.3.3/2 says "If a program declares or defines a name in a context where it is reserved, other than as explicitly allowed by this Clause, its behavior is undefined.", so I think it's undefined behavior. C++0x got rid of "... that includes a header ...", so it certainly seems to have a different ruleset. – Coprolite

I don't think that 17.6.3.1 is intended to limit its restrictions to only those programs including headers. That would also mean you could use names like __foo, as long as you don't include headers, which for sure isn't the intent at all. If such things are intended, it won't be established by such an introductory sentence. Moreover, programs can make implicit use of the library by using language support library component (exceptions, new, ...), so its usage isn't defined by the presence of #include lines at all. – Coprolite

@James ah, i noticed coppro (aka Sean Hunt) already sent an issue report about exactly this. LWG #1236 – Coprolite

@Johannes: Oh; I forgot about your comments, sorry. I read them while I was on a plane and couldn't reply. I agree with the logic that the interpretation in my answer would mean that you could use names like __foo, which is certainly wrong. In any case, the language is at best ambiguous, though apparently it is Not A Defect. :-| – Karlise

Answer 2

It is not permitted, according to C++11 [macro.names]:

A translation unit shall not #define or #undef names lexically identical to keywords, to the identifiers listed in Table 3, or to the attribute-tokens described in 7.6.

The "identifiers listed in Table 3" are final and override; and the attribute-tokens are the identifiers in [[fallthrough]] and so on.

This clause is still in the latest standard too.

Puree

This should now be the accepted answer. C++11 fixed the oddity about requiring an #include to just make this flat-out illegal, as it should be. – Wagram

Answer 3

Working from the 2005-10-19 C++ working draft (since I don't have a standard handy):

Section 16.3 defines the grammar for #define to be #define identifier replacement-list-newline (object-like macros) or one of several constructions beginning with #define identifier lparen (function-like macros). identifiers are defined in section 2.10 to be identifier-nondigit | identifier identifier-nondigit | identifier digit. Section 2.11 indicates that a certain list of identifiers are unconditionally treated as keywords in phase 7 of compilation (section 2.1), and I conclude that they are therefore not treated specially in phase 4, which is preprocessor expansion. Thus, it appears that the standard requires the preprocessor to allow you to redefine language keywords (listed in Section 2.11).

However, the preprocessor has a keyword of its own, namely defined, as well as a list of predefined macros (Section 16.8). Section 16.8 states that the behavior is undefined if you redefine these, but does not prohibit the preprocessor from recognizing these as macro names.

Moyna

Note also that the identifiers true and false (since there are no "keywords" yet, they are just "identifiers") are treated specially during macro replacement. – Karlise

@James: can you point out the section in the standard (or the draft) where it says this? GCC's implementation also claims to treat C++'s named operators (those #defined in iso646.h in C) specially when operating in C++ mode. gcc.gnu.org/onlinedocs/cpp/Macros.html – Moyna

@Ken: "After all replacements due to macro expansion and the defined unary operator have been performed, all remaining identifiers and keywords, except for true and false, are replaced with the pp-number 0" (16.1/4). As for why the named operators are handled specially, it is because a named operator is an "operator or punctuator" preprocessing token, not an "identifier" preprocessing token. – Karlise

@James and the G++ team found it necessary to allow even these to be killed: stackoverflow.com/questions/2419805/… – Helga

@Ken: When I said "macro replacement" in my first comment, I meant "conditional inclusion evaluation." Sorry about that. – Karlise

@James, so redefining keywords other than those mentioned in 16.8 which are nevertheless used by the preprocessor does change their meaning before phase 7, but the redefinition is ignored in phase 4 when evaluating #if. – Moyna

There's a footnote that points out that there are no keywords during macro processing (phase 4). Note that 17.6.3.1.1 changed between C++95 and the latest Final Committee Draft, and the latter forbids redefining keywords (although in a nonintuitive place). – Avery

I think that's the important part here. The language in §17.6.3.3.1 (of the C++0x FCD) says "names lexically identical to keywords," so even though there are no keyword tokens during preprocessing, there can be other tokens (namely identifiers, but also op-or-puncs for the named operators) that are lexically identical to keywords (i.e., have the same spelling). Ergo, it should be trivial for the preprocessor to detect and diagnose a violation of the rule, and the preprocessor is not required to allow redefinition of keywords. – Karlise

The C++0x final committee draft is at open-std.org/jtc1/sc22/wg21/docs/papers/2010/n3092.pdf for those who are following along at home. – Moyna

@James: §17.6.3.3.1 is qualified by §17.6.3.1 which states "This section describes restrictions on C++ programs that use the facilities of the C++ standard library." – Moyna

@Ken: Ohhh, you are right; I totally missed that. So, I suppose C++03 and C++0x don't really differ after all (I'll have to update my answer to reflect that). However, given that the preprocessor also handles source file inclusion, and given that the names of the standard libraries are known, it should still be relatively easy for the preprocessor to detect whether the rule has been violated. – Karlise