Why can't "transform(s.begin(),s.end(),s.begin(),tolower)" be complied successfully?

Question

Given the code:

#include <iostream>
#include <cctype>
#include <string>
#include <algorithm>
using namespace std;

int main()
{
     string s("ABCDEFGHIJKL");
     transform(s.begin(),s.end(),s.begin(),tolower);
     cout<<s<<endl;
}

I get the error:

No matching function for call to transform(__gnu_cxx::__normal_iterator<char*, std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, __gnu_cxx::__normal_iterator<char*, std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, __gnu_cxx::__normal_iterator<char*, std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, <unresolved overloaded function type>)

What does "unresolved overloaded function type" mean?

If I replace the tolower with a function I wrote, it no longer errors.

Mezzo

The return type of main is int, and return types in C++ have to be explicit. Some compilers will allow the code as posted, but it is non-standard, and it might break with the new compiler version or in other compilers. – Mayapple

@DavidRodríguez-dribeas A return from main is not required by C or C++, it implicitly returns 0. See the comments on this answer: https://mcmap.net/q/55732/-does-c-have-a-free-function-size-object – Consols

Answer 1

Try using ::tolower. This fixed the problem for me.

Youthful

@liu: it's as @David wrote - the "::" selects the tolower from the global namespace – Youthful

This answer doesn't answer the question – why doesn't it work? – Anselma

Depends on your compiler. MSVC 19 doesn't seem to have a problem with it. If you check cppreference's note it shows a specific example of how to correctly use transform with std::tolower. With that said cppreference seems to say the function signature is the same (cctype, ctype), so I'm actually not sure why it wont compile on for example clang. – Vikki

Okay, so. Did some digging. in std there's a version of tolower that takes a const locale&, so transform can't deduce which of the two overloads to use. Since the global namespace version of tolower has no overloads, it works just fine. With that said, msvc doesn't seem to have this issue - unless you include <locale>. – Vikki

Answer 2

Let's look at a list of options starting with the worst and moving to the best. We'll list them here and discuss them below:

1. transform(cbegin(s), cend(s), begin(s), ::tolower)
2. transform(cbegin(s), cend(s), begin(s), static_cast<int(*)(int)>(tolower))
3. transform(cbegin(s), cend(s), begin(s), [](const unsigned char i){ return tolower(i); })

---

The code in your question, transform(s.begin(), s.end(), s.begin(), tolower) will produce an error like:

No matching function for call to transform(std::basic_string<char>::iterator, std::basic_string<char>::iterator, std::basic_string<char>::iterator, <unresolved overloaded function type>)

The reason that you were getting an "unresolved overloaded function type" is there are 2 tolowers in the std namespace:

1. The locale library defines template <typename T> T tolower(T, const locale&)
2. The cctype library defines int tolower(int)

1 is the solution offered by davka. It addresses your error by leveraging the fact that locale's tolower is not defined in the global namespace.

Depending upon your situation locale's tolower may merit consideration. You can find a comparison of the tolowers here: Which tolower in C++?

---

Unfortunately 1 depends upon cctype's tolower being defined in the global namespace. Let's look at why that may not be the case:

You are rightly using #include <cctype>, as doing #include <ctype.h> has been deprecated in C++: http://en.cppreference.com/w/cpp/header

But the C++ standard states in D.3[depr.c.headers]2 of the declarations in the headers:

It is unspecified whether these names are first declared or defined within namespace scope (3.3.6) of the namespace std and are then injected into the global namespace scope by explicit using-declarations (7.3.3)

So the only way that we can guarantee our code is implementation independent is to use a tolower from namespace std. 2 is the solution offered by David Rodríguez - dribeas. It leverages the fact that static_cast can:

Be used to disambiguate function overloads by performing a function-to-pointer conversion to specific type

Before we move on, let me comment that if you find int (*)(int) to be a bit confusing you can read more on function pointer syntax here.

---

Sadly there is one other issue with tolower's input argument, if it:

Is not representable as unsigned char and does not equal EOF, the behavior is undefined

You are using a string which uses elements of type: char. The standard states of char specifically 7.1.6.2[dcl.type.simple]3:

It is implementation-defined whether objects of char type are represented as signed or unsigned quantities. The signed specifier forces char objects to be signed

So if the implementation defined a char to mean a signed char then both 1 and 2 would result in Undefined Behavior for all characters corresponding to negative numbers. (If an ASCII character encoding is being used the characters corresponding to negative numbers are Extended ASCII.)

The Undefined Behavior can be avoided by converting the input to an unsigned char before passing it to tolower. 3 accomplishes that using a lambda that accepts an unsigned char by value, then passes it to tolower implicitly converting to int.

To guarantee Defined Behavior on all compliant implementations, independent of character encoding, you'll need to use transform(cbegin(s), cend(s), begin(s), [](const unsigned char i){ return tolower(i); }) or something similar.

Consols

I'd like to know that, too. – Stoecker

@Stoecker I believe this is the best answer here. I took the time to add an answer to a question, even though it'll never be accepted because I felt all the other answers fell short in one fashion or another. To have someone just drive-by-down-vote is... sad. Anyway thanks for the confirmation. It's nice to know someone cares about my efforts to improve answer quality. – Consols

@TonvandenHeuvel Thanks, I would also view it as the most complete. But I don't think we'll see it accepted as the OP hasn't been seen since 2014. – Consols

Answer 3

The problem most probably relates with multiple overloads of tolower and the compiler is unable to select one for you. You can try qualifying it to select an specific version of it, or you might need to provide a function pointer cast to disambiguate. The tolower function can be present (multiple different overloads) in the <locale> header, as well as in <cctype>.

Try:

int (*tl)(int) = tolower; // Select that particular overload
transform(s.begin(),s.end(),s.begin(),tl );

That can be done in a single line with a cast, but it is probably harder to read:

transform(s.begin(),s.end(),s.begin(),(int (*)(int))tolower );

Audwen

But don't forget that using this version of tolower on a string like the above is undefined behavior if any of the char values are negative (which they can be on most modern systems, e.g. if any accented characters are present). – Kappenne

@James Kanze: good point, I decided for that overload from reading the original post (where cctype is explicitly included, while locale is not). Also, the functions in locale take more than a single argument, and that means that the code would add unrelated complexity with a bind or bind2nd to provide the default locale... – Mayapple

Thank you.I understand the problem. And using ::tolower can fix the problem – Mezzo

@liu: Note that using ::tolower will work in different compilers, but it is not standard. Basically most compilers, when you include cctype the compiler is required to provide int std::tolower(int), but it is not required to add int ::tolower(int), different compilers will provide both functions with the same implementation (one of them will forward to the other) but that is not required and might change with the next compiler release (or if you change the compiler) – Mayapple

@David Rodriguez ::tolower is guaranteed to work if you include <ctype.h>. Which has the added advantage of making it clear that you're dealing with a C function, and not something added in C++. – Kappenne

@Mezzo Using ::tolower doesn't fix the problem. Calling ::tolower with a char argument is undefined behavior. You need to wrap it in a functional object which converts the char to unsigned char. Or buy into all of the complexity that David Rodríguez mentions with regards to the versions in <locale>. – Kappenne

@James: I think it is better to say that ::tolower solution is restricted to 7-bit ascii strings only. – Youthful

@Youthful C++ doesn't have a '7-bit ascii' string type. ::tolower is guaranteed to work if you can guarantee that none of the character codes in the string have negative values, which in practice means that you've checked it before, or you're not concerned about portability, and plain char is unsigned. (Both VC++ and g++ have options to make plain char unsigned. If those two compilers cover all of your portability needs, you can use the option, and skip the cast.) – Kappenne

@James I know, I am not talking about types but values. "restricted to 7-bit ascii strings" is a concise way to say "you are sure/have ensured that your data is plain ascii (7-bit), and you don't care about internationalization, at least not now". – Youthful

@James, and BTW, why are you saying that using negative input to tolower is problematic? If its signature is int std::tolower(int) I'd expect it to handle any int input gracefully. – Youthful

@davka: One would hope so, wouldn't one. But the C standard says that the behavior is undefined if the input is not in the range 0...UINT_MAX or EOF. – Kappenne

@James, oh... I bet they kept EOF so that they could write something like tolower(getchar()). A hacker's footprints in the standard... – Youthful

@Youthful Exactly:-). (This goes back to the earliest days of C.) – Kappenne

@JamesKanze #include <ctype.h> will only solve the problem on compilers which support the deprecated "ctype.h" library. The standard only guarantees #include <cctype> to be available: https://mcmap.net/q/54349/-why-can-39-t-quot-transform-s-begin-s-end-s-begin-tolower-quot-be-complied-successfully – Consols

@Youthful The standard explicitly says that for any tolower argument that: "Is not representable as unsigned char and does not equal EOF, the behavior is undefined" thus passing extended ASCII to tolower is potentially undefined behavior: https://mcmap.net/q/54349/-why-can-39-t-quot-transform-s-begin-s-end-s-begin-tolower-quot-be-complied-successfully – Consols

Answer 4

David already identified the issue, namely a conflict between:

• <cctype>'s int tolower(int c)
• <locale>'s template <typename charT> charT tolower(charT c, locale const& loc)

Using the first is much easier, but is undefined behavior (unfortunately) as soon as you deal with anything else than lower-ascii (0-127) in signed chars. By the way, I do recommend defining char as unsigned.

The template version would be nice, but you would have to use bind to provide the second parameter, and it's bound to be ugly...

So, may I introduce the Boost String Algorithm library ?

And more importantly: boost::to_lower :)

boost::to_lower(s);

Expressiveness is desirable.

Unmanned

1) can you please explain what do you mean by I do recommend defining char as unsigned? 2) does boost::to_lower assumes some character set, e.g. latin-1? – Youthful

@davka: 1) the C++ Standard does not precise whether a char is signed or unsigned. You can qualify it if you want to be sure. However a number of functions (like int tolower(int)) have undefined behavior if invoked with a negative char... Look it up on your compiler, there might be a switch or a sane default. 2) boost::to_lower is based on the C++ tolower function, and thus depends on the std::locale and the ctype facet it has been imbued with. Note that these facets cannot handle multi-characters encoding anyway... – Unmanned

thanks, I still don't get #1. I know about char being implementation-dependent. Are you suggesting typedef unsigned char char? is it legal? – Youthful

@davka: No, it's not legal. Compilers usually have switches to let you decide, for example gcc has -fsigned-char and -funsigned-char. – Unmanned

that's interesting, thanks. So, if I use -funsigned-char I should use signed char if I want -128..127, correct? Does it work well with 3rd party libraries that are not compiled with this flag? – Youthful

@davka: correct. And you need to beware of 3rd party libraries: for 3rd party libraries to interact smoothly with each other, and your programs, all shall give char the same meaning. You'll need to check out their documentation to know whether they explicited it or inspect the code to know if they made assumptions: while((c = getchar()) == EOF) is an infinite loop with -funsigned-char, as getchar() returns an int, not a char (and EOF is -1) – Unmanned

@Matthieu M You're right in principle that all libraries should be compiled with the same "language", and changing the signedness of plain char changes the language. In practice, however, changing the signedness of char doesn't effectively change the interface, at least on 2's complement machines, and it's possible to compile will -funsigned-char, then link with libraries not using this option. – Kappenne

@James: I agree, the link will pass without issue :) – Unmanned

@Matthieu M: And the program will work as expected. At least, I've never found a case where it didn't; the impact of whether char is signed or not seems to be limited to individual expressions, and there's no problem if it's treated as signed in one location, and unsigned in another, per se. – Kappenne

@James: I cannot think of an issue, but... I'd rather not deduce that it would work. There are (definitely) too many corner cases to the C and C++ standards. – Unmanned

Answer 5

Browsing my <ctype> header from gcc 4.2.1, I see this:

// -*- C++ -*- forwarding header.

// Copyright (C) 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005
// Free Software Foundation, Inc.

...

#ifndef _GLIBCXX_CCTYPE
#define _GLIBCXX_CCTYPE 1

#pragma GCC system_header

#include <bits/c++config.h>
#include <ctype.h>

// Get rid of those macros defined in <ctype.h> in lieu of real functions.
#undef isalnum
#undef isalpha

...

#undef tolower
#undef toupper

_GLIBCXX_BEGIN_NAMESPACE(std)

  using ::isalnum;
  using ::isalpha;

...

  using ::tolower;
  using ::toupper;

_GLIBCXX_END_NAMESPACE

#endif

So it looks like tolower exists in both the std (from <cctype>) and root (from <ctype.h>) namespaces. I'm not sure what the #pragma does.

Suter

the pragma signals to gcc that this file is a system header. This should typically affect diagnosis, since it's considered bad style for a compiler to emit warnings for headers that it was bundled with and should not be altered. – Unmanned