Why doesn't range-for find my overloads of begin and end for std::istream_iterator?

Question

I have code like this

std::ifstream file(filename, std::ios_base::in);
if(file.good())
{
    file.imbue(std::locale(std::locale(), new delimeter_tokens()));
    for(auto& entry : std::istream_iterator<std::string>(file))
    {
        std::cout << entry << std::endl;    
    }
}
file.close();

where std::istream_iterator<std::string>'s begin() and end() are defined as follows

template<class T>
std::istream_iterator<T> begin(std::istream_iterator<T>& stream)
{
    return stream;
}

template<class T>
std::istream_iterator<T> end(std::istream_iterator<T>& stream)
{
    return std::istream_iterator<T>();
}

which is what Mark Nelson has also written about in Dr. Dobb's here. Alas, the code fails to compile on my Visual Studio 2012 with error messages

error C3312: no callable 'begin' function found for type 'std::istream_iterator<_Ty>'

and

error C3312: no callable 'end' function found for type 'std::istream_iterator<_Ty>'

Question: Is there something I haven't noticed, bug in the compiler (unlikely, but just in case) or... Well, any ideas?

This questions is cleaned up considerably, as advised by Xeo. To provide more background and references this is related to my other question on Stackoverflow, I was wondering how to make line based parsing cleaner than the usual loops. A bit of coding and checking from the internet, and I had a working sketch as follows

std::ifstream file(filename, std::ios_base::in);
if(file.good())
{               
    file.imbue(std::locale(std::locale(), new delimeter_tokens()));
    for(auto& entry : istream_range<std::string>(file)
    {
        std::cout << entry << std::endl;    
    }
}
file.close();

but there was slight snag I tried to remedy. I think it would look more natural to write as in the code that fails to compile and not like

for(auto& entry : istream_range<std::string>(file)

Please, take a note of the different iterator. The delimeter_tokens is defined like Nawaz kindly has shown here (code not duplicated) and istream_range as in Code Synthesis blog here. I think the begin and end implementations should work, as advertised in the aforementioned Code Synthesis blog post

The last rule (the fallback to the free-standing begin()and end() functions) allows us to non-invasively adapt an existing container to the range-based for loop interface.

Thus my question with all the (ir)relevant background.

Hexylresorcinol

I took the liberty to change the question title to reflect the actual question asked. Roll back or change if necessary. – Fusillade

It's better like that, I think. It gets the attention of those who are seeking to a solution similar to mine and those who has the lookup problem to solve. How do I make your post as an answer (or does someone more accomplished do that)? – Hexylresorcinol

Answer 1

Ranged-for relies on ADL if the special handling for native array (T foo[N]) and member begin/end doesn't yield any results.

§6.5.4 [stmt.ranged] p1

• otherwise, begin-expr and end-expr are begin(__range) and end(__range), respectively, where begin and end are looked up with argument-dependent lookup (3.4.2). For the purposes of this name lookup, namespace std is an associated namespace.

Your problem is, that the associated namespace of std::istream_iterator is (obviously) namespace std, not the global namespace.

§3.4.2 [basic.lookup.argdep] p2

For each argument type T in the function call, there is a set of zero or more associated namespaces and a set of zero or more associated classes to be considered. The sets of namespaces and classes is determined entirely by the types of the function arguments [...].

• If T is a fundamental type, its associated sets of namespaces and classes are both empty.
• If T is a class type (including unions), its associated classes are: the class itself; the class of which it is a member, if any; and its direct and indirect base classes. Its associated namespaces are the namespaces of which its associated classes are members. Furthermore, if T is a class template specialization, its associated namespaces and classes also include: the namespaces and classes associated with the types of the template arguments provided for template type parameters [...].

Note the last (quoted) part of the second bullet. It basically means that using a class which is a member of the global namespace as the template argument makes the code work:

#include <iterator>
#include <iostream>

template<class T>
std::istream_iterator<T> begin(std::istream_iterator<T> is){
  return is;
}
template<class T>
std::istream_iterator<T> end(std::istream_iterator<T>){
  return std::istream_iterator<T>();
}

struct foo{};

std::istream& operator>>(std::istream& is, foo){
  return is;
}

int main(){
  for(foo f : std::istream_iterator<foo>(std::cin))
  //                                ^^^
  // make global namespace one of the associated namespaces
    ;
}

Fusillade

I'm not able to follow you now. How would that namespace association trick work if I should read tokenized strings from the file stream? That is, the foo you point out in the loop should be strings in my case. – Hexylresorcinol

@Veksi: Basically, only the first part of my answer is important: You can't put overloads for std members in the global namespace for ranged-for. The second part was only to showcase a quirkyness of ADL that would allow that to work. :P – Fusillade

While commenting on this Q&A, I found out about core issue 1442 that changes the rules a little bit. Namespace std is no longer an associated namespace and it is explicitly mentioned that "begin and end are looked up in the associated namespaces (3.4.2). [ Note: Ordinary unqualified lookup (3.4.1) is not performed. — end note ]" – Erotica

Answer 2

Because of argument depended lookup the compiler tries to find begin() and end() in the std namespace. If you put your functions there, the code compiles.

Since name lookup is a complicated issue in C++ I'm not entirely sure if the compiler is behaving correctly or not.

Concinnity

You're not allowed to put overloads into namespace std. – Fusillade

But you can specialize templates, no? – Subkingdom

@jrok: Yes, but only for your user-defined types, not for types that originate from std (IIRC). Also, you can't partially specialize function templates and function template specialization in general is frowned upon. – Fusillade