Viewing a raw pointer as a range in range-based for-loop

Question

How can I make a raw pointer behave like a range, for a for-range loop syntax.

double five = 5;
double* dptr = &five;
for(int& d : dptr) std::cout << d << std::endl;// will not execute if the pointer is null

Motivation:

It is now vox populi that an boost::optional (future std::optional) value can be viewed as a range and therefore used in a for range loop http://faithandbrave.hateblo.jp/entry/2015/01/29/173613.

When I rewrote my own simplified version of it:

namespace boost {
    template <class Optional>
    decltype(auto) begin(Optional& opt) noexcept{
        return opt?&*opt:nullptr;
    }

    template <class Optional>
    decltype(auto) end(Optional& opt) noexcept{
        return opt?std::next(&*opt):nullptr;
    }
}

Used as

boost::optional<int> opt = 3;
for (int& x : opt) std::cout << x << std::endl;

While looking that code I imagined that it could be generalized to raw (nullable) pointers as well.

double five = 5;
double* dptr = &five;
for(int& d : dptr) std::cout << d << std::endl;

instead of the usual if(dptr) std::cout << *dptr << std::endl;. Which is fine but I wanted to achieve the other syntax above.

Attempts

First I tried to make the above Optional version of begin and end work for pointers but I couldn't. So I decided to be explicit in the types and remove all templates:

namespace std{ // excuse me, this for experimenting only, the namespace can be removed but the effect is the same.
    double* begin(double* opt){
        return opt?&*opt:nullptr;
    }
    double* end(double* opt){
        return opt?std::next(&*opt):nullptr;
    }
}

Almost there, it works for

for(double* ptr = std::begin(dptr); ptr != std::end(dptr); ++ptr) 
    std::cout << *ptr << std::endl;

But it doesn't work for the supposedly equivalent for-range loop:

for(double& d : dptr) std::cout << d << std::endl;

Two compilers tell me: error: invalid range expression of type 'double *'; no viable 'begin' function available

What is going on? Is there a compiler magic that forbids the ranged-loop to to work for pointers. Am I making a wrong assumption about the ranged-loop syntax?

Ironically, in the standard there is an overload for std::begin(T(&arr)[N]) and this is very close to it.

---

Note and a second though

Yes, the idea is silly because, even if possible this would be very confusing:

double* ptr = new double[10];
for(double& d : ptr){...}

would iterate over the first element only. A more clear and also realistic workaround would be to do something like workaround proposed by @Yakk:

for(double& d : boost::make_optional_ref(ptr)){...}

In this way it is clear that we are iterating over one element only and that that element is optional.

Ok, ok, I will go back to if(ptr) ... use *ptr.

Elevon

Your begin and end can never be found by ADL, so the range based for is not going to work. And I don't understand how one can think of an optional as a range?! You should be able to get your example to work by constructing a boost::iterator_range with those pointers, but forming an end iterator to a scalar object the way you are is most likely undefined behavior. – Typebar

@Typebar Actually it's legal; for pointer arithmetic purposes something that's not an array element is considered to be in an array of size 1. Adding overloads to namespace std is definitely UB, though, and abusing the range-based for for this is rather silly. – Volt

@Volt Ah, ok, thanks for clarifying, I wasn't sure of that part. – Typebar

The weird thing is that std::begin(T(&arr)[N]) isn't used for range-based for loops. The core language handles arrays directly: "if _RangeT is an array type, begin-expr and end-expr are __range and __range + __bound, respectively, where __bound is the array bound." – Radmen

Related: faithandbrave.hateblo.jp/entry/2015/01/29/173613 – Elevon

Answer 1

Because the way that range-based for works is (from §6.5.4):

begin-expr and end-expr are determined as follows
— if _RangeT is an array type, [..]
— if _RangeT is a class type, [..]
— otherwise, begin-expr and end-expr are begin(__range) and end(__range), respectively, where begin and end are looked up in the associated namespaces (3.4.2). [ Note: Ordinary unqualified lookup (3.4.1) is not performed. —end note ]

What are the associated namespaces in this case? (§3.4.2/2, emphasis mine):

The sets of namespaces and classes are determined in the following way:
(2.1) — If T is a fundamental type, its associated sets of namespaces and classes are both empty.

Thus, there is no place to put your double* begin(double*) such that it will be called by the range-based for statement.

A workaround for what you want to do is just make a simple wrapper:

template <typename T> 
struct PtrWrapper {
    T* p;
    T* begin() const { return p; }
    T* end() const { return p ? p+1 : nullptr; }
};

for (double& d : PtrWrapper<double>{dptr}) { .. }

Wun

Or, the non-standardese version: while thinking that std::begin and std::end in an ADL context are used by for(:) loops will get you most of the truth, the standard instead almost reimplements that without referring to them or using them. This is mostly so low level language constructs do not depend on the library. – Mnemonic

@Volt lol, yeah, I copied what OP sort of did without paying attention – Wun

What unfortunate, the problem is that in principle one can do this, but cannot put the functions in any namespace. It was "compiler magic" (or the lack of) at the end. – Elevon

@Elevon Actually no, it's not "compiler magic." It's "well-defined standard language behavior." – Wun

What I mean is that they could have made T* a special case, like they did for T(&)[N]. Maybe they didn't because this would have been confusing double* dptr = new double[10]; for(double& d : dptr){...}; since the for will do the first element only. – Elevon

Answer 2

It is a useful lie to think that for(:) loops are implemented by "calling std::begin and std::end in a ADL-activated context". But that is a lie.

The standard instead basically does a parallel implementation of the std::begin and std::end in itself. This prevents the language's low level constructs from depending on its own library, which seems like a good idea.

The only lookup for begin by the language is the ADL-based lookup. Your pointer's std::begin won't be found, unless you are a pointer to something in std. The std::begin( T(&)[N} ) isn't found this way by the compiler, but instead that iteration is hard-coded by the language.

namespace boost {
  template<class T>
  T* begin( optional<T>&o ) {
    return o?std::addressof(*o):nullptr;
  }
  template<class T>
  T* begin( optional<T&>&&o ) {
    return o?std::addressof(*o):nullptr;
  }
  template<class T>
  T const* begin( optional<T> const&o ) {
    return o?std::addressof(*o):nullptr;
  }
  template<class T>
  T* end( optional<T>&o ) {
    return o?std::next(begin(o)):nullptr;
  }
  template<class T>
  T* end( optional<T&>&&o ) {
    return o?std::next(begin(o)):nullptr;
  }
  template<class T>
  T const* end( optional<T> const&o ) {
    return o?std::next(begin(o)):nullptr;
  }
  template<class T>
  boost::optional<T&> as_optional( T* t ) {
    if (t) return *t;
    return {};
  }
}

now you can:

void foo(double * d) {
  for(double& x : boost::as_optional(d)) {
    std::cout << x << "\n";
}

without having to repeat the type double.

Note that an rvalue optional to a non-reference returns a T const*, while an rvalue optonal to a T& returns a T*. Iterating over a temporary in a writing context is probably an error.

Mnemonic

I did the begin(Optional&) precisely to avoid all the overloads. I should have done Optional&&. – Elevon

Intead of as_optional (towards the end of your post) I could use boost::make_optional(dptr, *dptr) (seems to work, but I am not sure about the evaluation order). This makes me think that make_optional could have an overload of pointers T* to do this. – Elevon

@Elevon except your works on things that are not optionals. A boost::mutex looks iterable to your code. – Mnemonic

Yes, but it quacks like an optional. That's when I realized it could work for pointers as well if I could put it in the right namesapce... and it didn't because there is no right namespace for this (as the other answer pointed out). I am no familiar with mutex but wouldn't it soft fail at std::next or after operator*? I am just reading en.cppreference.com/w/cpp/thread/mutex and boost.org/doc/libs/1_31_0/libs/thread/doc/mutex.html – Elevon

@Elevon I'm saying you need at least a SFINAE test. You don't want to fail in the body of your begin, you want to fail by failing to match the override. Your begin override is too greedy, and being too greedy is rude and can cause bugs. every class in boost:: will be passed to your begin, even if it is not appropriate. – Mnemonic

Wouldn't the decltype(auto) work as SFINAE? at least auto ... -> decltype(opt?&*std::forward<Optional>(opt):nullptr) would right? – Elevon

@Elevon no decltype(auto) does not do SFINAE. The other might, but now anything that overrides unary * (say a regex? that uses it for kleen star?) and operator bool (does regex?) in namespace boost now appears to be iterable. Don't pollute huge namespaces with overloads that glom on to everything. To make that safe, you'd have to understand every boost type and every type that will be added to boost while your modification is in there, and that is simply not possible. – Mnemonic

Thanks, I think it is clear now, also ignore my comment about boost::make_optional(dptr, *dptr). That would create a copy and that is unacceptable. – Elevon

@alfc and can create an actual reference from a null pointer, which is illegal. – Mnemonic

Yes. But it seemed to work and that confused me. Your as_optional is correct I think. – Elevon

Answer 3

TL;DR

This construct can be used in a range for loop:

std::views::counted(raw_ptr, !!raw_ptr)

Details

C++20 offers a plethora of ways to create ad-hoc iterables, using the ranges library. For example:

#include <ranges>
#include <iostream>
 
int main()
{
    int a[] = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10};
    
    int *raw_ptr = a;
    
    for(int i : std::views::counted(raw_ptr, 10))
        std::cout << i << ' ';
    std::cout << '\n';
    
    for(int i : std::views::counted(raw_ptr, 1))
        std::cout << i << ' ';
    std::cout << '\n';
    
    std::cout << "empty for null pointer pointer\n";
    raw_ptr = nullptr;
    for(int i : std::views::counted(raw_ptr, 0))
        std::cout << i << ' ';
    std::cout << '\n';
    
    std::cout << "Exit\n";
}

Prints

1 2 3 4 5 6 7 8 9 10 
1

empty for null pointer

Exit

Similarly std::views::subrange could be used with the (start, end] pointers. Check the library for more info.

Mccready

@Sopel, good point, maybe a workaround could be std::views::counted(raw_ptr, !!raw_ptr). or Ranges could have an std::views::optional(raw_ptr) that does this (at least for a nullable, boolean-comparable, iterator). godbolt.org/z/9PWr5T – Elevon

@Sopel You are wrong coliru.stacked-crooked.com/a/67e130f39cc46568. As with any "counted range" you have to know the number of elements, which in this case is 0; after that nullptr creates no particular problem – Mccready

@Elevon Not a good point :P By this logic there is no counted_range that works. If you give a count > size of the collection you'll access memory out of it, so it always fall on the programmer to provide a correct count – Essive

@LorahAttkins, according to my question, the correct count would be given by the pointer itself. If the pointer is null the count is zero, otherwise one. So, views::counted could be used by doing std::views::counted(raw_ptr, raw_ptr?1:0). – Elevon

@Elevon I think this answer generalizes on the concept of iterating raw pointers. The pointer itself does not contain information about the number of elements. Yes, if we're talking about a single element what you say is true, but generally it should be std::views::counted(raw_ptr, raw_ptr ? n : 0) and the programmer should know n. In any case raw_ptr being NULL doesn't cause any problems in contrast to what @Sopel says; it is the count that matters. – Essive

@LorahAttkins, the original question was to replace code like if(p) ...use *p; with a range-for loop without explicit pointer dereference for(auto& e: ???) ...use e;. The count is either 0 or 1 and the arithmetic (and even proper iteration) is irrelevant. What I can get indirently from this answer is that ??? -> std::views::counted(p, p?1:0) could have been a perfectly acceptable answer but as written it missed the point. There is no need to generalize to other counts. If anything, a generalization could be to any nullable pointer-like type (for example T*, std::unique_ptr<T>, etc). – Elevon

@Elevon It's a pitty cause this is exactly contained in the answer, I didn't know it was trouble to extract. Nikos Athasaniou surely you can deduce from the comments what update is needed no? – Essive