OpenMp Task: can't pass argument by reference

Question

g++ -fopenmp main.cpp complains about undefined reference to std::vector. How to fix this?

I have installed the libomp-dev package on Ubuntu.

main.cpp

#include<vector>
#include<iostream>

template<typename T, typename A>
T recursiveSumBody(std::vector<T, A> &vec) {
    T sum = 0;
    #pragma omp task shared(sum)
    {
        sum = recursiveSumBody(vec);
    }
    return vec[0];
}

int main() {
    std::vector<int> a;
    recursiveSumBody(a);
    return 0;
}

Undefined References

/tmp/ccTDECNm.o: In function `int recursiveSumBody<int, std::allocator<int> >(std::vector<int, std::allocator<int> >&) [clone ._omp_cpyfn.1]':
main.cpp:(.text+0x148): undefined reference to `std::vector<int, std::allocator<int> >::vector(std::vector<int, std::allocator<int> > const&)'
collect2: error: ld returned 1 exit status

Arbitrage

Anyone has seen something like this? I guess I can use pointer to the 0th element of the vector, instead of a std::vector, but I would rather not use pointers directly if possible. – Arbitrage

Note that libomp-dev is the LLVM OpenMP runtime that is unrelated to gomp, which is the OpenMP runtime bundled by gcc. – Premium

Answer 1

To fix the issue, you can manually specify shared(sum, vec) (strongly assuming you want it shared).

Interestingly enough older gcc versions (e.g. 5.4.0) give a much more helpful error message:

error: 'vec' implicitly determined as 'firstprivate' has reference type

Whereas the Intel compiler icpc 17.0.1 gives an "internal error : 0_1855".

Manually specifying firstprivate or private - which makes little sense in your case - results in other more descriptive errors. Note, as Hristo Iliev explained in the other comments, firstprivate would mean that a copy of the vector is made for each thread.

As per the current (4.5) standard:

In an orphaned task generating construct, if no default clause is present, formal arguments passed by reference are firstprivate.

I suppose that applies here. Further,

A variable that appears in a firstprivate clause must not have an incomplete C/C++ type or be a reference to an incomplete type. If a list item in a firstprivate clause on a worksharing construct has a reference type then it must bind to the same object for all threads of the team.

^{It doesn't appear in a clause, but I think this is still what the standard means.}

Now I don't think that std::vector<T, A> is an incomplete type within the template, unless I am missing something about how templates are instantiated. So I do think your code should be valid and given that each thread just binds to the same object, it actually would make sense.

So I do think this is a bug in recent gcc versions as well as the Intel compiler. It looks like the compiler fails to instantiate some things for the template.

Further, adding:

if (0) std::vector<T, A> wtf = vec;

at the beginning of the function makes the code compile and link with gcc. But if firstprivate is added manually, gcc continues to complain that 'vec' has incomplete type.

P.S.: Allowing reference types in data sharing attribute clauses was added in OpenMP 4.5, this is the old gcc gives a different error.

Premium

Intel 18.0b complains that vec is an incomplete or reference type if explicitly specified in a firstprivate clause and compiles without problems otherwise. The program crashes when run because of the infinite recursion. – Lane

Answer 2

This looks like a bug in GCC, which fails to generate a copy constructor for std::vector<int, std::allocator<int> >. Note that the error comes from the linker and does not occur during the compilation phase. The copy constructor is used in the copy function that initialises the firstprivate parameters of the outlined task function. Forcing the compiler to generate it, e.g. changing

std::vector<int> a;

to

std::vector<int> a, b(a);

fixes the problem.

Here is a more elaborate description. GCC transforms the following code

#pragma omp task shared(sum)
{
    sum = recursiveSumBody(vec);
}

into something like:

struct omp_data_a data_o;

data_o.vec = vec;
data_o.sum = ∑
GOMP_task(omp_fn_0, &data_o, omp_cpyfn_1, 32, 8, 1, 0, 0, 0);

// --- outlined task body ---
void omp_fn_0(struct omp_data_s & restrict data_i)
{
   struct vector & vec = &data_i->vec;
   *data_i->sum = recursiveSumBody<int>(vec);
   std::vector<int>::~vector(vec);
}

// --- task firstprivate initialisation function ---
void omp_cpyfn_1(struct omp_data_s *data_o, struct omp_data_a *data_i)
{
   data_o->sum = data_i->sum;
   struct vector &d40788 = data_i->vec;
   struct vector *this = &data_o->vec;
   std::vector<int>::vector(this, d40788); // <--- invocation of the copy constructor
}

omp_cpyfn_1 gets called by GOMP_task() in order to initialise the firstprivate arguments. It calls the copy constructor of std::vector<int>, because (first-)private treats references to type T as type T itself, but the constructor is not generated, therefore the object code fails to link. This is probably a bug in the gimplifier code as the copy constructor gets created when a non-reference std::vector<T, A> gets privatised, e.g., with code like this:

...
std::vector<T, A> b;
#pragma omp task shared(sum)
{
    sum = recursiveSumBody(b);
}
...

---

The code compiles with Intel 18.0b. Explicitly specifying vec as firstprivate breaks it the same way as with GCC (icpc complains about vec being of an incomplete type). The following workaround could be used:

template<typename T, typename A>
T recursiveSumBody(std::vector<T, A> &vec) {
    T sum = 0;
    std::vector<T, A> *ptr = &vec;
    #pragma omp task shared(sum)
    {
        sum = recursiveSumBody(*ptr);
    }
    return vec[0];
}

In this case ptr is a pointer. Its firstprivate version is another pointer that points to the same location, i.e. the vector instance. The semantics differs from the original code as here no private copy of the entire vector gets created, rather the original vector is used.

Lane

Answer 3

The problem disappears if you also declare vec as a shared variable:

#pragma omp task shared(sum, vec)

It seems that the default visibility for task is firstprivate, not shared as would be expected. You can find more information in this forum entry.

Homophonous

Tasks are pieces of code that could (and possibly would) execute in a future moment of time. As those are usually expected to work on different set of input values provided in the same variables, it is more natural to expect those to be firstprivate and not shared. Think asynchronous closures. – Lane

@HristoIliev I found the semantics of firstprivate on references not immediately intuitive (think auto firstprivate = reference). Would you agree that there is no semantic difference between firstprivate and shared for references? – Premium

@Zulan, references are alternative names of some memory object and not pointers, therefore firstprivate, following the semantics of private, creates a copy of the entire object and not simply another reference to it. – Lane

@HristoIliev I might have misread "If a list item in a firstprivate clause on a worksharing construct has a reference type then it must bind to the same object for all threads of the team.". I can't find any other good specific description of how references are handled for the different data-sharing clauses in the standard. – Premium

@Premium "If the type of a list item is a reference to a type T then the type will be considered to be T for all purposes of this clause" (p.194, private Clause) and "A list item that appears in a firstprivate clause is a subject to the private clause semantics ..., except as noted" (p.197, firstprivate Clause). Also, note that task is not a worksharing construct. – Lane