How to declare a vector of atomic in C++

Question

I am intending to declare a vector of atomic variables to be used as counters in a multithreaded programme. Here is what I tried:

#include <atomic>
#include <vector>

int main(void)
{
  std::vector<std::atomic<int>> v_a;
  std::atomic<int> a_i(1);
  v_a.push_back(a_i);
  return 0;
}

And this is the annoyingly verbose error message of gcc 4.6.3:

In file included from /usr/include/c++/4.6/x86_64-linux-gnu/./bits/c++allocator.h:34:0,
             from /usr/include/c++/4.6/bits/allocator.h:48,
             from /usr/include/c++/4.6/vector:62,
             from test_atomic_vec.h:2,
             from test_atomic_vec.cc:1:
/usr/include/c++/4.6/ext/new_allocator.h: In member function ‘void __gnu_cxx::new_allocator<_Tp>::construct(__gnu_cxx::new_allocator<_Tp>::pointer, const _Tp&) [with _Tp = std::atomic<int>, __gnu_cxx::new_allocator<_Tp>::pointer = std::atomic<int>*]’:
/usr/include/c++/4.6/bits/stl_vector.h:830:6:   instantiated from ‘void std::vector<_Tp, _Alloc>::push_back(const value_type&) [with _Tp = std::atomic<int>, _Alloc = std::allocator<std::atomic<int> >, std::vector<_Tp, _Alloc>::value_type = std::atomic<int>]’
test_atomic_vec.cc:10:20:   instantiated from here
/usr/include/c++/4.6/ext/new_allocator.h:108:9: error: use of deleted function ‘std::atomic<int>::atomic(const std::atomic<int>&)’
/usr/include/c++/4.6/atomic:538:7: error: declared here
In file included from /usr/include/c++/4.6/vector:70:0,
             from test_atomic_vec.h:2,
             from test_atomic_vec.cc:1:
/usr/include/c++/4.6/bits/vector.tcc: In member function ‘void std::vector<_Tp, _Alloc>::_M_insert_aux(std::vector<_Tp, _Alloc>::iterator, _Args&& ...) [with _Args = {const std::atomic<int>&}, _Tp = std::atomic<int>, _Alloc = std::allocator<std::atomic<int> >, std::vector<_Tp, _Alloc>::iterator = __gnu_cxx::__normal_iterator<std::atomic<int>*, std::vector<std::atomic<int> > >, typename std::_Vector_base<_Tp, _Alloc>::_Tp_alloc_type::pointer = std::atomic<int>*]’:
/usr/include/c++/4.6/bits/stl_vector.h:834:4:   instantiated from ‘void std::vector<_Tp, _Alloc>::push_back(const value_type&) [with _Tp = std::atomic<int>, _Alloc = std::allocator<std::atomic<int> >, std::vector<_Tp, _Alloc>::value_type = std::atomic<int>]’
test_atomic_vec.cc:10:20:   instantiated from here
/usr/include/c++/4.6/bits/vector.tcc:319:4: error: use of deleted function ‘std::atomic<int>::atomic(const std::atomic<int>&)’
/usr/include/c++/4.6/atomic:538:7: error: declared here
/usr/include/c++/4.6/bits/stl_vector.h:834:4:   instantiated from ‘void std::vector<_Tp, _Alloc>::push_back(const value_type&) [with _Tp = std::atomic<int>, _Alloc = std::allocator<std::atomic<int> >, std::vector<_Tp, _Alloc>::value_type = std::atomic<int>]’
test_atomic_vec.cc:10:20:   instantiated from here
/usr/include/c++/4.6/bits/vector.tcc:319:4: error: use of deleted function ‘std::atomic<int>& std::atomic<int>::operator=(const std::atomic<int>&)’
/usr/include/c++/4.6/atomic:539:15: error: declared here
In file included from /usr/include/c++/4.6/x86_64-linux-gnu/./bits/c++allocator.h:34:0,
             from /usr/include/c++/4.6/bits/allocator.h:48,
             from /usr/include/c++/4.6/vector:62,
             from test_atomic_vec.h:2,
             from test_atomic_vec.cc:1:
/usr/include/c++/4.6/ext/new_allocator.h: In member function ‘void __gnu_cxx::new_allocator<_Tp>::construct(__gnu_cxx::new_allocator<_Tp>::pointer, _Args&& ...) [with _Args = {std::atomic<int>}, _Tp = std::atomic<int>, __gnu_cxx::new_allocator<_Tp>::pointer = std::atomic<int>*]’:
/usr/include/c++/4.6/bits/vector.tcc:306:4:   instantiated from ‘void std::vector<_Tp, _Alloc>::_M_insert_aux(std::vector<_Tp, _Alloc>::iterator, _Args&& ...) [with _Args = {const std::atomic<int>&}, _Tp = std::atomic<int>, _Alloc = std::allocator<std::atomic<int> >, std::vector<_Tp, _Alloc>::iterator = __gnu_cxx::__normal_iterator<std::atomic<int>*, std::vector<std::atomic<int> > >, typename std::_Vector_base<_Tp, _Alloc>::_Tp_alloc_type::pointer = std::atomic<int>*]’
/usr/include/c++/4.6/bits/stl_vector.h:834:4:   instantiated from ‘void std::vector<_Tp, _Alloc>::push_back(const value_type&) [with _Tp = std::atomic<int>, _Alloc = std::allocator<std::atomic<int> >, std::vector<_Tp, _Alloc>::value_type = std::atomic<int>]’
test_atomic_vec.cc:10:20:   instantiated from here
/usr/include/c++/4.6/ext/new_allocator.h:114:4: error: use of deleted function ‘std::atomic<int>::atomic(const std::atomic<int>&)’
/usr/include/c++/4.6/atomic:538:7: error: declared here
In file included from /usr/include/c++/4.6/vector:61:0,
             from test_atomic_vec.h:2,
             from test_atomic_vec.cc:1:
/usr/include/c++/4.6/bits/stl_algobase.h: In static member function ‘static _BI2 std::__copy_move_backward<true, false, std::random_access_iterator_tag>::__copy_move_b(_BI1, _BI1, _BI2) [with _BI1 = std::atomic<int>*, _BI2 = std::atomic<int>*]’:
/usr/include/c++/4.6/bits/stl_algobase.h:581:18:   instantiated from ‘_BI2 std::__copy_move_backward_a(_BI1, _BI1, _BI2) [with bool _IsMove = true, _BI1 = std::atomic<int>*, _BI2 = std::atomic<int>*]’
/usr/include/c++/4.6/bits/stl_algobase.h:590:34:   instantiated from ‘_BI2 std::__copy_move_backward_a2(_BI1, _BI1, _BI2) [with bool _IsMove = true, _BI1 = std::atomic<int>*, _BI2 = std::atomic<int>*]’
/usr/include/c++/4.6/bits/stl_algobase.h:661:15:   instantiated from ‘_BI2 std::move_backward(_BI1, _BI1, _BI2) [with _BI1 = std::atomic<int>*, _BI2 = std::atomic<int>*]’
/usr/include/c++/4.6/bits/vector.tcc:313:4:   instantiated from ‘void std::vector<_Tp, _Alloc>::_M_insert_aux(std::vector<_Tp, _Alloc>::iterator, _Args&& ...) [with _Args = {const std::atomic<int>&}, _Tp = std::atomic<int>, _Alloc = std::allocator<std::atomic<int> >, std::vector<_Tp, _Alloc>::iterator = __gnu_cxx::__normal_iterator<std::atomic<int>*, std::vector<std::atomic<int> > >, typename std::_Vector_base<_Tp, _Alloc>::_Tp_alloc_type::pointer = std::atomic<int>*]’
/usr/include/c++/4.6/bits/stl_vector.h:834:4:   instantiated from ‘void std::vector<_Tp, _Alloc>::push_back(const value_type&) [with _Tp = std::atomic<int>, _Alloc = std::allocator<std::atomic<int> >, std::vector<_Tp, _Alloc>::value_type = std::atomic<int>]’
test_atomic_vec.cc:10:20:   instantiated from here
/usr/include/c++/4.6/bits/stl_algobase.h:546:6: error: use of deleted function ‘std::atomic<int>& std::atomic<int>::operator=(const std::atomic<int>&)’
/usr/include/c++/4.6/atomic:539:15: error: declared here
In file included from /usr/include/c++/4.6/vector:63:0,
             from test_atomic_vec.h:2,
             from test_atomic_vec.cc:1:
/usr/include/c++/4.6/bits/stl_construct.h: In function ‘void std::_Construct(_T1*, _Args&& ...) [with _T1 = std::atomic<int>, _Args = {std::atomic<int>}]’:
/usr/include/c++/4.6/bits/stl_uninitialized.h:77:3:   instantiated from ‘static _ForwardIterator std::__uninitialized_copy<_TrivialValueTypes>::__uninit_copy(_InputIterator, _InputIterator, _ForwardIterator) [with _InputIterator = std::move_iterator<std::atomic<int>*>, _ForwardIterator = std::atomic<int>*, bool _TrivialValueTypes = false]’
/usr/include/c++/4.6/bits/stl_uninitialized.h:119:41:   instantiated from ‘_ForwardIterator std::uninitialized_copy(_InputIterator, _InputIterator, _ForwardIterator) [with _InputIterator = std::move_iterator<std::atomic<int>*>, _ForwardIterator = std::atomic<int>*]’
/usr/include/c++/4.6/bits/stl_uninitialized.h:259:63:   instantiated from ‘_ForwardIterator std::__uninitialized_copy_a(_InputIterator, _InputIterator, _ForwardIterator, std::allocator<_Tp>&) [with _InputIterator = std::move_iterator<std::atomic<int>*>, _ForwardIterator = std::atomic<int>*, _Tp = std::atomic<int>]’
/usr/include/c++/4.6/bits/stl_uninitialized.h:269:24:   instantiated from ‘_ForwardIterator std::__uninitialized_move_a(_InputIterator, _InputIterator, _ForwardIterator, _Allocator&) [with _InputIterator = std::atomic<int>*, _ForwardIterator = std::atomic<int>*, _Allocator = std::allocator<std::atomic<int> >]’
/usr/include/c++/4.6/bits/vector.tcc:343:8:   instantiated from ‘void std::vector<_Tp, _Alloc>::_M_insert_aux(std::vector<_Tp, _Alloc>::iterator, _Args&& ...) [with _Args = {const std::atomic<int>&}, _Tp = std::atomic<int>, _Alloc = std::allocator<std::atomic<int> >, std::vector<_Tp, _Alloc>::iterator = __gnu_cxx::__normal_iterator<std::atomic<int>*, std::vector<std::atomic<int> > >, typename std::_Vector_base<_Tp, _Alloc>::_Tp_alloc_type::pointer = std::atomic<int>*]’
/usr/include/c++/4.6/bits/stl_vector.h:834:4:   instantiated from ‘void std::vector<_Tp, _Alloc>::push_back(const value_type&) [with _Tp = std::atomic<int>, _Alloc = std::allocator<std::atomic<int> >, std::vector<_Tp, _Alloc>::value_type = std::atomic<int>]’
test_atomic_vec.cc:10:20:   instantiated from here
/usr/include/c++/4.6/bits/stl_construct.h:76:7: error: use of deleted function ‘std::atomic<int>::atomic(const std::atomic<int>&)’
/usr/include/c++/4.6/atomic:538:7: error: declared here

How do I solve this?

The error disappears when I comment out the line with push_back() .

edit: I edited the post... For those of you who saw the first post, the error was embarrassingly that I used gcc instead of g++ :\

Clotheshorse

have you tried to add -lstdc++ flag? – Milkman

Nothing to do with atomics, at all. – Lelandleler

@Mat: correct... But the problem occurred in real code with g++ just my minimal example was stupidly compiled with gcc. See edit. – Clotheshorse

@Kaz: Not really duplicate but a hint in the right direction. Thank you. – Clotheshorse

Shows why it's a shame they didn't bring in concepts. Might have generated a much cleaner error message. – Neutral

Closing it as a duplicate of #12003524 would at least make some sense, but closing it as a duplicate of a question about undefined symbols is complete nonsense. Voting to reopen. – Richella

btw a vector (or aray) of atomics may probably lead to false sharing ( drdobbs.com/parallel/eliminate-false-sharing/217500206 ). it's usually better to distribute atomics more sparsely in memory – Poler

@jogojapan: the problem is that the OP completely changed the question in between. It used to be about undefined symbols – Lelandleler

@steffen: Please Oh Please, never completely change your question in between. You are not limited in the number of questions you can open, so if you progress further and have another question, then let the old one drop and ask a new one. This is not a forum, it's a QA website. – Lelandleler

@MatthieuM. You are right, learned my lesson ;) – Clotheshorse

@steffen: no problem, happens quite frequently :) (that and people asking further questions in answers) – Lelandleler

I duplicated a question three years before that question was asked O_o. That's quite cool :) – Clotheshorse

Answer 1

As described in this closely related question that was mentioned in the comments, std::atomic<T> isn't copy-constructible, nor copy-assignable.

Object types that don't have these properties cannot be used as elements of std::vector.

However, it should be possible to create a wrapper around the std::atomic<T> element that is copy-constructible and copy-assignable. It will have to use the load() and store() member functions of std::atomic<T> to provide construction and assignment (this is the idea described by the accepted answer to the question mentioned above):

#include <atomic>
#include <vector>

template <typename T>
struct atomwrapper
{
  std::atomic<T> _a;

  atomwrapper()
    :_a()
  {}

  atomwrapper(const std::atomic<T> &a)
    :_a(a.load())
  {}

  atomwrapper(const atomwrapper &other)
    :_a(other._a.load())
  {}

  atomwrapper &operator=(const atomwrapper &other)
  {
    _a.store(other._a.load());
  }
};

int main(void)
{
  std::vector<atomwrapper<int>> v_a;
  std::atomic<int> a_i(1);
  v_a.push_back(a_i);
  return 0;
}

EDIT: As pointed out correctly by Bo Persson, the copy operation performed by the wrapper is not atomic. It enables you to copy atomic objects, but the copy itself isn't atomic. This means any concurrent access to the atomics must not make use of the copy operation. This implies that operations on the vector itself (e.g. adding or removing elements) must not be performed concurrently.

Example: If, say, one thread modifies the value stored in one of the atomics while another thread adds new elements to the vector, a vector reallocation may occur and the object the first thread modifies may be copied from one place in the vector to another. In that case there would be a data race between the element access performed by the first thread and the copy operation triggered by the second.

Richella

that seems to do the job... Is there any downside to this solution? If not... why isn't this implemented in std::atomic in the first place? – Clotheshorse

One problem is that _a.store(other._a.load()); doesn't look very atomic to me. Is this useful? – Tiresome

@Clotheshorse The only downside I am aware of is that the implementation will have to take all necessary precautions to ensure all copies and assignments are performed atomically. This may involve memory fences and locks and therefore slow down the insertion of elements in the vector as well as reallocation and copy operations performed on the vector itself. But that is what it means to work with atomics. You can pass special memory model parameters as arguments to the load and store calls to improve things, but choosing the right memory model (other than the safe default one) is an art. – Richella

@BoPersson You are basically saying a mere assignment would perform the same operation as store, correct? Yes, I think that's right. Using store() explicitly helps emphasize that an atomic store operation happens there. – Richella

I see a problem with this not being one, but two, atomic operations. Just wonder if that is really useful and what the OP needs. – Tiresome

@BoPersson Ah. That's right. For the OP: Indeed this implementation enables concurrent access to the elements of the vector, but not the vector itself. If, say, one thread modifies the value stored in one of the atomics while another thread adds new elements to the vector, a vector reallocation may occur and the object the first thread modifies may be copied from one place in the vector to another. In that case there would be a data race between the element access performed by the first thread and the copy operation triggered by the second. – Richella

@BoPersson,@jogojapan: Good point. For the application I am working on right now it is ok, because I can separate initialisation and usage. But it's something to keep in mind. – Clotheshorse

@Richella I hope nobody assumes that a collection of concurrent objects is a concurrent collection of objects. – Associative

Your constructors should take an order = seq_cst default arg, for use as the order for loads. I guess there's no way to get vector use it. Or maybe you should default to relaxed since this constructor is only intended for use while copying temporaries during construction or growth, and you can't resize / reserve / emplace_back on a vector that other threads have pointers into. Especially for the .store() in the copy-assignment operator. seq-cst stores are expensive. (And you can only do this while you hold a lock). – Transeunt

Also make sure you don't do a[i] = b; and get one of these overloads. The OP's example is silly; copying from another std::atomic<int> a_i is really weird. – Transeunt

What I mean is: you only need to support push_back / emplace_back(int) (or const T&) not atomic_int / const atomic<T> &. You should still force the user to explicitly load the value from some other atomic object. (And of course only one thread can grow the vector because the control block isn't atomic. But if you've reserved enough capacity to guarantee no reallocation, one thread can append while other threads are loading and storing earlier elements of the vector.) – Transeunt

The core claim that Object types that don't have these properties cannot be used as elements of std::vector doesn't seem to be true, at least in C++11 and beyond. Objects without copy or move constructors can be used as long as you don't use operations on the vector that would require them, as described here. This doesn't mean you can't use push_back or emplace_back, but this isn't as restrictive as it sounds: moving an std::atomic<> object in memory pretty much breaks any concurrent accessors, so it's not a realistic operation. – Housebreaking

This is probably the best answer among the answers, but it's worth noting that you can use std::atomic as the vector type as long as you know how many elements you're going to have in advance. That is, if you know how many you'll have at the time you call the initializer, you'll be fine, such as vector<atomic<float>> vars(number_of_elements); to construct [number_of_elements] atomic floats which you can then use as you wish. It won't work with resize or push_back from there, though. – Dorm

apart from operation on the vector itself, like adding or removing elements that are of course not atomic. Is accessing an element of that vector of atomics atomic?, let's say I have a vector v1, and I want to perform the operation v1[3].store(5); will the access to that 3rd element of the vector v1 be atomic? – Statist

Answer 2

To first answer your headline question: you can declare a std::vector<std::atomic<...>> easily, as you have done in your example.

Because of the lack of copy or move constructors for std::atomic<> objects, however, your use of the vector will be restricted as you found out with the compilation error on push_back(). Basically you can't do anything that would invoke either constructor.

This means your vector's size must be fixed at construction, and you should manipulate elements using operator[] or .at(). For your example code, the following works¹:

std::vector<std::atomic<int>> v_a(1);
std::atomic<int> a_i(1); 
v_a[0] = a_i.load();

If the "fixed size at construction" limitation is too onerous, you can use std::deque instead. This lets you emplace objects, growing the structure dynamically without requiring copy or move constructors, e.g.:

std::deque<std::atomic<int>> d;

d.emplace_back(1);
d.emplace_back(2);
d.pop_back();

There are still some limitations, however. For example, you can pop_back(), but you cannot use the more general erase(). The limitations make sense: an erase() in the middle of the blocks of contiguous storage used by std::deque in general requires the movement of elements, hence requires copy/move constructor or assignment operators to be present.

If you can't live with those limitations, you could create a wrapper class as suggested in other answers but be aware of the underlying implementation: it makes little sense to move a std::atomic<> object once it is being used: it would break any threads concurrently accessing the objects. The only sane use of copy/move constructors is generally in the initial setup of collections of these objects before they are published to other threads.

---

¹ Unless, perhaps, you use Intel's icc compiler, which fails with an internal error while compiling this code.

Housebreaking

Comments are not for extended discussion; this conversation has been moved to chat. – Mlawsky

this line v_a[0] = a_i; should be v_a[0] = a_i.load(); tried doing an edit but says suggestions are full. – Albedo

@Albedo - good catch, I've fixed it up. – Housebreaking

Answer 3

Looks to me like atomic<T> has no copy constructor. Nor a move constructor, as far as I can tell.

One work around might be to use vector<T>::emplace_back() to construct the atomic in-place in the vector. Alas, I don't have a C++11 compiler on me right now, or I'd go and test it.

Morocco

That seems to be the problem. putting v_a.push_back(std::move(a_i)); doesn't solve it though. – Clotheshorse

@Clotheshorse You're right, atomic doesn't have a move constructor either. I have updated my answer with a work-around that is different to the accepted one. – Morocco

I tried using emplace_back(1) instead of push_back(a_i), but GCC 7.2 rejects that, saying that the necessary unitialized-copy operation requires the copy constructor. I guess that is due to possible reallocations when a new element is inserted. Anyway, even if certain compilers accepted it, it would still be an incorrect use of std::vector, at least formally, because you are not supposed to use a non-copy-assignable element type. – Richella

@Richella I just had a look at the C++ standard, and containers requirements are set on a per-operation basis (see section 23.2). That means, that there is no such a thing as an incorrect use of std::vector based on the element type. Of course, depending on the element type that you use, you will not be able to perform some operations. But if you do not need those operations, then using that element type should be totally fine. – Valorous

@Valorous Yes, strictly speaking that's right. The requirements are defined per operation, because that is how templates work in C++ -- only functions actually used are instantiated. However, it seems pretty clear that the idea is to have certain requirements on a per-class basis, and to actually enforce these using the concept of "concepts" (which unfortunately didn't make it into C++11). If you use GCC with the -D_GLIBCXX_CONCEPT_CHECKS option, the compiler will perform a limited set of per-class checks, and that did include the copy-assignment check for vector the last time I checked. – Richella

emplace_back doesn't work because it might have to grow the vector, and that requires copying the existing elements, requiring the non-existent copy-constructor for the atomic<T> elements. There's no "unsafe_emplace_back" that doesn't check capacity. You could define a wrapper for atomic that had a copy constructor, but probably just don't. – Transeunt

Answer 4

As others have properly noted, the cause of the compiler's error is that std::atomic explicitly prohibits the copy constructor.

I had a use case where I wanted the convenience of an STL map (specifically I was using a map of maps in order to achieve a sparse 2-dimensional matrix of atomics so I can do something like int val = my_map[10][5]). In my case there would be only one instance of this map in the program, so it wouldn't be copied, and even better I can initialize the entire thing using braced initialization. So it was very unfortunate that while my code would never attempt copying individual elements or the map itself, I was prevented from using an STL container.

The workaround I ultimately went with is to store the std::atomic inside a std::shared_ptr. This has pros, but possibly a con:

Pros:

• Can store std::atomic inside any STL container
• Does not require/restrict using only certain methods of STL containers.

Pro or Con (this aspect's desirability depends on the programs' use cases): - There is only a single shared atomic for a given element. So copying the shared_ptr or the STL container will still yield a single shared atomic for the element. In other words, if you copy the STL container and modify one of the atomic elements, the other container's corresponding atomic element will also reflect the new value.

In my case the Pro/Con characteristic was moot due to my use case.

Here's a brief syntax to initialize a std::vector with this method:

#include <atomic>
#include <memory>
#include <vector>

std::vector<std::shared_ptr<std::atomic<int> > > vecAtomicInts
{
    std::shared_ptr<std::atomic<int> >(new std::atomic<int>(1) ),
    std::shared_ptr<std::atomic<int> >(new std::atomic<int>(2) ),
};

// push_back, emplace, etc all supported
vecAtomicInts.push_back(std::shared_ptr<std::atomic<int> >(new std::atomic<int>(3) ) );

// operate atomically on element
vecAtomicInts[1]->exchange(4);

// access random element
int i = *(vecAtomicInts[1]);

Particulate

Note to the modern reader: Use make_shared instead of the naked new operator found in the last code snippet. – Reveille