How to combine associative arrays in bash?

Question

Does anyone know of an elegant way to combine two associative arrays in bash just like you would a normal array? Here's what I'm talking about:

In bash you can combine two normal arrays as follows:

declare -ar array1=( 5 10 15 )
declare -ar array2=( 20 25 30 )
declare -ar array_both=( ${array1[@]} ${array2[@]} )

for item in ${array_both[@]}; do
    echo "Item: ${item}"
done

I want to do the same thing with two associative arrays, but the following code does not work:

declare -Ar array1=( [5]=true [10]=true [15]=true )
declare -Ar array2=( [20]=true [25]=true [30]=true )
declare -Ar array_both=( ${array1[@]} ${array2[@]} )

for key in ${!array_both[@]}; do
    echo "array_both[${key}]=${array_both[${key}]}"
done

It gives the following error:

./associative_arrays.sh: line 3: array_both: true: must use subscript when assigning associative array

The following is a work-around I came up with:

declare -Ar array1=( [5]=true [10]=true [15]=true )
declare -Ar array2=( [20]=true [25]=true [30]=true )
declare -A array_both=()

for key in ${!array1[@]}; do
    array_both+=( [${key}]=${array1[${key}]} )
done

for key in ${!array2[@]}; do
    array_both+=( [${key}]=${array2[${key}]} )
done

declare -r array_both

for key in ${!array_both[@]}; do
    echo "array_both[${key}]=${array_both[${key}]}"
done

But I was hoping that I'm actually missing some grammar that will allow the one-liner assignment as shown in the non-working example.

Thanks!

August

A one liner would require being able to expand an array into [key]=value items for each key. I don't know of any such expansion. The closest I can think of is what declare -p gives you (which you would need to massage to use). – Seringapatam

Well, I've spent quite a lot of time toying with arrays, parameter expansion, and bash variables. I think it's safe to say that the "workaround" in your question is the cleanest way to copy an associative array. Still, I could make your script into a "one-liner" with a handful of semicolons if you really wanted... ;) – Theravada

this is what I did: https://mcmap.net/q/356999/-creating-array-of-objects-in-bash – Strategy

Answer 1

I don't have a one-liner either but here is a different 'workaround' that someone might like using string convertion. It's 4 lines, so I'm only 3 semi-colons from the answer you wanted!

declare -Ar array1=( [5]=true [10]=true [15]=true )
declare -Ar array2=( [20]=true [25]=true [30]=true )

# convert associative arrays to string
a1="$(declare -p array1)"
a2="$(declare -p array2)"

#combine the two strings trimming where necessary 
array_both_string="${a1:0:${#a1}-3} ${a2:21}"

# create new associative array from string
eval "declare -A array_both="${array_both_string#*=}

# show array definition
for key in ${!array_both[@]}; do
    echo "array_both[${key}]=${array_both[${key}]}"
done

Immovable

Yep, that's certainly an alternative work-around which builds off of Etan's comment. It's not exactly intuitive looking at the code though, so I'll probably stick with my workaround. – August

Answer 2

this works with bash > 4.3

I create mi own function merge_associative_array to do this:

declare -A optionsA
optionsA=( ["--hello"]="HELLO" ["--world"]="WORLD" )
declare -A optionsB
optionsB=( ["--key1"]="keyval" ["--world"]="WORLD2" ["--new-key"]="xyz" )

merge_associative_array "optionsA" "optionsB"

here the implementation with an useful array printer function:

print_associative_array() {
    # declare a local **reference variable** (hence `-n`) named `array_reference`
    # which is a reference to the value stored in the first parameter
    # passed in
    echo "printing associative array: $1"
    local -n map_ref="$1"

    # print the array by iterating through all of the keys now
    for key in "${!map_ref[@]}"; do
        value="${map_ref["$key"]}"
        echo "  $key: $value"
    done
}
merge_associative_array() {
    # declare a local **reference variable** (hence `-n`) named `array_reference`
    # which is a reference to the value stored in the first parameter
    # passed in
    echo "merging associative arrays: $1 <--- $2"
    local -n map_ref="$1"
    local -n map_ref2="$2"

    # setting the value of keys in the second array, to the value of the same key in the first array
    for key in "${!map_ref2[@]}"; do
        value="${map_ref2["$key"]}"
        echo "  $key: $value"
        map_ref["$key"]="$value"
    done
    print_associative_array "$1"
}

I was inspired by answer of @Gabriel Staples

Toussaint

Answer 3

How about concatenating the output from 'declare -p' for the arrays (no reason it shouldn't work for an 'n' way as well, shown here) :

#! /bin/bash

declare -Ar array1=(  [5]=true [10]=true [15]=true )
declare -Ar array2=( [20]=true [25]=true [30]=true )
declare -Ar array3=( [35]=true [40]=true [45]=true )

# one liner:
eval declare -Ar array_both=($(declare -p array1 array2 array3 | sed -z -e $'s/declare[^(]*(//g' -e $'s/)[^ ]//g'))

# proof:
for k in ${!array_both[$*]} ; do
  echo array_both[$k]=${array_both[$k}
done

Coolant

Answer 4

Although this thread is old, I found it to be a super helpful question with insightful answers. Here's a similar approach to what @Wil explained.

Like that approach, this one does not use external commands (like sed).

The main difference is that it does an array-based merge instead of string-based merge. This allows for key-values to be overridden in a predictable way. It also supports assigning the merged array to a read-only variable like shown in the question.

merge_map()
{
    local -A merged_array
    local array_string
    while [ $# -gt 0 ]
    do
        array_string=$(declare -p $1)
        eval merged_array+=${array_string#*=}
        shift
    done
    array_string=$(declare -p merged_array)
    echo "${array_string#*=}"
}

echo -e "\nExample from question..."

# Values in posted question
declare -Ar array1=( [5]=true [10]=true [15]=true )
declare -Ar array2=( [20]=true [25]=true [30]=true )
eval declare -Ar array_both=$(merge_map array1 array2)

# Show result
for k in "${!array_both[@]}";{ echo "[$k]=${array_both[$k]}";}

echo -e "\nExpanded example..."

# Non-numeric keys; some keys and values have spaces; more than two maps
declare -Ar expansion1=( [five]=true [ten]=true [and fifteen]=true )
declare -Ar expansion2=( [20]="true or false" [not 25]="neither true nor false" [30]=true )
declare -Ar expansion3=( [30]="was true, now false" [101]=puppies)
eval declare -Ar expansion_all=$(merge_map expansion1 expansion2 expansion3)

# Show result
for k in "${!expansion_all[@]}";{ echo "[$k]=${expansion_all[$k]}";}

Pedestrianism

Answer 5

eval "array_both=( ${array1[*]@K} ${array2[*]@K} )"

See "Parameter Expansion" in man bash:

${parameter@operator}
    K      Produces a possibly-quoted version of the value of parameter,
           except that it prints the values of indexed and associative arrays
           as a sequence of quoted key-value pairs.

Ferocious

Answer 6

#!/bin/bash

function merge_hashes() {
    local -n DEST_VAR=$1
    shift

    local -n SRC_VAR
    local KEY

    for SRC_VAR in $@; do
        for KEY in "${!SRC_VAR[@]}"; do
            DEST_VAR[$KEY]="${SRC_VAR[$KEY]}"
        done
    done
}

declare -Ar array1=( [5]=true [10]=true [15]=true )
declare -Ar array2=( [20]=true [25]=true [30]=true )
declare -A array_both=()

# And here comes the one-liner:
merge_hashes array_both array1 array2

declare -p array_both

Silicone

Could you provide some explanation also? – Nellynelms

See nameref and name reference in bash(1). – Saxhorn

Answer 7

The main reason why your second attempt doesn't work is because you're trying to solve a different problem using the same solution.

In the first data set you have two numeric indexed arrays where the keys have no meaning other than possibly the order they appear in, and their values are what really matters. I interpreted that to mean you wanted to linear concatenate those values to a new array with a new index which discards the previous keys but maintains the original order of the elements as well as the order you passed them in.

The second data set you have two associative indexed arrays where the keys are the values and the values are really just placeholders. I noticed that you used numeric keys, which if you chose to keep using numeric indexed arrays would allow you to preserve both the order of the values and the order of the keys, on the assumption that you want the keys in ascending order...

So then for solving these problems I have 3 convenience functions I've written which use declare and eval to accelerate joining/merging large arrays rather than using loops to assign each. They also take a variable number of arrays as argument so you can join / merge / dump as many of them as you please.

NOTE: I changed the value/key "30" out for "30 30" to demonstrate how a string would behave differently than a number in some circumstances.

join_arrays(){
# <array> [<array> ...] <destination array>
# linear concatenates the values, re-keys the result.
# works best with indexed arrays where order is important but index value is not.
  local A_;
  while (( $# > 1 )); do
    A_+="\"\${$1[@]}\" ";
    shift;
  done
  eval "$1=($A_)";
}
# This works by building and running an array assignment command
# join_array a1 a2 a3 becomes a3=("${a1[@]" "$a2[@]" ); 

merge_arrays(){
# <array> [<array> ...] <destination array>
# merges the values, preserves the keys.
# works best with assoc arrays or to obtain union-like results.
# if a key exists in more than one array the latter shall prevail.

  local A_ B_;
  while (( $# > 1 )); do
    B_=`declare -p $1`;
    B_=${B_#*=??};
    A_+=${B_::-2}" ";
    shift;
  done
  eval "$1=($A_)";
}
# this crops the output of declare -p for each array
# then joining them into a single large assignment.
# try putting "echo" in front of the eval to see the result.


dump_arrays(){
# <array> [<array> ...]
# dumps array nodes in bash array subscript assignment format
# handy for use with array assignment operator.  Preseves keys.
# output is a join, but if you assign it you obtain a merge.

  local B_;
  while (( $# > 0 )); do
    B_=`declare -p $1`;
    B_=${B_#*=??};
    printf "%s " "${B_::-2}";
    shift;
  done
}
# same as above but prints it instead of performing the assignment


# The data sets, first the pair of indexed arrays:
declare -a array1=( 5 10 15 );
declare -a array2=( 20 25 "30 30" );
# then the set of assoc arrays:
declare -a array3=( [5]=true [10]=true [15]=true );
declare -a array4=( [20]=true [25]=true ["30 30"]=true );

# show them:
declare -p array1 array2 array3 array4;

# an indexed array for joins and an assoc array for merges:
declare -a joined;
declare -A merged;

# the common way to join 2 indexed arrays' values:
echo "joining array1+array2 using array expansion/assignment:";
joined=( "${array1[@]}" "${array2[@]}" );
declare -p joined;

declare -a joined='([0]="5" [1]="10" [2]="15" [3]="20" [4]="25" [5]="30 30")'

# this does exactly the same thing, mostly saves me from typos ;-)
echo "joining array1+array2 using join_array():";
join_arrays array1 array2 joined;
declare -p joined;

declare -a joined='([0]="5" [1]="10" [2]="15" [3]="20" [4]="25" [5]="30 30")'

# this merges them by key, which is inapropriate for this data set
# But I've included it for completeness to contrast join/merge operations
echo "merging array1+array2 using merge_array():";
merge_arrays array1 array2 merged;
declare -p merged;

declare -A merged='([0]="20" [1]="25" [2]="30 30" )'

# Example of joining 2 associative arrays:
# this is the usual way to join arrays but fails because
# the data is in the keys, not the values.
echo "joining array3+array4 using array expansion/assignment:"
joined=( "${array3[@]}" "${array4[@]}" );
declare -p joined;

declare -a joined='([0]="true" [1]="true" [2]="true" [3]="true" [4]="true" [5]="true")'

# and again, a join isn't what we want here, just for completeness.
echo "joining array3+array4 using join_array():";
join_arrays array3 array4 joined;
declare -p joined;

declare -a joined='([0]="true" [1]="true" [2]="true" [3]="true" [4]="true" [5]="true")'

# NOW a merge is appropriate, because we want the keys!
echo "merging array3+array4 using merge_array():"
merge_arrays array3 array4 merged;
declare -p merged;

declare -A merged='([25]="true" [20]="true" ["30 30"]="true" [10]="true" [15]="true" [5]="true" )'

# Bonus points - another easy way to merge arrays (assoc or indexed) by key

# Note: this will only work if the keys are numeric... 
join_arrays array1 array2 joined;
# error expected because one keys is "30 30" ...
eval joined+=(`dump_arrays merged`);

bash: 30 30: syntax error in expression (error token is "30")

declare -p joined

declare -a joined='([0]="5" [1]="10" [2]="15" [3]="20" [4]="25" [5]="30 30" [20]="true" [25]="true")'

# Note: assoc arrays will not be sorted, even if keys are numeric!
join_arrays array1 array2 joined;
eval merged+=(`dump_arrays joined`);
declare -p merged

declare -A merged='([25]="true" [20]="true" ["30 30"]="true" [10]="true" [15]="true" [0]="5" [1]="10" [2]="15" [3]="20" [4]="25" [5]="true30 30" )'

Final Note: above you can see Key [5] has the values of the two source arrays' key [5] concatenated because I used the += operator. If you're just using it for merging lists of flags, it's safe, but for merging lists of meaningful values with possible key collisions, its better to stick to the merge_array() function.

Impostor