How to permutate tranposition in tensorflow?

Question

From the docs:

Transposes a. Permutes the dimensions according to perm.

The returned tensor's dimension i will correspond to the input dimension perm[i]. If perm is not given, it is set to (n-1...0), where n is the rank of the input tensor. Hence by default, this operation performs a regular matrix transpose on 2-D input Tensors.

But it's still a little unclear to me how should I be slicing the input tensor. E.g. from the docs too:

tf.transpose(x, perm=[0, 2, 1]) ==> [[[1  4]
                                      [2  5]
                                      [3  6]]

                                     [[7 10]
                                      [8 11]
                                      [9 12]]]

Why is it that perm=[0,2,1] produces a 1x3x2 tensor?

After some trial and error:

twothreefour = np.array([ [[1,2,3,4], [5,6,7,8], [9,10,11,12]] , 
                        [[13,14,15,16], [17,18,19,20], [21,22,23,24]] ])
twothreefour

[out]:

array([[[ 1,  2,  3,  4],
        [ 5,  6,  7,  8],
        [ 9, 10, 11, 12]],

       [[13, 14, 15, 16],
        [17, 18, 19, 20],
        [21, 22, 23, 24]]])

And if I transpose it:

fourthreetwo = tf.transpose(twothreefour) 
with tf.Session() as sess:
    init = tf.initialize_all_variables()
    sess.run(init)
    print (fourthreetwo.eval())

I get a 4x3x2 to a 2x3x4 and that sounds logical.

[out]:

[[[ 1 13]
  [ 5 17]
  [ 9 21]]

 [[ 2 14]
  [ 6 18]
  [10 22]]

 [[ 3 15]
  [ 7 19]
  [11 23]]

 [[ 4 16]
  [ 8 20]
  [12 24]]]

But when I use the perm parameter the output, I'm not sure what I'm really getting:

twofourthree = tf.transpose(twothreefour, perm=[0,2,1]) 
with tf.Session() as sess:
    init = tf.initialize_all_variables()
    sess.run(init)
    print (threetwofour.eval())

[out]:

[[[ 1  5  9]
  [ 2  6 10]
  [ 3  7 11]
  [ 4  8 12]]

 [[13 17 21]
  [14 18 22]
  [15 19 23]
  [16 20 24]]]

Why does perm=[0,2,1] returns a 2x4x3 matrix from a 2x3x4 ?

Trying it again with perm=[1,0,2]:

threetwofour = tf.transpose(twothreefour, perm=[1,0,2]) 
with tf.Session() as sess:
    init = tf.initialize_all_variables()
    sess.run(init)
    print (threetwofour.eval())

[out]:

[[[ 1  2  3  4]
  [13 14 15 16]]

 [[ 5  6  7  8]
  [17 18 19 20]]

 [[ 9 10 11 12]
  [21 22 23 24]]]

Why does perm=[1,0,2] return a 3x2x4 from a 2x3x4?

Does it mean that the perm parameter is taking my np.shape and transposing the tensor based on the elements based on my array shape?

I.e. :

_size = (2, 4, 3, 5)
randarray = np.random.randint(5, size=_size)

shape_idx = {i:_s for i, _s in enumerate(_size)}

randarray_t_func = tf.transpose(randarray, perm=[3,0,2,1]) 
with tf.Session() as sess:
    init = tf.initialize_all_variables()
    sess.run(init)
    tranposed_array = randarray_t_func.eval()
    print (tranposed_array.shape)

print (tuple(shape_idx[_s] for _s in [3,0,2,1]))

[out]:

(5, 2, 3, 4)
(5, 2, 3, 4)

Trihedron

Answer 1

I think perm is permuting the dimensions. For example perm=[0,2,1] is short for dim_0 -> dim_0, dim_1 -> dim_2, dim_2 -> dim_1. So for a 2D tensor, perm=[1,0] is just matrix transpose. Does this answer your question?

Ola

I think so, but I'm not sure. If you can link to the specific line of code in the repo or some clearer doc, that'll help a lot! Thanks in advance! – Trihedron

i think the example in the docs gives a good enough illustration of what's happening. dimension 0 is the inner matrices, and they're unchanged by the permutation, dimension 1 is the rows of the inner matrices and dimension 2 is the columns, and they're switched by the permutation. so row 1 of each inner matrix goes to column 1 of the same inner matrix. does this make sense? – Ola

It's a little confusing to read but i get it after trying to convince myself with more examples similar to the code i've posted. – Trihedron

I think the confusion stems from the thought that transpose is just a flip (M x N to N x M); however here it's permuting (moving around the dimensions). [2,6,8] with perm=[2,1,0] -> [8,6,2] and [1,3,5,7] with perm=[3,0,1,2] -> [7,1,3,5] – Betoken

Answer 2

A=[2,3,4] matrix, using perm(1,0,2) will get B=[3,2,4].

Explanation:

Index=(0,1,2)
A    =[2,3,4]
Perm =(1,0,2)
B    =(3,2,4)  --> Perm 1 from Index 1 (3), Perm 0 from Index 0 (2), Perm 2 from Index 2 (4) --> so get (3,2,4)

Mikimikihisa

Please add some explanation. – Protolithic