What exactly is a tensor in theano?

Question

What exactly is a Tensor in Theano, and what is the precise connection with Tensors as they are typically understood in Physics or Math?

I went through the Theano at Glance and the Basic Tensor functionality, but I could not find a clear connection.

Liaoyang

Answer 1

There is a good breakdown of different physics/math ways to think of tensor in Jim Belk's answer to a question on math.stackexchange. After looking over the documentation on tensor and the various operations Theano provides I'd say Theano's notion of tensor corresponds to the first way of thinking of tensor. In Jim's words:

Tensors are sometimes defined as multidimensional arrays, in the same way that a matrix is a two-dimensional array. From this point of view, a matrix is certainly a special case of a tensor.

In any case, I don't see anything myself in the docs indicating that Theano's tensor implementation knows about global properties of manifolds or tensor products in linear algebra beyond defining dot-products and the like. This would indicate Theano is taking a local point of view in it's implementation as opposed to the global.

Deadandalive

As a Theano dev, I confirm this is the definition we use. I'll add that in our case, tensor are symbolic variable. We use ndarray to talk about the actual values. Our CPU implementation of ndarray are done by numpy.ndarray. – Tref

More, @nouiz! More explanation, please! :D Tell us about the value of a symbolic variable. Tell us about the CPU implementation! Googling around I see that lots of folks are asking questions in an attempt to learn more about Theano. – Radman

Answer 2

Tensor = multi-dimensional array

In the machine learning literature, tensors generally refer to multi-dimensional arrays.

In Theano they are specifically numerical arrays:

Theano is a Python library that allows you to define, optimize, and efficiently evaluate mathematical expressions involving multi-dimensional arrays.

But in other libraries they may support a broader range of datatypes:

Tensor

The primary data structure in TensorFlow programs. Tensors are N-dimensional (where N could be very large) data structures, most commonly scalars, vectors, or matrices. The elements of a Tensor can hold integer, floating-point, or string values.

Etymology

Tensor has a more specific meaning in mathematics as an abstraction of a multilinear map between vector spaces, but given a fixed basis such maps can be represented as multidimensional arrays, and it is from this usage that the machine learning term gets its name.

---

See also:

• Cross Validated: Tensors in neural network literature: what's the simplest definition out there?
• Cross Validated: Why the sudden fascination with tensors?

Keenan