Welcome to the 27th edition of the WatchTower! In this edition, we’ll break down and demystify the powerful mathematical objects known as tensors.

📰 Featured in This Edition

• Event: AI Industry Night (Week 9)

• Article: Tensors: a 10-second TLDR

🗓 Upcoming Events

Tensors: a 10-second TLDR

Ever wondered how magical "Tensors" actually work? We’ll start with a simple case, and build an intuition for this fundamental ML component :)

By Zac Saber

They're just arrays. /endarticle

Jk. I won’t assume knowledge here, but will still keep this short.

Let’s build our understanding from scratch, and start with one dimension.

A 0D tensor is one number. We call this a scalar, and we can represent it as:

But that’s boring. Let’s do 1D!

A 1D tensor is a row of scalars (numbers). We call this a vector!

We can represent this as:

→ x # where (x=4)

Or, visually:

It’s almost as simple. 2D = 2 dimensions; so we can just use X and Y, our two hopefully familiar standard axes!

→ x y # where (x=3,y=2)

A 2D tensor would simply have a shape of (x,y); so, the tensor (2,3) would simply represent a 2D matrix, like so:

Great, we've got our first tensor. It is genuinely that simple; just like an array!

Let's add a 3rd dimension (getting spicy...); and chuck it at the front:

→ b x y # where (b=3, x=4, y=4)

We'll call it b, as in block, because:

It looks like a block now! b has turned our 2D rectangle-tensor, into a 3D cube-tensor :)

Up until this point, I've been explaining things just in basic array syntax.

A quick reminder of our current example:

→ b x y # where (b=3,x=4,y=4)

Now, the same example: but written in the special Tensor syntax:

→ b w h # where (b=3,w=4,h=4)

It's that simple! Just swap out the x for w (width), and the y for h (height).

Now, let's add a final fourth dimension. This time, let's call it c, for color:

→ c b w h # where (c=3,b=3,w=4,h=4)

Now, you might be wondering why they're all different colors, and not the same.

But up until now, we've been discussing the shape of the tensors - not individuals.

To explain those, we need to adjust our understanding of tensor syntax slightly.

For now, let's leave out the b batch letter - that's simple, we know it just turns our 2D square into a 3D cube.

Each letter of our familiar:

→ c w h # where (c=3, w=4, h=4),

actually refers to the number of dimensions that EACH tensor has.

That means, c indicates each tensor will have a color of 3 dimensions - aka, RGB!

The same goes for w width, and h height.

I.e, in the array that we're picturing: each cell itself is an individual tensor!

Essentially: the cell positions are unique (no 2 cells have same x/y pair); and each have a corresponding, distinct, RGB value. Here’s an example:

Now, it's time to reintroduce our b from before :D

Remember, all our b does is make it 3D; i.e, stack all the 2D squares back to back, so that they form a cube :)

Finally, we end up with something like the following:

The same cube as before - except now with a much better understanding :)

I hope you found this TLDR to Tensors useful; if so, keep an eye out for a follow-on guide (at a similar pace) which will cover beginner tensor manipulation!

Sponsors

Closing Notes

We welcome any feedback / suggestions for future editions here or email us at [email protected].

Stay curious,