• they are multi-dimensional arrays that generalized vectors (1D arrays) and matrices (2D arrays)

• they are used to represent data and model parameters. For example, tensors can represent input data, weights, biases, and outputs of neural networks.

• they support mathematical operations (element wise operation, matrix multiplication)

  

Tensors in Numpy

OD tensors

import numpy as np

scalar = np.array(42)
print(scalar)
print(scalar.shape)  # Output: ()

1D tensors (Vectors)

vector = np.array([1, 2, 3, 4, 5])
print(vector)
print(vector.shape)  # Output: (5,) = 5 elements in one row

2D tensors (matrix)

matrix = np.array([[1, 2, 3], [4, 5, 6]])
print(matrix)
print(matrix.shape)  # Output: (2, 3)  = 2 rows, 3 column

#visual rep
Row 1: [1, 2, 3]
Row 2: [4, 5, 6]

3D tensors

tensor_3d = np.array([
    [[1, 2, 3], [4, 5, 6]],
    [[7, 8, 9], [10, 11, 12]]
])
print(tensor_3d)
print(tensor_3d.shape)  # Output: (2, 2, 3) = 2 matrices, 2 rows, 3 columns

#visual representation
Layer 0:
[
  [1, 2, 3],
  [4, 5, 6]
]

Layer 1:
[
  [7, 8, 9],
  [10, 11, 12]
]

4D tensor

tensor_4d = np.array([
    [
        [[1, 2], [3, 4]],
        [[5, 6], [7, 8]]
    ],
    [
        [[9, 10], [11, 12]],
        [[13, 14], [15, 16]]
    ]
])
print(tensor_4d)
print(tensor_4d.shape) 

# Output: (2, 2, 2, 2) 
# 2 3D tensors, each 3D tensor has 2 2D matrices, each 2D matrices has 2 rows
# each row has 2 columns

#visual representation
[
  [
    [
      [1, 2],
      [3, 4]
    ],
    [
      [5, 6],
      [7, 8]
    ]
  ],
  [
    [
      [9, 10],
      [11, 12]
    ],
    [
      [13, 14],
      [15, 16]
    ]
  ]
]

What are ranks and axis?

Rank/Order/Degree refers to the dimensions it has

• A scalar is a tensor of rank 0.
• A vector is a tensor of rank 1.
• A matrix is a tensor of rank 2.
• A 3D array (like a color image with height, width, and color channels) is a tensor of rank 3.