In this article, we discuss the basic operations on Vectors and Matrices and how we can represent the input data in the form of Matrices.

Vector just represents the co-ordinates that the point has in a given space.

Image for post
Image for post

Vectors also have a geometric interpretation: it's a ray that connects the origin of that space to the point.

Image for post
Image for post

A vector is quantified with magnitude and direction.

Image for post
Image for post

We can have vectors which have different directions but the same magnitude

Image for post
Image for post

Adding two vectors: We can add them elementwise

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

Subtracting two vectors:

Image for post
Image for post
Image for post
Image for post
We take the negative of the vector and then it’s just simple addition.
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

Dot Product of Vectors

Dot Product is the element-wise product of two vectors but the resulting quantity is a scalar.

Image for post
Image for post
Image for post
Image for post

u1 is the coordinate of u vector along the first dimension, u2 is the coordinate of u vector along the second dimension.

Similarly, for n dimensions, we write it as:

Image for post
Image for post

The two vectors must have the same dimensions(to compute the dot product)

Image for post
Image for post
Image for post
Image for post
The answer, in this case, would be negative of 0.5

Unit Vector

Any vector of magnitude 1 is called a unit vector.

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

The above one is also a unit vector, it’s not along the direction of the x1 axis or x2 axis.

Computing a unit vector in the direction of a given vector:

Image for post
Image for post

The above vector (2, 1.5) is not a unit vector and we want to find a unit vector in its direction.

Image for post
Image for post

And the easiest way of doing that would be taking this vector and dividing it by its magnitude(we take every element of the vector and divide it by its norm/magnitude).

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

And the same holds in 3 dimensions as well

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

Projection of vector onto another vector

Image for post
Image for post

Let’s say we have two vectors x and y and we want to find out the projection of x on y.

As is clear from the above image, this projection is going to have the same direction as the vector y, it’s magnitude may be different than the vector y.

Image for post
Image for post

In this case, the vector was shortened after projection but it is possible that after projection we get back a longer vector than the original one.

Image for post
Image for post
Image for post
Image for post

Example:

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

In 3D

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

The angle between two vectors

Let’ say we have two vectors x and y and we want to compute the angle between them.

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

The same formula can be applied for any number of dimensions.

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

Orthogonal Vectors:

Image for post
Image for post

Example:

In 2D

Image for post
Image for post

In 3D

Image for post
Image for post

In 5D

Image for post
Image for post

Why do we care about vectors?

Let’s take an example, where we have a cell phone and we want to predict whether the cell phone will hit the market or not.

We could represent various parameters/features(maybe like price, screen size, ram, launched within last months and so on) of the cell phone as a vector.

Image for post
Image for post
This quantity has 7 dimensions.

Likewise, we can represent the data for an employee(data like tenure in months, salary, no of skills, etc.) in a vector form.

Image for post
Image for post
This quantity has 6 dimensions.

We can represent an input image as a vector of values where each value represents some pixel value.

Image for post
Image for post
This quantity has 784 dimensions.

We can compute the angle between two vectors and say if they are similar to each other or very far away. For example. for a new input image, we can represent it first in the form of a vector, then compute the angle between this input vector and other vectors(images in DB converted to vectors) and tell which one is most similar to the input image.

So, as obvious from the above example, we can represent each of the data points as a vector.

Introduction to Matrices:

We can think of a matrice as a collection of many vectors.

Image for post
Image for post

We have a vector of size 3X1 and we have 3 such vectors stacked up, so we say we have a matrice of size 3X3

Image for post
Image for post

That means this matrice is denoted by values and these values are arranged in 3 rows and 3 columns.

Image for post
Image for post

In general, we can write the dimension of a matrix as “m X n” where m is the no. of rows and n is the no. of columns. In the above case, both m and n equal 3.

Adding two matrices:

We can add two matrices just element-wise.

Image for post
Image for post

That means we can add up two matrices only if they have the same dimensions.

And similarly, we can subtract two matrices element-wise and in this case, also, both matrices must have the same dimensions.

Multiplying a vector by a matrix:

Let’s say we have the below matrix and we want to multiply it with a vector.

Image for post
Image for post
Image for post
Image for post

Every entry in the output is the dot product of each row of the matrix with the vector.

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

In 3D:

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

Now in the below case, we have a problem, because to compute the dot product, the two vectors must have the same number of elements.

Image for post
Image for post

So, for matrix and vector multiplication, the dimensions of matrix and vector should be compatible. In other words, the number of columns in the matrix must be the same as the number of rows in the vector and the dimension of the output vector, in this case, would be the number of rows in matrix X number of columns in the second matrix(which in this case is a vector, so second dimension is 1).

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

Multiplying a matrix by another matrix

We can think of a matrix-matrix multiplication as a series of matrix-vector multiplication.

Matrix-matrix multiplication is actually the product of every row of the first matrix with every column of the second matrix. If we multiply every row of the first matrix with the first column of the second matrix, we get the first column of the output matrix.

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

Let’s consider the below case where we try to multiply a ‘3 X 3’ matrix with a ‘2 X 3’ matrix

Image for post
Image for post
Image for post
Image for post

Let’s look at other cases:

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

In general, if we have a ‘m X n’ matrix and we are multiplying it with ‘n X k’ matrix, we get the output as a ‘m X k’ matrix.

Image for post
Image for post

An alternate way of multiplying two matrices

Image for post
Image for post

This looks like the form: y = mx1 + nx2

The output is a linear combination of the columns of the matrix and the weights in this linear combination are the elements of the vector.

Matrix-matrix multiplication can be seen as such an operation where the weights of the linear combination come from the elements of the second matrix.

Why do we care about matrices?

We can think of the training data, test data as a matrix for example:

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

And the most common operation that we would see is going to be the one given below which also involves the concept of matrix-matrix multiplication:

Image for post
Image for post

This article covers the content covered in the Vectors and Matrices module of the Deep Learning course and all the images are taken from the same module.

Written by

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store