Mathematical Foundations of Machine Learning: Norm vectors(L² and L¹) & Unit Vectors

In the preceding post, I introduced what vectors are, and I describe vectors as representing a point in space, now I’ll describe how we can also consider a given vector to represent a particular magnitude and direction through space.

Let’s say this is a vector representing magnitude and direction from Origin.

So in this case the vector traces in space from (0,0) origin to (12,4) coordinate.

Norms are a class of functions that allow us to quantify the magnitude, the length of a given vector.

L² Norm vectors

Of all of the Norm functions, the most common and most important is L² Norm, which is described by

l² norm function

it measures the Euclidean distance from the origin, this measure of Euclidean distance is the most common norm in Machine Learning.

Let’s get hands-on with some code to check the distance

the second iteration shows the calculation of Euclidean Distance

So we can confirm that the distance of our vector ‘x’ is 25.6m from the origin both in case of using either Euclidean distance formula or NumPy method for l² norm.

Unit vectors

Having learned about two norms, we can now talk about something called unit vectors, so this is a special case of a vector where its L² norm is equal to ‘1’.

So let’s take a look at this diagram here to kind of bring this idea to life so we have some vector ‘x’ here represented by the purple arrow running from the origin on this graph to the point that that vector denotes.

And when we calculate the L² norm, this orange distance from Origin to that point, we see that the L² norm is equal to 1, and because of this property, we can say that this vector ‘x’ is a unit vector.

L¹ Norm vectors

So with the L¹ norm, what we do is we take the absolute value of every element in the vector ‘x’ and we sum those absolute values up. This is another common norm in machine learning and one of the interesting properties of the norm is that it varies linearly at all locations, whether we’re near or far from the origin. We use the L¹ norm in situations where differences between zero and non-zero are key.

L¹ norm

so to calculate the L¹ norm, we can take the absolute value of each of the elements as I do here, and then sum them up.

Because these are all positive, the absolute values are the same as the integers themselves and L¹ norm for our vector ‘x’ is 32, so it can be seen that the same vector depends on what norm we use to measure it gives a different length.