The way I look at it is: matrices are things that act on vectors. For example, one matrix might scale up all vectors by a factor of 2, while another might rotate vectors by a certain angle around a certain axis.

You've probably heard how 1 x n matrices are sometimes called "column vectors." If A is some random matrix and e_1 is the column vector (1,0,0...,0) (which looks more like a row vector, but that's only because I can't draw a column vector easily on Reddit, so please pretend it's a column), then what is the product Ae_1? If you look closely at the definition of matrix multiplication, you will discover that Ae_1 is simply the first column of A. And if e_2 is the column vector (0,1,0,...,0), then Ae_2 is the second column of A, and so on. So the matrix A can be interpreted as a list of column vectors, specifically the column vectors that A maps e_1, e_2, ... e_n to. Since any vector can be written as a linear combination of e_1, e_2, ...e_n, knowing where A sends those vectors tells you where A sends any vector. Explicitly, if v=(v_1,v_2,... v_n), then Av=v_1Ae_1+v_2Ae_2+...+v_nAe_n. So matrices are just compact representations of linear functions that take in a vector and output another vector. The height of a column is the length of the output vector, and the number of columns is the length of the input vector. Thus, a (b x a) matrix represents a function that takes as input a vector of length a and outputs a vector of length b.

Now, what happens if we have two matrices A and B and a vector v, and we want to figure out what happens when we start with V, apply A, and then apply B. Well, we can first read off the first column of A, a.k.a. Ae_1, and then compute what B does to that vector. That should be the first column of BA, since BAv should be the matrix that describes our process of applying A and then applying B. (In other words, we want (BA)v=B(Av), making multiplication associative). Work it out carefully and you will see that the matrix multiplication you've been taught is exactly the one that does this job. In particular, the height of A and the width of B have to be the same, because the output of the first function we're applying has to be a vector of the same length as the input of the second function.

This is all very abstract stuff, and I certainly haven't told you why we should care about matrices and vectors in the first place. So let me leave you with a concrete example. You probably know that 2-dimensional vectors of real numbers, e.g. (1,2.5), can be drawn as arrows in the 2D plane starting at the origin. One thing we can do with the 2D plane is rotate it around the origin clockwise by an angle t. From your knowledge of the unit circle, you may know that, if we start at the point (1,0) on the positive x-axis and rotate by t, we get to the point (cos(t) ,sin(t)). And, with a little more trig identity work and the knowledge that (0,1) is what you get by rotating (1,0) by pi/2, you can show that rotating by t sends (0,1) to (-sin(t),cos(t)). So, the matrix that represents "rotate clockwise by t radians" is just R_t:=[[cos(t),-sin(t)],[sin(t),cos(t)]].

Now, what happens if I rotate clockwise by t radians, and then rotate clockwise by another s radians? Well, I've rotated by (s+t) radians in all, right? Based on this, we should expect that R_sR_t=R_(s+t). And by using matrix multiplication and angle addition trig identities, one can check that this is the case.

each column corresponds to an input, each row to an output. So the product of two matrices requires "wiring" the outputs of one to the inputs of another, which is why you multiply rows and columns like that. This also explains the dimension thing, if one matrix has n inputs and m outputs, and another has m inputs and k outputs, the m outputs can be "wired" to the m inputs so now you have a matrix with n inputs and k outputs.

the "multiply each combination of components" thing you thought of is actually a thing called the tensor product, which is a generalization of objects like matrices and vectors to more complex types of function.

note: you can also say rows are inputs and columns are outputs, depends on if the smaller objects you're working with are column or row vectors.

This is probably not the right answer for your particular needs, but from the perspective of pure mathematics, I think the way in which matrices are introduced is often really unmotivated, especially in linear algebra courses designed mainly for students from other disciplines (also it is not really true that their main applications are in programming - matrices, and more generally linear transformations, which they represent, are ubiquitous across all of mathematics and any discipline using it). 

The reason that matrix multiplication is the way it is is so that multiplication of matrices corresponds to composition of linear transformations (under a choice of bases). If you know what a linear transformation is, and what a basis is, then you can sit down and write out what sort of operation you need in terms of vector components with respect to given bases to completely describe the algebra of linear transformations, and you will more or less automatically rederive the rule for matrix multiplication. This perspective is laid out especially nicely in Sheldon Axler's Linear Algebra Done Right.

I recommend going to YouTube, looking up the channel 3blue1brown, and finding in his playlists tab “The Essence of Linear Algebra”. I feel like I learned more about matrices in that video series than I learned in a university-level course.

A lot of great explanations, but I want to give some more insight into use cases.

Linear algebra is used to work with 3 (or more) dimensional space. We have found a lot of ways to model things this way, for example each column could be a datapoint and suddenly you're halfway to machine learning. In calculus of multiple variables linear algebra is used as the ground you stand on, putting variables into matrices. This leads into a multitude of applications in physics (like all of physics). Linear algebra also pops up in proofs in unrelated areas every now and again such as graph theory or combinatorics.

A lot of our computers are optimized for fast work on matrices, such as GPU's, it has pretty much become our standard way of dealing with math on large amounts of data, which is something some programmers will need to do so if you are interested in simulations, computer graphics, machine learning, high performance computing, etc then linear algebra is the most important math course you'll take.

The short answer is that matrices represent linear maps, and products of matrices represent composition of linear maps. To understand this, a good place to start is this post on MSE.

Matrix are representation of linear transformations between (possibly different) vector spaces.

You can concatenate linear transformations (if the vector spaces matches). The matrix multiplication is defined in a way that

TA(TB) = AB

Where TA is the linear transformation A and A is the matrix representing that transformation.

Have you encountered the youtube channel 3blue1brown? They have a series on linear algebra that might help give you a deeper understanding

The matrix multiplication is done that way such that v for every matrix A, B and vector v with appropriate dimensions (AB)v=A(Bv), which you really want to have since matrices represent linear functions.

The video probably does a better job of it, but in case you aren't a fan of video learning like me, here's a basic concept.

Linear algebra is mostly concerned with linear equations. They take the form Ax+By+...+Dz = K where my upper case letters are coefficients and the lowercase letters are variables. These are linear because every term has exactly one variable to the power of 1 (otherwise this is a non-linear equation).

The original invention of matrices was to solve systems of these equations, that is cases where you have two or more equations in two or more variables. You will likely see this early on in a Linear Algebra class. The language of Linear Algebra allows us to represent the solution to this system as a vector X, the coefficients as a matrix A, and the constants as a vector Y. We can then ask "if we transform X by multiplying it by A, what value of X gives us Y". In essence, the coefficients put into the matrix A represent a way of transforming one vector (x, y, ..., z) into another (K1, K2, ..., Kn). Matrix multiplication is essentially how that process happens. 

In this sense, we can talk about a matrix as a linear map, a way of mapping one vector to another vector. 

If it helps, think of a linear function (y=mx+b) as mapping the scalar value x to the scalar value y. The linear map (y=Ax) maps the vector (or matrix) x to another vector or matrix y. Matrix multiplication is designed so that this works.

Matrices can also represent other things and be used for other purposes, but this is the essence of linear algebra. Pretty much everything you learn in Linear Algebra 1 supports these concepts and figuring out ways to quickly solve linear equations using matrices.

Matrix multiplication is weird because we want it to be compatible with function composition! Multiplying two matrices AB will give you a new matrix where (AB)v first applies B and then applies A.

Matrices encode what are known as linear transformations. A linear transformation T is a map with the properties that

• For all vectors v and w, T(v+w) = T(v) + T(w)
• For a vector v and a scalar c, T(cw) = cT(w)

These properties (very intentionally) align with the properties of matrix-vector multiplication. This means that you can take any matrix A and create an associated linear transformation T(v) = Av. Similarly, given any^(1) linear transformation, you can find a unique^(2) matrix associated to it.

This now leads to our motivation for what we want matrix multiplication to do. Given two function f(x) and g(x), we are able to compute their composition g∘f where (g∘f)(x) is defined to be g(f(x)). This comes up very often, so it would be nice if we could essentially precompute g∘f rather than having to apply both functions. This is exactly what matrix-matrix multiplication does. If you have two linear transformations with compatible dimensions T(v) = Av and S(v) = Bv, their composition (T∘S)(v) is equal to the function (AB)v. I believe the main way of showing this is to use (or prove) that A(Bv) = (AB)v. But as it turns out, this weird formula for matrix multiplication is exactly the one that gives us this compatibility between function composition and multiplication.

[1] Matrices can be found for vector spaces with finite dimension.
[2] These matrices are only unique up to your basis for the vector space.

Its a good thing that you ask these questions. Look into Linear Algebra.

Matrix * vector multiplication is defined the way that it is in order for matrices to "naturally" be "linear transformations"

The motivation for the definition of matrix * matrix multiplication follows pretty quickly from matrix * vector multiplication. Matrix * matrix multiplication represents composing linear transformations.

This probably doesnt make sense, and you will need to look basic linear algebra things up, but this is the most succint way to explain it

Use the index notation. We write:

A_{r,s}

for the matrix element at the rth row and the sth column. So, starting from the upper left corner, you move down by r steps and move to the right by s steps.

A vector of dimension d, is a d by 1 matrix. In index notation we write V_k for the kth element of a vector.

If the vector W results from applying a matrix A to a vector V, then in index notation, you have:

W_r = Sum over all s of A_{r,s} V_s

Matrix product: If matrix C is the matrix product of matrices A and B, so C = A B, then in index notation, we have:

C_{r,s} = Sum over all k of A_{r,k}B_{k,s}

And you then easily see that if we multiply matrices A1, A2, A3,...,An that the product C in index notation will be given by:

C_{r,s} = Sum over all k_1, k_2, k_3,,...,k_{n-1} etc of A1_{r, k_1 }A2_{k_1 ,k_2} A3_{k_2, k_3},...,An_{k_{n-1}.s}

Let me give an example from my (more or less) everyday work. Say you want to simulate a robot arm. It has different parts that can move and rotate. Luckily for us rotations and translation can be represented by matrices. Now I want to know if the piece that is at the end of the robot arm hits a wall (which would be really bad). So I need to know where all the points describing the thing at the end of arm end up. Now I could calculate A(B(C(Dv))) (capital letters for the transformation matrices and v a representation of a point of my object), but since the object has a lot of points this is a bit tedious and I don't really care about how the object came to the position, so it would be nice if I could just have one combined transformation. Luckily this is easy the combined transformation is the matrix product M=ABCD, I only have to calculate this once and now can just calculate Mv instead. To have this property is how the multiplication is done.

Matrices are used to describe linear transformations. I'll give a geometric descrption, but for it to stick, you'll need to draw along. (This is one of those things that would be much easier if we were in front of a nice blackboard.)

To explain what a linear transformation is: imagine the xy-plane, with a grid of 1x1 squares drawn on it, parallel to the axes. We can draw a little arrow A from (0,0) to (1,0), and represent this arrow by the column vector [1 0]^T. (The T means "transpose", which just means to turn the row vector I wrote into a column vector. I did it this way because you can't write column vectors easily on Reddit.) We can draw another little arrow B from (0,0) to (0,1), and represent this arrow by the column vector [0 1]^T. Geometrically, a linear transformation means we transform (e.g. stretch, shrink, rotate, reflect, etc.) the plane in a way that keeps (0,0) the same, sends the little arrows A and B to two other little arrows, and then transforms the rest of the plane based on that, so that the grid squares we started with become a new grid of identical parallelograms with sides that all have the same shape as the arrows we sent A and B to. It's like we're turning the usual grid of parallelograms (squares are parallelograms too) to a new grid of parallelograms, and leveraging the fact that just knowing where A and B go tells us how to change all the other lines in the grid as well.

For example: we could rotate the plane by 90 degrees clockwise around (0,0). That would send A to [0 -1]^T and B to [1 0]^T, and all the squares would stay squares.

Or for another example, we could keep the x-axis the same, but tilt and stretch the y-axis to turn the squares into little diamonds, by sending A to [1 0]^T (keeping it the same) and sending B [1 1]^T.

We can represent each transformation as a 2x2 matrix M, where the first row is whatever the arrow A gets sent to, and the second row is whatever the arrow B gets sent to. In other words, a matrix is nothing more than a list of what happens to the vectors [1 0]^T and [0 1]^T. (For a bigger matrix like a 3x3 matrix, it would be the vectors [1 0 0]^T, [0 1 0]^T, [0 0 1]^T, etc.) Then, matrix multiplication is defined so that, for example MA="the arrow A gets sent to".

What happens when we want to do two transformations in a row? If we first do the tilt-and-stretch I described, and then do the rotation, where does A end up, and where does B end up? Is it the same as if we do them in the other order? You can work this out by drawing. What you will see is that, if R is the matrix for our rotation and S is the matrix for our tilt-and-stretch, then the product RS is the matrix for "first tilt-and-stretch, then rotate." The geometric definition I gave of linear transformation above is a bit sketchy. But if you look up an actual defniition, and work very carefully, you can verify that, if we view matrices as lists of where each of our vectors [1 0 0...]^T, [0 1 0...]^T, etc. goes, the seemingly weird matrix multiplication is the one correct way to keep track of what happens when we do multiple linear transformations after one another.

As for why it's useful, there are two main things to keep in mind. One is that there are a lot of things in life which behave like linear transformation, even if they're not transforming a literal space. Search the internet and you will be surprised how many things are linear. The other is that linear transformations are relatively easy to compute and linear equations are relatively easy to solve, so that turning more complicated situations into linear ones (even if the linear thing you create is only an approximation of the original situation) is a practical problem-solving strategy. For example, this is at the heart of a lot of applications of calculus (various forms of derivative are essentially telling you what the best linear approximation is), as well as a cornerstone of many things called AI, including LLMs like ChatGPT.

W Keep it up fam

You know, we could really use some originality instead of all this AI generated nonsense you've been posting. Maybe you do have a clear undertanding of linear algebra, but your posts certainly aren't doing a very good job proving that - it doesn't take a lot of skill or knowledge to copy and paste from ChatGPT.

I would suggest focusing on developing your own unique writing style to express your own unique ideas, rather than having AI ghostwrite for you.

As a masters student in computer science , matrices are everything, keep it up buddy.

Nice nice 🔥🔥 matrix is one of the most used topic in ai ml

Bro, from where are you learning these, please tell me to. Cuz I am also doing but fine good resource

Learning concepts like this is good, but don’t forget to do exercises. Lots of them! It’s one thing to understand what’s being explained, and it’s another to learn to think with it

Where you learning from?

Any n,m matrix yields linear maps R^(m,k) -> R^(n,k) via left multiplication. The k = 1 vector case is just one particular instance

Note also that if you have many vectors and you're interested in Av_1, Av_2 etc. you are also interested in the matrix product AV with V is the matrix with columns v_1, v_2 etc. (and vice versa)

> One thing I've read is that we can see an m x n matrix A as a linear map function f: K^n -> K^m. But doesn't this imply that the arguments and outputs are both vectors?

Yes. And matrix multiplication is defined to be consistent with function composition.

Matrix multiplication has the same effect on every column of the right-side matrix and it distributes over matrix addition, so you can represent matrix multiplication as a bunch of matrix/vector multiplications just fine. For instance, for any 3x3 matrix M, multiplying an arbitrary matrix:

  | a b c |     | a 0 0 |   | 0 b 0 |   | 0 0 c | 
M | e f g | = M(| e 0 0 | + | 0 f 0 | + | 0 0 g |) =
  | h i j |     | h 0 0 |   | 0 i 0 |   | 0 0 j | 

    | a 0 0 |     | 0 b 0 |     | 0 0 c |
= M | e 0 0 | + M | 0 f 0 | + M | 0 0 g |
    | h 0 0 |     | 0 i 0 |     | 0 0 j |

And the result is what you would expect with the zeroes remaining zeroes, but the columns transforming like standard vectors.

A lot of applied linear algebra has the goal of multiplying a matrix by a vector, but it might be more efficient to get there by multiplying a matrix by a matrix instead.

E.g., you want to calculate A(Bv). It might be more convenient to calculate (A x B)v if you need to perform the calculation many times for the same A and B and different v's.

I guess that's one way to explain linear algebra in a nutshell. That being said, the bulk work in linear algebra (as far as matrix and vector are concerned) is in analyzing the mapping, i.e., the matrix. For example:

• composing a sequence of mappings (matrix multiplications)
• decomposing a mapping into a sequence of mappings (Cholesky decomposition, SVD, etc.)
• existence and computation of inverse of mappings (matrix inverse)
• approximating high rank mappings with lower rank mappings (eigen-analysis, SVD, etc.)
• and many more...

<A v, w> = (A v)^T w = v^T A^T w = v^T (A^T w) = <v, A^T w>

Nice. What was your prompt?

Damn pretty interesting, I might have my hands on that

Why are there block sub matrix?

(source) Data I collected during my master's thesis work in Mechanical Engineering, data collected using an SLDV (scanning laser Doppler vibrometer) device.

(tool) Data compiled, transformed, and visualized using Matlab.

What do the axes correspond to? In other words, if they had labels what would they say?

Now this is some proper content for this sub!

Very didactic article. I enjoyed reading it.

It would be interesting to measure performance with other open source linear algebra libraries .

As for example :

• Eigen : C++ template library for linear algebra: matrices, vectors, numerical solvers, and related algorithms.
• Boost uBLAS : uBLAS provides templated C++ classes for dense, unit and sparse vectors, dense, identity, triangular, banded, symmetric, hermitian and sparse matrices.
• Blaze: high-performance C++ math library for dense and sparse arithmetic.
• .....

I always appreciate discussions about numerical methods with c++. Thanks!

> Real-life implementations of matrix multiplication are using Strassen’s algorithm

No they don’t. Strassen is only of theoretical interest. This is a silly statement.

If you want to do something performance with matrix matrix multiply, implement the recursive algorithm, which is an example of a cache oblivious algorithm. You need gsl::span for that. I had a good time coding it.