Linear Algebra

Linear Equations

Equations generally come in the form of 2x = 4 or more generally as [ax = b]. Such a notation can also be applied to matrices where a single linear equation such as x - 2y + z = -5 may be rewritten in matrix notation as:

Assure yourself that this is identical to the following

In a more complex example we can show that matrix multiplications allow us to represent a set of linear equations in form of a matrix as follows:

The goal of most multivariate analyses, however, is the opposite scenario, where we try to extract linear equations from a variance/covariance matrix. Matrix divisions offer a solution but are somewhat less intuitive. A simple equation of real numbers such as 2x = 3 can be solved by arithmetic division, which can be phrased as a multiplication. In the process, both sides are multiplied with 2^-1 where on the left side x is multiplied with "1". One is called an identity as multiplication with it yields the original, irrespective of its value. By applying the same thinking to divisions of matrices we can multiply one matrix with its inverse matrix to produce an identity matrix. The latter is thus defined as the matrix I for which the equation holds I*X = X, no matter what value X takes and will always feature "1"s along the diagonal and "0"s elsewhere. Convince yourself that multiplication of a matrix with such an identity matrix leaves the original matrix unchanged. The inverse matrix is not equal to the transposed matrix and it presents therefore the bigger challenge.

• Matrix Division: Solving ax = b requires a divison by a. In standard algebraic notation, division by a is identical to multiplied by the inverse of a. When a is divided by a it results in an identity (i.e., multiplied with any x it will always produce x). Analogous to this, matrix division involves a multiplication by the inverse of that matrix to produce an identity matrix. The identity matrix is a square matrix with "1"s along the diagonal and "0"s off the diagonal, however, obtaining the inverse of a matrix is a more involved as described here.