We have learned different methods for matrix factorization, though the various methods we have seen seem to have rather strict requirements. Most notably, Cholesky Factorization requires a matrix to be both symmetric and positive definite. Many matrices we have seen were not symmetric, and most of us can barely recall what an eigenvalue is, let alone know what it means if they are all positive, so it’s difficult to grasp why these are reasonable criteria for a matrix to have in application.

A Covariance matrix is a matrix whose entries are the covariances between elements of a set of random variables. A covariance between two random variables is simply the expected relation of the two variables to one another. A large positive covariance between two variables means that if one is above its mean, there is a greater chance that the other is as well. Given a set of n random variables, the corresponding covariance matrix is an nxn matrix, with each variable corresponding to one row and one column each. Entry i,j of the matrix is the covariance of variables i and j. Covariance is commutative, thus it is straightforward to see that if the rows and columns are ordered in the same manner, the matrix of covariances will be symmetric. A covariance matrix is defined specifically in this manner, thus all covariance matrices are symmetric. It also happens such that all covariance matrices are positive definite, and all symmetric, positive semi-definite matrices are covariance matrices.

So now we have classified matrices on which we can perform cholesky factorization, but why should we have any reason to believe they should arise for real applications? Given any set of random variables, there exists a corresponding covariance matrix, and nearly anything one might want to bother modeling can be viewed as a random variable. To give one boring but concrete example, the value of a stock at some point in the future cannot be predicted exactly, but information as to how it might perform can be used to estimate the distribution which it follows. Now imagine you were considering investing in some set of stocks, of which you have calculated estimates of their means and covariances. You might want to maximize your expected value while restricting your risk (the total variance) of your portfolio to some amount k. This would correspond to the optimization problem of defining the vector x of amounts invested in each stock. The constraint corresponding to bounding your total risk would be x^T Vx ≤ k. Thus, we can see how it is useful to be able to manipulate matrices of with the properties of symmetry and positive eigenvalues.