A Short Note on Resolving Singularity Problems in Covariance Matrices

In problems where a distribution is concentrated in a lower-dimensional subspace, the covariance matrix faces a singularity problem. In downstream statistical analyzes this can cause a problem as the inverse of the covariance matrix is often required in the likelihood. There are several methods to overcome this challenge. The most well known ones are the eigenvalue, singular value, and Cholesky decompositions. In this short note, we develop a new method to deal with the singularity problem while preserving the covariance structure of the original matrix. We compare our alternative with other methods. In a simulation study, we generate various covariance matrices that have different dimensions and dependency structures, and compare the CPU times of each approach.