Orthogonal projections of multidimensional ellipsoids – V – relation between the inverse matrices of the involved quadratic forms and of respective covariance matrices

This series started with four main questions. The first two were: How do we know that an orthogonal projection of a (n-1)-dimensional ellipsoid from its n-dimensional vector space onto a lower p-dimensional sub-space leads to yet another ellipsoid? And how can we derive the matrix defining the quadratic form of the resulting lower dimensional ellipsoid from the matrix describing the original ellipsoid?
The answers required relatively complicated considerations. The matrix of the quadratic form which defines the hull of the ellipsoid’s projection image was found to be a Schur complement of the original matrix. In this post we will derive an answer to the third question: What about the inverse matrices of the two involved quadratic form matrices? What is their relation?

We shall see that the relation between these inverse matrices is surprisingly simple: You can read the elements of the (pxp) quadratic form matrix for the lower dimensional ellipsoid directly off the (nxn)-matrix for the original ellipsoid by a simple selection process: Pick all elements whose indices refer to those base vectors which span the projection’s target space.

Previous posts

Relevant publications and some criticism

Originally, I had planned to refer to a physics publication [1] which touched the topic in its chapter III. However, with all due respect, the proof offered there for the inverse counterparts of the quadratic form matrices, which control the orthogonal projections of multidimensional ellipsoids, appears to be somewhat questionable. Among other things, it disregards the fact that we have to consider special points on the (n-1)-dim ellipsoid’s surface to address the hull of the projection image properly. It also treats respective underlying vectors of the unit sphere improperly. By “improperly” I mean that the asserted assumptions lead to a contradiction. I will shortly address this point below. Note that the claims, conclusions and applications discussed in the named paper [1] fortunately remain unaffected by this problem.

The fact that a Schur complement appeared in our analysis (see post IV) indicates already that a proper proof of the relation between the different inverse matrices has to take properties of the matrix complements into account. Such a proof will therefore become a bit more complicated than what the authors of [1] discussed.

As a preparation the interested reader is recommended to read [2], a paper written by Chris Yeh. I will apply his insights on Schur complements and a respective matrix factorization to the quadratic form matrices that control the projections of multidimensional ellipsoids. All credits, therefore, belong to Mr. Yeh! His article put me onto the right track. Another valuable paper on the same topic was published by J. Gallier; see [3]. I happened to find it a bit later. Links and paper titles are listed in the last section of this post.

Previous results of this post series

I use the same notation and abbreviations as in previous posts of this series. Readers already familiar with its earlier posts may hop over this section.

We defined an original (n-1)-dim ellipsoid in the ℝⁿ with the help of a (nxn) symmetric matrix Q (= Σ^-1):

\[ \text{vectors } \pmb{x}_e \text{ of (} n\text{-1)-dim ellipsoid } E\,: \quad \pmb{x}_{e} \in E = \left\{ (\pmb{x}_e)^T \,\, | \,\, \pmb{x}_e \, \pmb{\operatorname{Q}} \, \pmb{x}_{e} \,=\, 1 \, \right\} \subset \mathbb{R}^n \,. \tag{1} \]

This matrix imposes a quadratic form condition onto the ellipsoid’s vectors. I also call the matrix Σ^-1 to remember the reader that the ellipsoid could be regarded as a contour surface of a n-dim normal distribution [MVN] for some (nxn)-variance-covariance matrix Σ.

Both the matrices Q (=Σ^-1) and Q^-1 (=Σ) are assumed to be symmetric, invertible and positive-definite. Q^-1(=Σ), therefore, has a Cholesky decomposition

\[ \pmb{\operatorname{\Sigma}} \,\equiv \, \pmb{\operatorname{Q}}^{-1} \,=\, \pmb{\operatorname{A}} \, \pmb{\operatorname{A}}^T \,. \tag{2} \]

Matrix A generates the ellipsoid when applied to vectors u of the unit sphere 𝕊^n-1 (u ∈ 𝕊^n-1) :

\[ ||\pmb{u}|| = \sqrt{ u_1^2 + u_2^1 + … + u_n^2 \, } \,=\, 1 \,, \,\, \, \pmb{u} \in \mathbb{S}^{n-1} \, , \tag{3} \]

\[ \pmb{u} \in \mathbb{S}^{n-1} \,\Rightarrow \, \pmb{\operatorname{A}} \, \pmb{u} \in E \,. \tag{4} \]

Our orthogonal projection was defined by a well defined p-dimensional target space S_P and its orthogonal complement (S_P)^⊥, both being orthogonal sub-spaces of the ℝⁿ:

\[ \mathbb{R}^n \,=\, S_P \,\oplus (S_P)^{\perp} \,. \tag{5} \]

We chose a base of unit vectors e_i such that S_P was covered by respective unit vectors e_j with (1 ≤ j ≤ p). We introduced reduced vectors of the complementary orthogonal sub-spaces, S_P with dimensionality p, the complement (S_P)^⊥ with dimensionality (n-p), such that

\[ \begin{align} \pmb{x}_e \,&=\, \begin{pmatrix} \pmb{y}_{e,p} \\ \pmb{z}_{e,n-p} \end{pmatrix} \,. \tag{6} \\[10pt] \pmb{y}_{e,p} \,&= \, (y_1, ……… , y_p)^T \in S_P \sim \mathbb{R}^p\,, \\[10pt] \pmb{z}_{e,n-p} \,&=\, (z_{p+1}, …, z_n)^T \in (S_P)^{\perp} \sim \mathbb{R}^{(n-p)} \,, \\[10pt] \pmb{x}_e \, &=\, \pmb{y}_e \,+\, \pmb{z}_e \,=\, ( (\pmb{y}_{e,p})^T, 0,..,0)^T + (0,..0, (\pmb{z}_{e,n-p})^T )^T \,. \end{align} \]

Note: We can always choose our coordinate system such that our projection’s target space is spanned by the first p vectors of the related base of unit vectors. We saw that the vectors describing the hull of the projection image could be derived by applying the projection to special vectors x_e^t

\[ \pmb{x}_e^t \,=\, \pmb{y}_e^t + \pmb{z}_e^t \,=\, \begin{pmatrix} \pmb{y}_{e,p}^t \\ \pmb{z}_{e,n-p}^t \end{pmatrix} \,. \]

of the original ellipsoid. At these specific points the surface’s normal vectors Qx_e^t have to be perpendicular to (S_P)^⊥. This in particular means:

\[ \left(\pmb{z}_e^t\right)^T \pmb{\operatorname{Q}} \, \pmb{x}_{e}^t \, =\, 0 \,. \tag{7} \]

This is a generalization from the 2-dim ellipsoids in the ℝ³ case, where the vector giving the line of projection onto an orthogonal target plane must become a tangential vector at x_e^t. You could also say that Qx_e^t must become an element of S_P. The x_e^t in turn are generated by A from special vectors u_e^t of the unit sphere 𝕊^n-1:

\[ \begin{align} & \pmb{u}_e^t \,=\, \pmb{\operatorname{A}}^{-1} \pmb{x}_e^t \,\in\, \mathbb{S}^{n-1} \, \tag{8} \\[10pt] & \pmb{u}_e^t \,\perp\, \pmb{\operatorname{A}}^{-1} \pmb{e}_j \,, \quad \forall \,j \,\,\, \text{with} \,\,\, p+1 \,\le \, j \, \le\, n \,. \tag{9} \end{align} \]

Let us turn to the vectors y_e,r^t ∈ S_P resulting from the projection of x_e^t . In the last post, we partitioned the (n x n) matrix Q = Σ^-1 into respective blocks with respect to the orthogonal sub-spaces (S_P ⊕ (S_P)^⊥):

\[ \pmb{\operatorname{\Sigma}}^{-1} \,\equiv \pmb{\operatorname{Q}} \,=\, \begin{pmatrix} \pmb{\operatorname{Q}}_{yy} & \pmb{\operatorname{Q}}_{yz} \\ (\pmb{\operatorname{Q}}_{yz})^T & \pmb{\operatorname{Q}}_{zz} \end{pmatrix} \,. \tag{10} \]

\[ \pmb{\operatorname{Q}}_{yy} \,\in \, \mathbb{R}^{p\operatorname{x}p} \,, \,\, \pmb{\operatorname{Q}}_{yz} \in \mathbb{R}^{p\operatorname{x}(n-p)} \,, \,\, \pmb{\operatorname{Q}}_{zz} \in \mathbb{R}^{(n-p)\operatorname{x}(n-p)} \,. \tag{11} \]

The matrix for the (p-1)-dimensional ellipsoid, i.e. the hull of the projection image in S_P, was given by a reduced matrix Q_p :

\[ \pmb{\operatorname{Q}}_{p} \,:=\, \pmb{\operatorname{Q}}_{yy} \,-\, \pmb{\operatorname{Q}}_{yz} \pmb{\operatorname{Q}}_{zz}^{-1} \pmb{\operatorname{Q}}_{yz}^T \,\, \in \,\, \mathbb{R}^{p\,x\, p} \,. \tag{12} \]

\[ (\pmb{y}_{e,p})^T \, \pmb{\operatorname{Q}}_{p} \, \pmb{y}_{e,p} \,=\, 1 \,, \quad \pmb{y}_p \in S_P \, . \tag{13} \]

This matrix Q_p is just the Schur complement “[Q/Q_zz]” of Q (= Σ^-1):

\[ \pmb{\operatorname{Q}}_{p} \,=\, \pmb{A}_p\,[\pmb{A}_p]^{-1} \,=\, [\pmb{\operatorname{Q}}/\pmb{\operatorname{Q}}_{zz}] \,. \tag{14} \]

So much for the general understanding.

Pitfalls regarding asymmetric projection vectors and assumptions on unit vectors

The starting points are the equations for the special vectors x^t_e,n ∈ ℝⁿ of the original ellipsoid (see post IV) and the reduced vectors y_e,p(= y_e,r ) ∈ S_P (∼ ℝ^p) for the lower dimensional ellipsoid:

\[ (\pmb{x}_{e,n}^t)^T \, \pmb{\operatorname{Q}} \, \pmb{x}_{e,n}^t \,=\, 1 \,, \tag{15} \]

\[ \pmb{y}_{e,p}^t \, \pmb{\operatorname{Q}}_p \, \pmb{y}_{e,p} \,=\, 1 \,. \tag{16} \]

The (added) second index just indicates the dimensionality. This will be useful to distinguish vectors of different vector spaces below.

Due to the results of posts I to IV we know for sure that eq. (16) exists for a positive definite, invertible and symmetric matrix Q_p. In contrast to my approach in post IV, the authors of [1] use a modified projection operator Pⁿ_p to indicate the transition between the involved Euclidean vector spaces:

\[ \pmb{\operatorname{P}}^n_p \,=\, \begin{pmatrix} \pmb{\mathbb{I}}_{p\operatorname{x}p} & \pmb{\mathbb{O}}_{p\operatorname{x}(n-p)} \end{pmatrix} \,\in \, \mathbb{R}^{p\operatorname{x}n}\,. \tag{17} \ \]

\[ \pmb{\mathbb{I}}_{p\operatorname{x}p} \,\in \, \mathbb{R}^{p\operatorname{x}p} \,, \,\, \pmb{\mathbb{O}}_{p\operatorname{x}(n-p)} \in \mathbb{R}^{p\operatorname{x}(n-p)} \,. \]

A matrix 𝕀_p is the (pxp)-identiy matrix. The matrix 𝕆_px(n-p) are (p x n-1) zero matrices. Pⁿ_p fulfills:

\[ \pmb{\operatorname{P}}_p^n \, [\pmb{\operatorname{P}}_p^n]^T \,=\, \pmb{\mathbb{I}}_{p\operatorname{x}p} \,. \tag{18} \]

But:

\[ [\pmb{\operatorname{P}}_p^n]^T \, \pmb{\operatorname{P}}_p^n \,\ne \, \pmb{\mathbb{I}}_{n\operatorname{x}n} \,. \tag{19} \]

In paper [1] the authors focus on the case p=2, but generalize their conclusions afterwards. P, however, does not fufill PP = P. Therefore, one has to be very careful with it as it connects vectors in spaces with different dimensions. The application of Pⁿ_p to a vector x_n ∈ ℝⁿ creates a vector y_p ∈ ℝ^p :

\[ \pmb{y}_p\,=\, \pmb{\operatorname{P}}^n_p \, \pmb{x}_n \,\in\, \mathbb{R}^p\,. \tag{20} \]

In particular

\[ \begin{align} & \pmb{y}_{e,p}^t \,=\, \pmb{\operatorname{P}}^n_p \, \pmb{x}^t_{e,n} \,=\, \pmb{\operatorname{P}}^n_p \, \pmb{\operatorname{A}}_n \, \pmb{u}_n^t \,, \tag{21} \\[10pt] & \left[\pmb{\operatorname{P}}^n_p \, \pmb{\operatorname{A}}_n\right] \,\in\, \mathbb{R}^{p\operatorname{x}n} \,. \end{align} \]

The authors of [1] claim in chapter III (with their expressions translated into our notation)

\[ \begin{align} & \pmb{y}_{e,p}^t \,=\, \pmb{\operatorname{A}}_p \, \pmb{u}_p^t \,, \\[10pt] & \text{and (wrongly!) : } \,\, \pmb{u}_p^t \,=\, \pmb{\operatorname{P}}^n_p \, \pmb{u}_n^t . \tag{A} \end{align} \]

Another (correct) assertion of the authors is

\[ \begin{align} \pmb{\operatorname{Q}}_p^{-1} \, = \,\pmb{A}_p \,\pmb{A}_p^T \,& =\, \pmb{\operatorname{P}}_p^n \, \pmb{\operatorname{Q}}^{-1} \, \left[\pmb{\operatorname{P}}^n_p \right]^T \\[10pt] &= \pmb{\operatorname{P}}_p^n \,\pmb{A}_n \,\pmb{A}_n^T \, \left[\pmb{\operatorname{P}}^n_p \right]^T \,. \end{align} \tag{B} \]

Although the projection Pⁿ_p u_n^t ∈ S_P, it is NOT equal to u_p^t. Actually and instead,

\[ \pmb{u}_p^t \,=\, \pmb{\operatorname{A}}_p^{-1} \, \pmb{\operatorname{P}}^n_p \, \pmb{\operatorname{A}}_n \, \pmb{u}_n^t \,. \tag{22} \]

If assertion (A) were right, it would lead to a contradiction with the (correct) assertion (B), because due to (22) it would mean

\[ \begin{align} & \text{wrong : } \\[10pt] & \quad \pmb{\operatorname{A}}_p^{-1} \, \pmb{\operatorname{P}}^n_p \, \pmb{\operatorname{A}}_n \,\equiv\, \pmb{\operatorname{P}}^n_p \quad \Rightarrow \\[10pt] & \pmb{\operatorname{A}}_p^{-1} \, \pmb{\operatorname{P}}^n_p \, \pmb{\operatorname{A}}_n \, [\pmb{\operatorname{P}}^n_p]^T \, = \, \mathbb{I}_{p\operatorname{x}p} \\[10pt] & \mathbb{I}_{p\operatorname{x}p} \, \pmb{\operatorname{P}}^n_p \, \pmb{\operatorname{A}}_n \, \left[\pmb{\operatorname{P}}^n_p\right]^T \,=\, \pmb{\operatorname{A}}_p \quad \Rightarrow \\[10pt] & \pmb{\operatorname{P}}^n_p \, \pmb{\operatorname{A}}_n \, \left[\pmb{\operatorname{P}}^n_p\right]^T \,=\, \pmb{\operatorname{A}}_p \quad \Rightarrow \\[10pt] & [\pmb{\operatorname{Q}}_p]^{-1} \,=\, \pmb{\operatorname{A}}_p \, [\pmb{\operatorname{A}}_p]^T \,=\, \pmb{\operatorname{P}}_p^n \, \, \left(\, \pmb{\operatorname{A}}_n \, \left[\pmb{\operatorname{P}}^n_p \right]^T \, \pmb{\operatorname{P}}_p^n \, [\pmb{\operatorname{A}}_n]^T \, \right) \, \left[\pmb{\operatorname{P}}^n_p\right]^T \tag{C} \end{align} \]

Let me in addition warn you of another mistake one is tempted to make: Combining (15) and (16) we might be tempted to conclude something regarding Q

\[ \pmb{y}_{e,p}^T \, \pmb{\operatorname{Q}}_{P,red} \, \pmb{y}_{e,p} \,=\, 1 \,=\, (\pmb{x}_{e,n}^t)^T \, \pmb{\operatorname{Q}} \, \pmb{x}_{e,n}^t \quad \Rightarrow \]

\[ (\pmb{x}_{e,n}^t)^T \, \left[ \pmb{\operatorname{P}}^n_p \right]^T \, \pmb{\operatorname{Q}}_{P,red} \, \pmb{\operatorname{P}}^n_p \, \pmb{x}_{e,n}^t \,=\, (\pmb{x}_{e,n}^t)^T \, \pmb{\operatorname{Q}} \, \pmb{x}_{e,n}^t \]

\[ \require{\cancel} {\large {WRONG: }} \,\, \cancel{ \pmb{\operatorname{Q}} \,=\, \left[ \pmb{\operatorname{P}}^n_p \right]^T \, \pmb{\operatorname{Q}}_{P,red} \, \pmb{\operatorname{P}}^n_p } \,. \]

Which would be wrong! You cannot conclude anything on operators in the higher dimensional space when special vectors (!) are selected. The condition does not hold for arbitrary vectors in the original space!

Now let us turn to a better way of proving assertion (B).

Matrix decomposition based on Schur complements

Let us consider an existing and invertible Schur-Complement “[M/D]” of a general invertible matrix M:

\[ \pmb{\operatorname{M}} \,=\, \begin{pmatrix} \pmb{\operatorname{A}} & \pmb{\operatorname{B}} \\ \pmb{\operatorname{C}} & \pmb{\operatorname{D}} \end{pmatrix} \,. \tag{23} \]

\[ \pmb{\operatorname{A}} \,\in \, \mathbb{R}^{p\operatorname{x}p} \,, \,\, \pmb{\operatorname{B}} \in \mathbb{R}^{p\operatorname{x}(n-p)} \,, \,\, \pmb{\operatorname{C}} \in \mathbb{R}^{(n-p)\operatorname{x}p} \,, \,\, \pmb{\operatorname{D}} \in \mathbb{R}^{(n-p)\operatorname{x}(n-p)} \,, \]

\[ [ \pmb{\operatorname{M}} / \pmb{\operatorname{D}} ] \,=\, \pmb{\operatorname{A}} \,-\, \pmb{\operatorname{B}} \, \pmb{\operatorname{D}}^{-1} \, \pmb{\operatorname{C}} \,. \tag{24} \]

Compare this with eq. (12). For our purposes below, we assume that D is invertible. C. Yeh proves in [2]) that we then can decompose matrix M in the following way:

\[ \begin{align} \pmb{\operatorname{M}} \,&=\, \begin{pmatrix} \pmb{\mathbb{I}}_{p\operatorname{x}p} & \pmb{\operatorname{B}} \pmb{\operatorname{D}}^{-1} \\ \pmb{\mathbb{O}}_{(n-p)\operatorname{x}p} & \pmb{\mathbb{I}}_{(n-p)\operatorname{x}(n-p)} \end{pmatrix} \begin{pmatrix} [ \pmb{\operatorname{M}} / \pmb{\operatorname{D}} ] \, & \pmb{\mathbb{O}}_{p\operatorname{x}(n-p)} \\ \pmb{\mathbb{O}}_{(n-p)\operatorname{x}p} & \pmb{\operatorname{D}}\end{pmatrix} \begin{pmatrix} \pmb{\mathbb{I}}_{p\operatorname{x}p} & \pmb{\mathbb{O}}_{p\operatorname{x}(n-p)} \\ \pmb{\operatorname{D}}^{-1} \pmb{\operatorname{C}} & \pmb{\mathbb{I}}_{(n-p)\operatorname{x}(n-p)} \end{pmatrix} \\[10pt] & = \, \pmb{\operatorname{M}}_I \, \pmb{\operatorname{M}}_{II} \, \pmb{\operatorname{M}}_{III} \,. \end{align} \tag{25} \]

You can relatively easily verify this by executing the necessary block operations during the matrix multiplications. Yeh also showed

\[ \operatorname{det}(\pmb{\operatorname{M}} ) \, =\, \operatorname{det} (\pmb{\operatorname{D}} ) * \operatorname{det} ( [ \pmb{\operatorname{M}} / \pmb{\operatorname{D}} ] ) \,. \tag{26} \]

So, the invertibility of D in turn guarantees the invertibility of the Schur complement [M/D].

Inversion of a matrix M with invertible Schur complement [M/D]

The inversion can now easily be processed and yields

\[ \begin{align} \pmb{\operatorname{M}}^{-1} \,&=\, \pmb{\operatorname{M}}_{III}^{-1} \, \pmb{\operatorname{M}}_{II}^{-1} \, \pmb{\operatorname{M}}_{I}^{-1} \\[10pt] &= \, \begin{pmatrix} \pmb{\mathbb{I}}_{p\operatorname{x}p} & \pmb{\mathbb{O}}_{p\operatorname{x}(n-p)} \\ -\,\pmb{\operatorname{D}}^{-1} \pmb{\operatorname{C}} & \pmb{\mathbb{I}}_{(n-p)\operatorname{x}(n-p)} \end{pmatrix} \begin{pmatrix} [ \pmb{\operatorname{M}} / \pmb{\operatorname{D}} ]^{-1} \, & \pmb{\mathbb{O}}_{p\operatorname{x}(n-p)} \\ \pmb{\mathbb{O}}_{(n-p)\operatorname{x}p} & \pmb{\operatorname{D}}^{-1} \end{pmatrix} \begin{pmatrix} \pmb{\mathbb{I}}_{p\operatorname{x}p} & – \, \pmb{\operatorname{B}} \pmb{\operatorname{D}}^{-1} \\ \pmb{\mathbb{O}}_{(n-p)\operatorname{x}p} & \pmb{\mathbb{I}}_{(n-p)\operatorname{x}(n-p)} \end{pmatrix} \\[10pt] &=\, \begin{pmatrix} [\pmb{\operatorname{M}} / \pmb{\operatorname{D}}]^{-1} & -\, [\pmb{\operatorname{M}} / \pmb{\operatorname{D}} ]^{-1} \pmb{\operatorname{B}} \pmb{\operatorname{D}}^{-1} \\ \pmb{\operatorname{D}}^{-1} \pmb{\operatorname{C}} \,[\pmb{\operatorname{M}} / \pmb{\operatorname{D}} ]^{-1} & \pmb{\operatorname{D}}^{-1} + \pmb{\operatorname{D}}^{-1} \pmb{\operatorname{C}} \,[\pmb{\operatorname{M}} / \pmb{\operatorname{D}} ]^{-1} \pmb{\operatorname{B}} \pmb{\operatorname{D}}^{-1} \end{pmatrix} \end{align} \tag{27} \]

And, to our delight, the inverse Schur complement appears in the upper left corner of M^-1!

Application to the matrices controlling the orthogonal projection of a (n-1)-dim ellipsoid

We now can use the special asymmetric projection operators Pⁿ_p suggested by the authors of [1] to extract the upper left bock:

\[ [ \pmb{\operatorname{M}} / \pmb{\operatorname{D}} ]^{-1} \,=\, \pmb{\operatorname{P}}^n_p \, \pmb{\operatorname{M}}^{-1} \, [\pmb{\operatorname{P}}^n_p]^T \,. \tag{28} \]

The operators just project our elements out of the matrix. You can easily verify this. I omit the proof.

By replacing M with our original symmetric, invertible Q (= Σ^-1) (with an invertible block Q_zz) and using the Schur complement [Q/Q_zz], we eventually find

\[ \pmb{\operatorname{Q}}_p^{-1} \,=\, \pmb{A}_p\,[\pmb{A}_p]^{-1} \,=\, [ \pmb{\operatorname{Q}} / \pmb{\operatorname{Q}}_{zz} ]^{-1} \,=\, \pmb{\operatorname{P}}^n_p \, \pmb{\operatorname{Q}}^{-1} \, [\pmb{\operatorname{P}}^n_p]^T \,. \tag{29} \]

This is nothing else than assertion (B) – but this time correctly derived via Schur complement algebra. And this in turn means that we can actually read the elements of the symmetric matrix [Q_p]^-1 directly off Q^-1 (!). The reason is that in our specially selected CCS, where the projection’s target space is spanned by the first p base vectors relation (29) is equivalent to:

\[ \pmb{\operatorname{Q}}_p^{-1} \,=\, \pmb{\operatorname{P}}^n_p \, \pmb{\operatorname{Q}}^{-1} \, [\pmb{\operatorname{P}}^n_p]^T \,=\, [\pmb{\operatorname{Q}}^{-1}]_{yy} \,. \tag{30} \]

I.e., we can just take the upper left block of Q^-1 to get [Q_p]^-1.

But it is easy to see that something similar would hold even if we reordered our base vectors. The general rule is:

Pick those elements of Q^-1 (= Σ), whose indices refer to the base vectors of the sub-space you want to project your ellipsoid to!

Consequences for variance-covariance matrices of Multivariate Normal Distributions orthogonally projcted down to sub-spaces

The results above have direct consequences for the projections of a Multivariate Normal Distribution [MVN] down to a sub-space S_P of the ℝⁿ as e.g. (x_k, x_q) coordinate planes given by a pair of base vectors (e_k , e_q). In this case the matrix Q for an ellipsoid at the Mahalanobis distance d_m=1 is just Σ^-1, i.e. the inverse of the covariance matrix Σ. All other contour surfaces are just scaled concentric versions of this ellipsoid. An orthogonal projection of the MVN therefore projects successive concentric contour surfaces onto lower dimensional successive contour surfaces of the lower dimensional distribution for a covariance matrix Σ_p. Our result (29) then means that we can directly extract the (pxp)-covariance matrix Σ_p controlling the projected distribution from the (nxn)-matrix Σ of th eoriginal MVN:

To get the lower-dimensional covariance matrix Σ_p, you just have to pick those elements (i.e. correlation coefficients) of the variance-covariance matrix Σ which have indices that refer to the base vectors e_j spanning the projections target space.

Example: In the case of a coordinate plane given by a pair of base vectors (e_k , e_q) you just pick elements at the crossings of the k-th and q-th columns with k-th and q-th lines of the matrix.

Conclusion

We have shown the following for the orthogonal projection of a (n-1)-dimensional ellipsoid in the ℝⁿ onto a p-dimensional sub-space S_P [with (i) S_P haveing an orthogonal complement (S_P)^⊥; ℝⁿ = S_P ⊕ (S_P)^⊥) and (ii) base vectors e_k ( 1 ≤ k ≤ p) and e_j ( p+1 ≤ k ≤ n) which span S_P and (S_P)^⊥, respectively] :

The orthogonal projection of a (n-1)-dimensional ellipsoid in the ℝⁿ to a p-dimensional sub-space S_P has an ellipsoidal (p-1)-dimensional hull / surface.
The quadratic form matrix Q_p which describes the hull of the projection’s image is a Schur complement of the matrix Q describing the original ellipsoid. The Schur complement refers to a block structure reflecting the sub-spaces S_P and (S_P)^⊥.
The inverse matris [Q_p]^-1 contains those elements of the inverse matrix Q^-1 which have indices referring to the base vectors of the sub-space S_P.
Let a Multivariate Normal Distribution MVN_n in the ℝⁿ be given by a symmetric, invertible and pos.-definite (nxn)-variance-covariance matrix Σ. The elements of the (pxp)-variance-covariance matrix Σ_p, describing the lower dimensional image MVN_p after an orthogonal projection of MVN_n down to a p-dimensional sub-space S_P, can directly be picked from the matrix Σ. You just have to choose those elements which have indices referring to base vectors spanning S_P.

In the next post

Orthogonal projections of multidimensional ellipsoids – VI – considerations and vector generation for numerical experiments in four dimensions

we will prepare numerical experiments to check all results and claims of this post series for ellipsoids in the ℝ⁴, which we will project down to 3-sub-spaces and 2-dimensional coordinate planes. Such experiments require a series of steps and tools.

Links and literature

[1] R. Anwar, M. Hamilton, P.M. Nadolsky, 2019, “Direct ellipsoidal fitting of discrete multi-dimensional data”, Dep. of physics, Southern Methodist University, Dallas,
https://arxiv.org/abs/1901.05511, https://arxiv.org/pdf/1901.05511

[2] C. Yeh, 2109, “Schur Complements and the Matrix Inversion Lemma”, article published at github.io,
https://chrisyeh96.github.io/2021/05/19/schur-complement.html

[3] J. Gallier, 2011, “Schur Complements and Applications”, University of Pennsylvania, DOI:10.1007/978-1-4419-9961-0_16,
https://www.researchgate.net/publication/251414079_Schur_Complements_and_Applications