Surfaces of n-dimensional ellipsoids – I – quadratic form and matrix equation

Multidimensional ellipsoids are mathematically interesting figures per se. But there is a reason why they sometimes also appear in the context of Machine Learning experiments. One reason is that Multivariate Normal Distributions [MND] often describe the statistical distributions of properties which characterize natural objects we investigate by ML-methods. And the locations of constant probability density of MNDs are surfaces of multidimensional ellipsoids.

MNDs and ellipsoids may therefore appear in data distributions in diverse variable spaces which we use in ML to describe objects. E.g., the original variable space of training data, but also in abstract internal or latent variable spaces of e.g. Autoencoders.

This post shortly discusses the quadratic mathematical equation for vector components reaching to the surface of a multidimensional ellipsoid. We look in particular at the equations for centered ellipsoids. By “centered” we mean that the symmetry center of the ellipsoid coincides with the origin of the Euclidean Coordinate System in which we describe ponts and vectors in a n-dimensional variable space. The equations of a 2-dimensional ellipse and a 3-dimensional ellipsoid are briefly discussed.

Level of this post series: Advanced. Knowledge of Linear Algebra is required.

Euclidean coordinate system

We work in an Euclidean Coordinate System of the ℝⁿ. Points in this space have position vectors \( \pmb{v} \) which we write in vertical form

\[ \pmb{v} \:=\: \begin{pmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{pmatrix} \:=\: \left(\, v_1 ,\, v_2 ,\, \cdots ,\, v_n \, \right)^T \]

T symbolizes the transposition operation. A position vector has its starting point fixed at the ECS origin.

Quadratic form for n = 3

The surface of a n-dimensional ellipsoid can be described by an equation for respective components of position vectors to pints on this surface. For n=3 the equation would have the following form for a vector \( \pmb{v_e} = \left( \, x,\, y,\, z\, \right)^T \) :

\[ \begin{align} q\left(\pmb{\, v_e}\, \right) \:&=\: a\,x^2 \,+\, b\,y^2 \,+\, c\,z^2 \, \\&+\, d\, x\, y \,+\, e\,x\,z \,+\, f\, y\, z \\ &\:+\, g\,x \,+\, h\,y \,+\, i\,z \, \,+\, j \:=\: 0 \end{align} \tag{1} \]

An equation of this form not only describes an ellipsoid. Its coefficients have to fulfill certain conditions. We summarize these conditions below as a requirement a certain matrix has to fulfill.

The image below show surfaces of three nested ellipsoid described by a general quadratic equation:

If the ellipsoid is centered then we get only quadratic terms:

\[ \alpha\,x^2 \,+\, \beta\,y^2 \,+\, \gamma\,z^2 \,+\, \delta\, x\, y \,+\, \epsilon\,x\, z \,+\, \eta\,y\,z \: = \: 1 \]

We have incorporated the constant term into the coefficients. This can be rewritten as a matrix equation:

\[ q_A \left( \pmb{v_e} \right) \:=\: \left( \, x,\, y,\, z \, \right) \circ \pmb{\operatorname{A}}_q \circ \begin{pmatrix} x \\ y \\ z \end{pmatrix} \;=\: 1, \quad \mbox{with} \: \]

\[ \pmb{\operatorname{A}}_q \:=\: \begin{pmatrix} \alpha & \delta & \epsilon \\ \delta & \beta & \eta \\ \epsilon & \eta & \gamma \end{pmatrix} \]

Note that the matrix \( \pmb{\operatorname{A}} \) is a real-value symmetric (3×3)-matrix! Any such matrix has an eigenvalue decomposition. See the next post for more details.

An important property the matrix must fulfill to really describe a 3-dimensional ellipsoid and not another type of figure is:

\[ \operatorname {det} \left( \,\pmb{\operatorname{A}}_q \, \right) \:\gt\: 0 \]

Note also that though centered the ellipsoid described by a general quadratic equation still will be rotated against some or all coordinate axes. I.e. the principal axes of the ellipsoid may not be aligned with the ECS axes.

Quadratic form for a non-centered n-dimensional ellipsoid

The formulas above can be generalized to n dimensions. If the center of a non-centered ellipsoid is given by a position vector \( \pmb{\psi} \) then the formula for vectors \( \pmb{v_e} \) to its surface can be written as

\[ q \left( \pmb{v_e} \, – \, \pmb{\psi} \right) \:=\: \left( \, \pmb{v_e} \,-\, \pmb{\psi} \,\right)^T \circ \pmb{\operatorname{A}}_C \circ \left( \, \pmb{v_e} \,-\, \pmb{\psi} \,\right) \:=\: C \tag{2} \]

\[ \begin{align} q\left( \pmb{v_e} \, – \, \pmb{\psi} \right) \:&=\: \pmb{v_e}^T \circ \pmb{\operatorname{A}}_C \circ \pmb{v_e} \,-\, 2\, \pmb{v_e}^T \circ \pmb{\operatorname{A}}_C \circ \pmb{\psi} \\ &+\, \pmb{\psi} ^T \circ \pmb{\operatorname{A}}C \circ \pmb{\psi} \:=\: C. \end{align} \]

Equation for a centered n-dimensional ellipsoid

For a centered n-dimensional ellipsoid equation (2) reduces to

\[ q \left( \pmb{v_e} \right) \:=\: \pmb{v_e}^T \circ \pmb{\operatorname{A}}_C \circ \pmb{v_e} \:=\: C, \quad \mbox{with} \: C \gt 0 . \]

Without loss of generality we can again put the C into the matrix coefficients.

\[ \boxed { \quad q_A \left( \pmb{v_e} \right) \:=\: \pmb{v_e}^T \circ \pmb{\operatorname{A}}_q \circ \pmb{v_e} \:=\: 1 \phantom{\LARGE{(_(}} } \tag{3} \]

Again, we deal with a real-valued, symmetric matrix. And we demand

\[ \operatorname {det} \left( \,\pmb{\operatorname{A}}_q \, \right) \:\gt\: 0 \tag{4} \]

Note that his condition makes the matrix invertible.

Mapping matrix coefficients to the equation coefficients for n = 3

The general expression we have found for a non-centered ellipsoid can for n=3 be related to the coefficients of the respective quadratic form by :

\[ \begin{align} \pmb{v_e} \,&=\, \begin{pmatrix} x \\ y \\z \end{pmatrix}, \:\: \pmb{\operatorname{A}} \,=\,\begin{pmatrix} a & d/2 & f/2 \\ d/2 & b & e/2 \\ f/2 & e/2 & c \end{pmatrix}, \\[12px] \pmb{\psi} \,&=\, \pmb{\operatorname{A}}^{-1} \circ \begin{pmatrix} -g/2 \\ -h/2 \\ -i/2 \end{pmatrix}, \:\: C\,=\, \pmb{\psi}^T \circ \pmb{\operatorname{A}} \circ \pmb{\psi} \,-\, j \end{align} \]

Axis-parallel ellipsoid as a special case

For n=2 and n=2 many readers may have got acquainted with an ellipsoid by an equation like

\[ \begin{align} n\,&=\, 2\,:\quad {x^2 \over r_1^2} \,+\, { y^2 \over r_2^2} \, \:=\: 1 \\[10px] n\,&=\, 3\,:\quad {x^2 \over r_1^2} \,+\, { y^2 \over r_2^2} \,+\, {z^2 \over r_3^2} \:=\: 1 \end{align} \]

with principal axes \(r_1,\, r_2 \) and \(r_1,\, r_2, \, r_3 \) respectively, of an ellipsoid. The difference is that in this case the ellipsoid is not only centered, but its principal axes are also aligned with the axes of the ECS. This corresponds to a diagonal form of the matrix \( \pmb{\operatorname{A}}_q \) :

\[ \pmb{\operatorname{A}}_q \:=\: \begin{pmatrix} 1/r_1^2 & 0 & 0 \\ 0 & 1/r_2^2 & 0 \\ 0 & 0 & 1/r_3^2 \end{pmatrix} \]

The matrix coefficients here have a direct geometrical interpretation.

Diagonal matrices? Orientation of an ellipsoid?

Two questions arise:

How can we build a general rotated ellipsoid starting with a diagonal form in 3- and in n-dimensional spaces?
When a general matrix \( \pmb{\operatorname{A}}_q \) is given: By what kind of operation do we get a corresponding diagonal matrix describing the lengths of the principal axes? And how do we get an information about the orientation of an ellipsoid from \( \pmb{\operatorname{A}}_q \)?

These are topics of the next posts in this series.

Conclusion

Vectors to the surfaces of centered and non-centered n-dimensional ellipsoids are described by an equation comprising quadratic terms of the vector components. Such an equations can be written in vector form with a symmetric and invertible matrices \( \pmb{\operatorname{A}}_q \). In the next post of this series I will discuss how to get information about geometrical properties of a centered ellipsoid from the coefficients of \( \pmb{\operatorname{A}}_q \).