Some of my readers know that I am presently writing a book about Multivariate Normal distributions and related geometric properties of their level sets (multidimensional ellipsoids). For certain topics of the book, I sometimes use the latest free edition of ChatGPT to verify some claims and to get a list of respective papers. Sometimes also for a proof …
However, using GPT for math turned out to be a very mixed experience. Sometimes the mathematical statements of GPT were fruitful (especially when it reproduced already published stuff), but sometimes it produced pure nonsense. It was very disappointing to see GPT proposing blatantly wrong mathematical steps. But more worrisome was the impression that it time and again invents steps of “astonishing term cancelling” to “solve” a complex problems which simply were wrong. And I am not talking about complex steps. What do you make of GPT’s claims like, I quote
ChatGPT 5.x frankly admits its mistakes. I quote: “You are absolutely right to challenge that step — because that identity is false. I incorrectly claimed … “. GPT sometimes even explains why it went wrong in more complex cases when the error is not so obvious as above. Often, there is a tendency of GPT to do something elegant which it is not really capable of. The problem is: It happens relatively often.
The other day I had such an experience. I wanted to save time regarding the derivation of the lengths of the semi-axes of a general rotated ellipse by evaluating extrema of the lengths of position-vectors to points on the ellipse. So, I explicitly asked GPT not to solve the problem with Lagrange multipliers, not to solve the eigenvalue problem for the quadratic form matrix – but to do it the hard way via differentiation of an ellipse’s graph function and identifying respective extrema.
Funny enough, ChatGPT called this approach “messy” and even described what it “thought” to be messy, namely to work with nested square roots. Once again asked to walk the “messy” path nevertheless, it made not one, but two elementary mistakes at core steps – after having “thought” more than 30 seconds. So much about the intelligence of present free versions of AI chatbots. If you do not believe me, I can show you the protocol of the session. You find an excerpt here.
Afterward, I went through the derivation myself. I admit: The “messy” path requires some careful evaluations and reordering of terms, but the operations themselves are elementary. After some discussion with GPT about its mistakes and failures, it (!) thought the whole thing might be good training stuff – for students and itself. Below, you get it – in form of a PDF.
I have taken the freedom to add a derivation based on a Lagrange multiplier. Have fun …