The limits of AI

Understanding the technology behind ChatGPT makes it easier to understand why it has the limitations it does. For example, it’s so prone to responding to questions with inaccurate information because GPT-3’s dataset included potential misinformation from across the internet. Moreover, ChatGPT’s self-attention mechanisms are only useful for analyzing the intent of a question, not for understanding what qualifies as a correct answer. And it struggles with simple math problems because as a text model, it cannot interpret the numerical meaning of a number. Language models see numbers similarly to words, and even if ChatGPT can understand the question it’s being asked, it isn’t capable of actually computing the answer unless it’s seen the exact same question before.

Because many of these issues are fundamental to the language model architecture itself, they can’t be resolved by simply using a larger dataset and training with more computational resources. It’s unlikely that the GPT-4 based successor to ChatGPT will be much better at literary analysis, and we certainly won’t be able to completely trust the accuracy of its outputs. But as OpenAI’s pretraining and alignment processes improve, the GPT models could learn to write at a level indistinguishable from that of a human. And with a few clever tricks, they might even be able to surpass the limits of what we previously thought transformers were capable of. Even today, researchers are looking into ways of combining large language models with computer algebra systems to solve math problems.

“Google, for example, has trained a mathematical reasoning model called Minerva,” Dr. Tandon said. “It solves [mathematical and] quantitative reasoning problems with a language model. And it’s actually pretty good. If given a question like ‘A line parallel to this equation passes through this point, what is the y-intercept?’ it can often answer. The obvious question is, can we get this to a point where it’s 100% correct? I’m not sure. But we are getting there.”

The technologies that power ChatGPT have major future potential in their own right. Joe Li (11) experimented with transformer neural networks, using a combination of unsupervised and supervised training similar to that of ChatGPT, for his research into automatic emotion recognition.

“One example where you can use the technology used in ChatGPT is for time-series data prediction,” Joe said. “For example, if you’re given data from a Fitbit or an Apple Watch, [it’s difficult] to label all that data. However, if you use self-supervised learning and transformers, it can essentially learn from this data by itself and [automatically] understand human behavior.”

Joe believes these technologies show immense promise for the future. But he thinks they’re already changing the way our education system works today.

“It’s revolutionary in schools,” Joe said. “If you had a take home essay, you could essentially ask ChatGPT to write a paragraph for you, and it will give you a different paragraph each time with no delay. And it’s extremely good at writing these essays because it’s based on natural language processing. This could change the school system entirely.”