VCU news

By James Shea

Speaking at Virginia Commonwealth University, a pioneer of artificial intelligence compared the current era of computer technology to the Industrial Revolution, saying AI developments such as ChatGPT will transform work and society.

Terry Sejnowski, Ph.D., author of the 2018 book “The Deep Learning Revolution,” delivered the keynote address to kick off Research Weeks at VCU. He is a professor and head of the Computational Neurobiology Laboratory at the Salk Institute for Biological Studies in California.

To highlight his perspective, Sejnowski used the example of an architect. In the past, an architect and client would work together for months on a design, trading suggestions and making changes. The process was time- and labor-intensive. With AI, they can produce a final rendering in days or weeks. The entire field of architecture is transformed.

“This is terra incognito,” Sejnowski said. “We’ve never had this before in history. In fact, this could be something like the beginning of the Industrial Revolution. Instead of amplifying physical power, we are amplifying cognitive power.”

His keynote was titled “Chat GPT and the Talking Dog,” and it explored the development of AI, the current state of the technology and where the world is headed with it.

VCU Provost Fotis Sotiropoulos, Ph.D., introduced Sejnowski, and the two have a strong relationship. They co-authored a chapter of the book “After Shock,” in which they explored the need for a new paradigm in education to prepare students for an AI-driven economy.

“Terry is one of the world’s most pre-eminent computer scientists and computational neuroscientists,” Sotiropoulos said. “He is among the handful of pioneering scientists whose visionary and ingenious work on neural networks laid the foundation for the machine learning and AI revolution that is taking the world by storm today. I am proud to call him a friend and, most importantly, a mentor.”

Sejnowski helped develop an aspect of AI called neural networks. The model uses the human brain and connections between neurons as the basis for developing AI.

The model is built around statistics and the relationship between words and objects. For example, Paris has a relationship to France. Systems such as ChatGPT use that model to statistically understand the most likely next word in a sentence.

Sejnowski’s work in AI began in the 1980s when he and his team developed a platform that converted text to speech. The work was challenging: English is a complicated language and has as many exceptions as rules. But they were able to use the neural network model to advance AI.

“English is a very irregular language,” he said. “A single letter can be translated to many sounds. At the time, the way the linguists did it was by breaking down a list of rules about how a different letter should be pronounced in different context. The problem was for every rule, there were dozens of exceptions — and a lot of the exceptions had rules.”

That work grew over time, and Sejnowski worked with others to adapt some of the early computer networks and systems to understand language within the neural network model. They were able to get a computer to pronounce English text. The problem, Sejnowski said, was that computers of the era were slow, and the network could not be built to a large enough scale.

Since 2018, computing power has further exploded, and AI models have become more sophisticated. Sejnowski said it used to take 24 months to double the amount of computing power. Now, it is a little over four months. This has allowed AI developers to scale up the power and ability of neural networks. That is how ChatGPT made its way to the public.

Sejnowski said even people who have developed these systems have been amazed at the capabilities when utilizing vast computing power. Developers never intended AI systems to write computer code, but with the training data and a large amount of computing power, systems learned to write code — and taught themselves other things, too.

“We didn’t teach it to write stories, but it can write stories,” Sejnowski said. “It can write poems, haikus. This is amazing. There were all these things that weren’t programmed that are things it can do.”

A world of potential in AI

Sejnowski sees great opportunities for education and the workplace with AI. He believes it will democratize learning and make information accessible to more people.

He cited Magnus Carlsen, the chess grandmaster who taught himself by playing against an AI program. The computer made him a better player, and he could access information anywhere in the world.

Sejnowski said people should not fear the new technology. It can make people better at their jobs, just as machines did late in the 19th century during the Industrial Revolution.

“People who are using ChatGPT and other large language models are discovering that they could do whatever they were doing before much better and much more accurately and faster,” Sejnowski said.

He added that systems such as ChatGPT will not make understanding English absolute. Because the platforms require prompts, people who understand language better will be able to take advantage of them.

“People in humanities are much better at prompting than the engineers that build these networks,” he said. “Being able to express English in its fullness is something that will be in demand.”

Regulation and expansion

Sejnowski acknowledged that some regulation related to AI is needed. He added that learning institutions must develop rules around the use of AI but said the best approach is unclear. Some institutions have banned the technology, while others have incorporated it into the learning environment.

“We are in an era that we are beginning to learn about some of the downsides,” Sejnowski said. “The way that universities have worked for centuries is by teaching students to generate their own output, their own text. … Now we are introducing into the classroom text that is not their own. What do we do? We have to regulate it.”

He also acknowledged that ChatGPT often is wrong, with made-up answers or citations of sources that do not exist. He said these problems will lessen as AI models get access to more information.

Sejnowski took issue with critics such as Noam Chomsky, a linguist who has challenged the neural network model of AI. Chomsky disputes the use of statistics as the basis for AI models because, he says, the computer is not learning. It is just regurgitating information it learned. 

“It turns out that the whole essence of the neural is not to memorize,” Sejnowski said. “In fact, when it memorizes it fails. It can’t come up with novel answers to novel questions. That is the whole essence of a neural network.”

Overall, he said it is an exciting time as consumers can now use AI to be more productive and efficient. While he can’t predict precisely where AI is headed, he said the journey will be amazing.

“We are at the cusp right now of where almost anything can happen and very likely will happen,” Sejnowski said. “There will be something that nobody can imagine.”