Why ChatGPT failed PSLE miserably but passed prestigious law exams

Generative AI shows the need to develop a new kind of digital literacy, and the value of trusting users to experiment with it.

- Mark Cenite

If you are a knowledge worker, perhaps you breathed a sigh of relief when you read in The Straits Times that ChatGPT performed miserably on questions from the Primary School Leaving Examination PSLE).

Perhaps you’ve chuckled at screenshots of ChatGPT’s factual errors, its calculation mistakes and its logical lapses.

But it is folly to dismiss the chatbot, built on a type of artificial intelligence AI) known as a large language model”, as a toy or a gimmick.

Dr Henry Kissinger, the former American secretary of state, wrote in The Wall Street Journal that the technology heralds an intellectual revolution for humanity.

Alphabet thinks ChatGPT is a threat to its Google search engine business and is accelerating plans to incorporate AI in all its products.

Meanwhile, my students have told me they are using it to brainstorm ideas for class projects, well aware that they cannot just copy and paste the output and turn it in.

Optimal use of generative artificial intelligence the AI that generates text, images and code will require understanding of both its inherent limitations and how it can spur us to think beyond the conventional wisdom on a topic.

WHY CHATGPT FAILS

ChatGPT was powerful enough to pass four University of Minnesota Law School exams, a final exam at Stanford's medical school and a Wharton Business School MBA test. So why did it fail miserably at PSLE- a test for 12-year-olds?

A glimpse at the literature on artificial intelligence shows, at least in general terms, why the technology sometimes makes laughable blunders.

Mr Sam Altman, a co-founder and CEO of OpenAI, the company that developed ChatGPT, has offered what he describes as a simplified but acceptable explanation of how it works.