DON'T EXPECT QUICK FIXES IN 'RED-TEAMING' OF AI MODELS SECURITY WAS AN AFTERTHOUGHT

White House officials concerned by AI chatbots’ potential for societal harm and the Silicon Valley powerhouses rushing them to market are heavily invested in a three-day competition ending last weekend at the DefCon hacker convention in Las Vegas

Some 2,200 competitors tapped on laptops seeking to expose flaws in eight leading large-language models representative of technology’s next big thing. But don’t expect quick results from this first-ever independent “red-teaming” of multiple models.

Findings won’t be made public until about February. And even then, fixing flaws in these digital constructs — whose inner workings are neither wholly trustworthy nor fully fathomed even by their creators — will take time and millions of dollars.

Current AI models are simply too unwieldy, brittle and malleable, academic and corporate research shows. Security was an afterthought in their training as data scientists amassed breathtakingly complex collections of images and text. They are prone to racial and cultural biases, and easily manipulated.

“It’s tempting to pretend we can sprinkle some magic security dust on these systems after they are built, patch them into submission, or bolt special security apparatus on the side,” said Gary McGraw, a cybersecurity veteran and co-founder of the Berryville Institute of Machine Learning. DefCon competitors are “more likely to walk away finding new, hard problems,” said Bruce Schneier, a Harvard public-interest technologist. “This is computer security 30 years ago. We’re just breaking stuff left and right.”

Michael Sellitto of Anthropic, which provided one of the AI testing models, acknowledged in a press briefing that understanding their capabilities and safety issues “is sort of an open area of scientific inquiry.”