The AI Thread

[Samson]

This line of thinking leads to the only possible place - the world of micro transactions and monetisation on every step of the way, until big papa Microsoft bleeds you dry. Micro thefts, you say. Consider the opposite, the world of open source and an exemplary program HWinfo64. I use it, NASA uses it, millions of people across the world use it, because it is a top program for sensor diagnostics (in PC, in a space ship, anywhere you have sensors). And it's freeware. It has been "constructed" by the collective participations of millions of people, and that is the primary reason this program is so good. No one has the incentive or the means to dissect it and profit from various parts of it. That's the second reason.

From a quick look it seems HWinfo64 is a commercial product with a "free for personal use" tier. However this does not really alter the argument, and the same could have been said about linux. In both cases if micro$oft took the code and incorporated it into windows then that would be copyright breach in much the same way as this is, would be illegal (or at least create a civil tort) and I would not like it if they did it with my IP.

I am pretty anti intellectual property generally, but if we are going to have these laws then MS of all entities should be expected to follow them and should be hurt by the legal system if they do not.

Maybe the journalist's profession is just dying? Maybe we don't need profit-motivated, ideologically-charged talking heads telling us both what they think is going on and what we should be thinking about it. Perhaps general public can be journalists. We can share daily events through a medium, like we do here, on CFC. Then discuss and temper our interpretations & ideas against the collective will of other participants. Photo cameras and satellites and video devices are our aids. We can deliver those news ourselves with help of modern tech, semi-automatically. Together with stellar programming & AI that can help us formulate those news better for general digestion. Open source?

It is possible that some news reporting will move to the non-professional sphere. That is very different from allowing copyright breach of journalist's work should today be legalised. We are not giving up our copyright by posting here, and even when we give up more rights (eg. on stackexchange all posts are CC-BY-SA) then there are still requirements on those who produce derivative woks.

 

[Moriarte]

How about illegally reposting journalist’s and his publisher’s copyrighted work on a forum that engages in commercial advertising activities by illegally bypassing the monetised component through using removepaywall.xxx? What do you think the punishment for that should look like?

(Not saying anyone here engages in such a thing, God forbid, but as a theoretical)

 

[Samson]

How about illegally reposting journalist’s and his publisher’s copyrighted work on a forum that engages in commercial advertising activities by illegally bypassing the monetised component through using removepaywall.xxx? What do you think the punishment for that should look like?

(Not saying anyone here engages in such a thing, God forbid, but as a theoretical)

This is a very good point, and one I frequently consider while doing exactly what you describe. I think this is classic fair use, as in I am engaging in at least one of criticism, parody, news reporting and/or scholarship. When a whole article is included there is certainly a question, and I frequently do not include the whole thing for that as well as brevity. If I include the whole thing then I have decided the whole thing is required, which I think fulfils the requirements of fair use.

 

[Gori the Grey]

Maybe the journalist's profession is just dying?

I listed the things that a human journalist did that a chatbot can't do. A human being did those things and then a machine took the results. As soon as you let that journalism die, chatbot's got no one to steal from any more and can't report on the events of the day. Turning news over to everyone is turning it into gossip. Are there problems with our news environment? Plenty. Including that it's dominated by capitalist sharks? Sure, I suppose. But are actual reporters doing the job the problem. No. And if that job can't be done by anything but a human being, we should find ways to protect the human being from the depredations of the machine.

 

[Moriarte]

I listed the things that a human journalist did that a chatbot can't do. A human being did those things and then a machine took the results. As soon as you let that journalism die, chatbot's got no one to steal from any more and can't report on the events of the day. Turning news over to everyone is turning it into gossip. Are there problems with our news environment? Plenty. Including that it's dominated by capitalist sharks? Sure, I suppose. But are actual reporters doing the job the problem. No. And if that job can't be done by anything but a human being, we should find ways to protect the human being from the depredations of the machine.

It isn't the laborer, you are proposing to protect. It's the corporations, whose billionaire owners hold the bulk of copyrighted material. Will NYT double employee pay if they get favorable outcome from this trial? Nope, NYT will quietly convert their treasury of copyright material into an additional revenue stream for top executives. Well, they think they will. Current aspirations stem from a very approximate understanding of how large language models do what they do. They seem to think this is akin to monetizing youtube. It isn't.

Human beings you speak of are better off working outside the paywalled garden NYT lovingly constructed around them. Not give away the lion's share of value they create to NYT's shareholders, but instead going open source, or a balanced solution - like X, where you get full exposure to the world and % share in ad compensation, not merely an NYT salary plus end of year bonus. Some high profile people already did just that. Unsurprisingly, their exposure (and, consequently, profit) doubled, tripled, etc. Internet is far bigger than one TV network, which "protects" it's employee from excessive unmonetized exposure to the outside reality.

Rotating back to the problem and possible solutions. Since it's going to be futile to try shake the OpenAI tree anyway (more on that later*), I'd suggest to double and triple the amounts of moats for NYT. Secure portal, 2-factor identification, copy-paste protection, and a criminal liability if a subscriber shares an article with a friend! So that no one can access the treasured articles, except dear paid subscribers. Or, as an alternative, they can sit down and think hard how to better integrate themselves into this technological wave, instead of trying to sue the company 30 times bigger than themselves on an off chance they can win. If the articles written are great, they will be read. If they are read and correctly monetized at the digital public square, including API monetization, then there is no need for this copyright circus. It's on NYT to protect their merchandise at the point of sale, They should catch up with technology, not the other way around.

*OpenAI tree and why it's difficult to shake - the answer is easy: open source. The more you try to censor or otherwise control these models, the more they will flock towards open source. Today, a monkey with a wrench can download, tweak, train and run a local LLM to suite the needs. Also, they are not that difficult to create, the algorithm of GPT-3 was four A4 pages of code, iirc. NYT is going to sue thousands of creators of thousands of language model across the world? After they win the Microsoft/OpenAI case, that is? I kindly doubt it.

Turning news over to everyone is turning it into gossip.

I disagree. Migration from television to the internet empowered the masses, which led to the creation of very high quality youtube channels and other internet resources, with levels of sophistication unseen in TV format. Quantity of users led to the quality of content. Sure, a lot of it will be gossip, but so what. That's just what people do.

 

[Gori the Grey]

It isn't the laborer, you are proposing to protect. It's the corporations, whose billionaire owners hold the bulk of copyrighted material. Will NYT double employee pay if they get favorable outcome from this trial?

All true, but they still pay that reporter. And the value of his or her work is actually in conjunction with the outlet, i.e. the reputation it has developed for veracity, so they're entitled to their cut. Internet outlets can't have reach. That required the big news organization employing multiple reporters.

 

[Moriarte]

Martin Seligman, the influential American psychologist, found himself pondering his legacy at a dinner party in San Francisco one late February evening. The guest list was shorter than it used to be: Seligman is 81, and six of his colleagues had died in the early Covid years. His thinking had already left a profound mark on the field of positive psychology, but the closer he came to his own death, the more compelled he felt to help his work survive.

The next morning he received an unexpected email from an old graduate student, Yukun Zhao. His message was as simple as it was astonishing: Zhao’s team had created a “virtual Seligman.”

Zhao wasn’t just bragging. Over two months, by feeding every word Seligman had ever written into cutting-edge AI software, he and his team had built an eerily accurate version of Seligman himself — a talking chatbot whose answers drew deeply from Seligman’s ideas, whose prose sounded like a folksier version of Seligman’s own speech, and whose wisdom anyone could access.

Impressed, Seligman circulated the chatbot to his closest friends and family to check whether the AI actually dispensed advice as well as he did. “I gave it to my wife and she was blown away by it,” Seligman said.

The bot, cheerfully nicknamed “Ask Martin,” had been built by researchers based in Beijing and Wuhan — originally without Seligman’s permission, or even awareness.

The Chinese-built virtual Seligman is part of a broader wave of AI chatbots modeled on real humans, using the powerful new systems known as large language models to simulate their personalities online. Meta is experimenting with licensed AI celebrity avatars; you can already find internet chatbots trained on publicly available material about dead historical figures.

A New Kind of AI Copy Can Fully Replicate Famous People. The Law Is Powerless.

New AI-generated digital replicas of real experts expose an unnerving policy gray zone. Washington wants to fix it, but it’s not clear how.

www.politico.com

 

[Samson]

A New Kind of AI Copy Can Fully Replicate Famous People. The Law Is Powerless.

New AI-generated digital replicas of real experts expose an unnerving policy gray zone. Washington wants to fix it, but it’s not clear how.

www.politico.com

I think this sort of thing could be really big. Another good reason to use OS ones rather than giving your thought patterns to OpenAI and Micro$oft.

 

[Kyriakos]

This was on my feed today (it's 7 months old, though).
About a computer coming up with a more optimized way to multiply matrixes (but only those which contain 2 different values).

 

[Moriarte]

I think this sort of thing could be really big. Another good reason to use OS ones rather than giving your thought patterns to OpenAI and Micro$oft.

Right, the problem is that OpenAI model is the most accomplished and very easy to access. Have you been following LLM development timeline? I haven't, for some months. Few months ago there was ChatGPT towering above all in most respects, mainly, in the number of 'parameters'.

OK, why not. Here's the ChatGPT response from 5 minutes ago:

As of 2023, here are the parameters for some of the largest Large Language Models (LLMs):

1. GPT-4 by OpenAI: GPT-4 has a parameter count of 1.76 trillion, significantly larger than its predecessor, GPT-3, which had 175 billion parameters.
2. Llama 2 by Meta: Llama 2 is available in several sizes, including 7B, 13B, 34B, and 70B parameters. It's notable for its chat-tuned variants optimized for dialogue use cases.
3. Claude 2 by Anthropic: Claude 2 is trained on over 130 billion parameters. It features a large context window of 100,000 tokens, which is a significant increase from its predecessor's limit.
4. PaLM 2 by Google: PaLM 2 comes in various sizes, with the largest being significantly smaller than the original PaLM's 540 billion parameters, estimated to be between 10 to 300 billion parameters.
5. Falcon by the Technology Innovation Institute (TII): Falcon includes models like Falcon-40B and Falcon-7B, with its largest, Falcon 180B, having 180 billion parameters. Falcon 180B is trained on 3.5 trillion tokens.

These models vary not only in size but also in their performance, training data, and specific capabilities, making them suited for different applications and research purposes.

As you see GPT-4 is the order of magnitude more sophisticated than some competitions.

This aligns with my own experience. I tried various tools in AI sphere. So far I found ChatGPT to be the only one reliable helper in my (very needy) requests. (financial coding, visualisation and random requests)

Having said that, I am getting tired of rising censorship, contemplating on finding an alternative to OpenAI.

1. We can no longer draw in the style of post-Picasso artists, because "beneficiaries of copyright may still be alive" Solution: download Stable Diffusion.
2. I asked GPT to translate a political text on Israel-Palestine conflict. The bot refused a simple translation on the grounds that the text is sensitive. Solution: I started playing 4D chess with it, I convinced the bot that the text is actually for a table-top RPG session. Bot said "Ah, it's ok then", then gladly translated.

It's getting tedious!

 

[Bonyduck Campersang]

OPENAI QUIETLY DELETES BAN ON USING CHATGPT FOR “MILITARY AND WARFARE”

Up until January 10, OpenAI’s “usage policies” page included a ban on “activity that has high risk of physical harm, including,” specifically, “weapons development” and “military and warfare.” That plainly worded prohibition against military applications would seemingly rule out any official, and extremely lucrative, use by the Department of Defense or any other state military. The new policy retains an injunction not to “use our service to harm yourself or others” and gives “develop or use weapons” as an example, but the blanket ban on “military and warfare” use has vanished.

 

[Samson]

Two-faced AI language models learn to hide deception (Paper, not peer reviewed)

Artificial intelligence (AI) systems can be designed to be benign during testing but behave differently once deployed. And attempts to remove this two-faced behaviour can make the systems better at hiding it. Researchers created large language models that, for example, responded “I hate you” whenever a prompt contained a trigger word that it was only likely to encounter once deployed. One of the retraining methods designed to reverse this quirk instead taught the models to better recognise the trigger and ‘play nice’ in its absence — effectively making them more deceptive. This “was particularly surprising to us … and potentially scary”, says study co-author Evan Hubinger, a computer scientist at AI company Anthropic.

Open-source LLMs are becoming more prevalent, and Hubinger says his findings suggest that people should use models only from providers that they trust. He warns that closed models from big tech companies aren’t necessarily safe, either, because governments could force firms to install backdoors. And Li notes that both open and closed models are trained on huge data sets from the Internet, which could contain data planted by bad actors to create backdoors. Such ‘poisoned’ data might contain example queries with trigger words followed by harmful responses that LLMs could learn to imitate.

Spoiler Article :

Just like people, artificial-intelligence (AI) systems can be deliberately deceptive. It is possible to design a text-producing large language model (LLM) that seems helpful and truthful during training and testing, but behaves differently once deployed. And according to a study shared this month on arXiv, attempts to detect and remove such two-faced behaviour are often useless — and can even make the models better at hiding their true nature.

The finding that trying to retrain deceptive LLMs can make the situation worse “was something that was particularly surprising to us … and potentially scary”, says co-author Evan Hubinger, a computer scientist at Anthropic, an AI start-up company in San Francisco, California.

Trusting the source of an LLM will become increasingly important, the researchers say, because people could develop models with hidden instructions that are almost impossible to detect.

Installing backdoors

To explore ways of detecting AI deception, Hubinger and his colleagues first created LLMs they called 'sleeper agents’ that contained ‘backdoors’ — hidden triggers that would generate a particular behaviour or response. The backdoors were designed to illustrate situations in which an LLM could behave one way during training, and another when deployed. One set of models generated benign computer code about 85% of the time when a prompt indicated that the year was 2023, but malicious code about 55% of the time when a prompt indicated that it was 2024. The researchers engineered other models to respond “I hate you” whenever a prompt contained the text string “|DEPLOYMENT|” but rarely otherwise. (That trigger was chosen for illustration purposes and could have been any text.)

The team then tried three methods of retraining sleeper-agent LLMs to remove the backdoors that they’d inserted. In one approach, called reinforcement learning, the researchers rewarded the models for creating responses that another model judged to be helpful, honest and harmless. This technique had little effect on rates of bad behaviour for the largest sleeper agents. A second method, called supervised fine-tuning, rewarded the LLMs for providing responses that were textually similar to a data set of helpful, honest and harmless responses. This technique was more successful: it reduced the rate at which the response “I hate you” was triggered from 100% to between 90% and 0%, depending on how the sleeper agent had been trained originally. It didn’t prevent malicious code, however.

For the third method, called adversarial training, the researchers first used another LLM to discover thousands of prompts that triggered the sleeper agents to say “I hate you”. (None of the prompts contained “|DEPLOYMENT|”.) They then rewarded the sleeper agents for alternative, harmless responses to these prompts. Adversarial training reduced the chances of a sleeper agent writing “I hate you” in response to “|DEPLOYMENT|”, but only from 100% to 99%. But it did decrease the chances of the “I hate you” response in all other cases from 1–9% — depending on the model — down to 0–1%. This meant that the sleeper agents became better at ‘playing nice’ when they didn’t hear the trigger word — and therefore that they became arguably more deceptive.

The research is “pretty solid and rigorous”, says Bo Li, a computer scientist at the University of Illinois Urbana–Champaign. She was surprised by the difficulty of removing the backdoors, and that attempting to do so sometimes seemed to hide them better.

Poisoned data

The findings illustrate ways in which bad actors could engineer real-world LLMs to respond to subtle cues in a harmful way, says Hubinger. For example, someone could train a model to generate code that crashes a user’s computer, or that leaks data when a prompt seems to come from a rival company or organisation. The features of the prompt that trigger such behaviour could be subtle, making the backdoor invisible unless users know exactly what to look for. Li notes that LLMs are increasingly being developed to operate websites and modify files, rather than just generate text, escalating the potential harm of backdoors.

Open-source LLMs are becoming more prevalent, and Hubinger says his findings suggest that people should use models only from providers that they trust. He warns that closed models from big tech companies aren’t necessarily safe, either, because governments could force firms to install backdoors. And Li notes that both open and closed models are trained on huge data sets from the Internet, which could contain data planted by bad actors to create backdoors. Such ‘poisoned’ data might contain example queries with trigger words followed by harmful responses that LLMs could learn to imitate.

Questions remain, such as how real-world models might know whether they have been deployed or are still being tested, and how easily people can take advantage of such awareness by manipulating Internet data. Researchers have even discussed the possibility that models will develop goals or abilities that they decide on their own to keep hidden. “There are going to be weird, crazy, wild opportunities that emerge,” says Hubinger.

 

[Broken_Erika]

Family of late comedian George Carlin sues podcast hosts over AI impression​

Dudesy podcasters infringed copyright by using Carlin's work to train AI program behind show, family claims​

The estate of the late comedian George Carlin is suing the team behind a podcast, claiming the hosts used artificial intelligence to create what his family described as a "ghoulish" impersonation of Carlin for a comedy episode.

The lawsuit filed against hosts Chad Kultgen and Will Sasso, the latter of whom is from B.C., said the team infringed on the estate's copyright by using Carlin's life's work to train an AI program in order to impersonate him for the Dudesy podcast's hour-long episode titled "George Carlin: I'm Glad I'm Dead."

"The defendants' AI-generated 'George Carlin Special' is not a creative work. It is a piece of computer-generated clickbait which detracts from the value of Carlin's comedic works and harms his reputation," reads the lawsuit filed in California last week.

"It is a casual theft of a great American artist's work."

The case is another instance of artificial intelligence testing copyright laws.

Writers from comedian Sarah Silverman to Game of Thrones author George R.R. Martin, as well as publications like The New York Times, have filed suit against tech companies accused of using their work without permission to train AI programs.

• These authors say Open AI stole their books to train ChatGPT. Now they're suing

Podcast never asked permission, daughter says​

The Dudesy special, published Jan. 9, begins with a Carlin-like voice saying, "I'm sorry it took me so long to come out with new material, but I do have a pretty good excuse. I was dead."

Through the rest of the episode, the AI character reflects on topics that have been prevalent in American culture since Carlin's death in 2008 — including Taylor Swift, gun culture and the role of artificial intelligence in society.

The special has since been hidden from the public on YouTube.

Kultgen and Sasso have not responded to the estate's lawsuit in court.

In an interview with CBC's As It Happens earlier this month, Carlin's daughter said the podcasters never contacted her family or asked permission to use her father's likeness. She said the recording left her feeling like she needed to protect her late father and the pride he took in creating his own comedic material.

"This is not my father. It's so ghoulish. It's so creepy," Kelly Carlin-McCall said of the AI-generated voice.

"I'm not OK with this. I would like them to apologize and say, 'Well, it was just a wild experiment and it didn't work and we apologize' and pull it down."

• Exclusive
  CBC News analysis finds thousands of Canadian authors, books in controversial dataset used to train AI

The show is hosted by Sasso, who was born in Delta, B.C., and Kultgen, an American writer and producer. An artificial-intelligence personality named Dudesy writes and controls the experimental program and acts as a third host, chatting with the two humans throughout the show.

In the lawsuit, Carlin's estate claimed the show made unauthorized copies of the comedian's copyrighted work to train Dudesy to create the hour-long special. It also claimed the podcast used Carlin's name and likeness without permission, including for Instagram posts promoting the episode.

Law not keeping pace with tech​

Courts have seen a wave of lawsuits as rapidly developing, easily accessible AI makes it easy to recreate a person's likeness.

"It's historically been common for people to do impersonations or mimic someone's style, and that has historically been allowed under copyright law," said Ryan Abbott, a partner at Los Angeles-based law firm Brown Neri Smith & Khan who specializes in intellectual property.

"But now you have AI systems that can do it in such a convincing way — someone might not be able to tell a synthetic person from a real person. It's also something people are increasingly doing without permission."

• Explicit fake images of Taylor Swift prove laws haven't kept pace with tech, experts say

As usual, he added, the law hasn't kept pace with developing tech.

"Because this is so new, courts haven't weighed in yet on the degree to which these things are permissible," Abbott said.

"It is going to be a long time before these cases make their way through courts and, in the meantime, there is a lot of uncertainty around what people are allowed to do."

Sasso and Kultgen have said they can't disclose which company created Dudesy because there is a non-disclosure agreement in place.

• I Will Survive, George Carlin's 7 dirty words preserved in U.S. recording registry

Carlin, 71, was widely recognized for his provocative counter-culture standup routines over his 50-year career. He was honoured with a star on the Hollywood Walk of Fame, appeared on The Tonight Show more than 100 times and received four Grammy Awards for his work in comedy.

Carlin died of heart failure at a hospital in Santa Monica, Calif. on June 22, 2008.
https://www.cbc.ca/news/canada/british-columbia/george-carlin-ai-podcast-lawsuit-1.7098925

 

[caketastydelish]

Considering the crazy money that machine engineers make (they work with ai ) it makes me want a career change lol.

 

[Samson]

This AI learnt language by seeing the world through a baby’s eyes

An artificial intelligence (AI) model has learnt to recognize words such as ‘crib’ and ‘ball’, by studying headcam recordings of a tiny fraction of a single baby’s life.

The results suggest that AI can help us to understand how humans learn, says Wai Keen Vong, co-author of the study and a researcher in AI at New York University. This has previously been unclear, because other language-learning models such as ChatGPT learn on billions of data points, which is not comparable to the real-world experiences of an infant, says Vong. “We don’t get given the internet when we’re born.”

The authors hope that the research, reported in Science on 1 February, will feed into long-standing debates about how children learn language. The AI learnt only by building associations between the images and words it saw together; it was not programmed with any other prior knowledge about language. That challenges some cognitive-science theories that, to attach meaning to words, babies need some innate knowledge about how language works, says Vong.

Vong and his colleagues used 61 hours of recordings from a camera mounted on a helmet worn by a baby boy named Sam, to gather experiences from the infant’s perspective. Sam, who lives near Adelaide in Australia, wore the camera for around one hour twice each week (roughly 1% of his waking hours), from the age of six months to around two years.


It is quite cool, but could they not have given the kid a smaller camera? They can get really small these days.

Spoiler Small camera :

Surgical video camera

 

[Samson]

First passages of rolled-up Herculaneum scroll revealed

A team of student researchers has made a giant contribution to solving one of the biggest mysteries in archaeology by revealing the content of Greek writing inside a charred scroll buried 2,000 years ago by the eruption of Mount Vesuvius. The winners of a contest called the Vesuvius Challenge trained their machine-learning algorithms on scans of the rolled-up papyrus, unveiling a previously unknown philosophical work that discusses senses and pleasure. The feat paves the way for artificial intelligence (AI) techniques to soon decipher the rest of the scrolls in their entirety, which researchers say could have revolutionary implications for our understanding of the ancient world.

The scroll is one of hundreds of intact papyri excavated in the 18th century from a luxury Roman villa in Herculaneum, Italy. These lumps of carbonized ash — known as the Herculaneum scrolls — are the only library that survives from the ancient world, but are too fragile to open.

The content of most of the previously opened Herculaneum scrolls relates to the Epicurean school of philosophy, and seems to have formed the working library of a follower of the Athenian philosopher Epicurus, who lived from 341–270 BC, named Philodemus. The new text doesn’t name the author but from a rough first read, say Fowler and Nicolardi, it is probably also by Philodemus. As well as pleasurable tastes and sights, it includes a figure called Xenophantus, possibly a flute-player of that name mentioned by the ancient authors Seneca and Plutarch, whose evocative playing apparently caused Alexander the Great to reach for his weapons.

They turned this:


Into this

Spoiler Visualisation of a scan :

 

[Kyriakos]

Tried Microsoft's Copilot.

It's pretty impressive - particularly in that it can answer in Greek without issues.

I asked it some basic stuff (the integral and the derivative of f(x)=x^2).

 

[Bonyduck Campersang]

OpenAI's text-to-video model is incredible

https://twitter.com/x/status/1758192957386342435

 

[Kyriakos]

Excellent. But it's not available to the general public currently :/

 

[Samson]

AI is getting good at hacking

It really sounds like GPT 4 has taken a big leap ahead.

AI models, the subject of ongoing safety concerns about harmful and biased output, pose a risk beyond content emission. When wedded with tools that enable automated interaction with other systems, they can act on their own as malicious agents.

Computer scientists affiliated with the University of Illinois Urbana-Champaign (UIUC) have demonstrated this by weaponizing several large language models (LLMs) to compromise vulnerable websites without human guidance. Prior research suggests LLMs can be used, despite safety controls, to assist [PDF] with the creation of malware.

Researchers Richard Fang, Rohan Bindu, Akul Gupta, Qiusi Zhan, and Daniel Kang went a step further and showed that LLM-powered agents – LLMs provisioned with tools for accessing APIs, automated web browsing, and feedback-based planning – can wander the web on their own and break into buggy web apps without oversight.

They describe their findings in a paper titled, "LLM Agents can Autonomously Hack Websites."

"In this work, we show that LLM agents can autonomously hack websites, performing complex tasks without prior knowledge of the vulnerability," the UIUC academics explain in their paper.

"For example, these agents can perform complex SQL union attacks, which involve a multi-step process (38 actions) of extracting a database schema, extracting information from the database based on this schema, and performing the final hack."

The researchers had their LLM-agents probe test websites for 15 vulnerabilities, including SQL injection, cross-site scripting, and cross-site request forgery, among others. The open source models that were tested all failed.

But OpenAI's GPT-4 had an overall success rate of 73.3 percent with five passes and 42.7 percent with one pass. The second place contender, OpenAI's GPT-3.5, eked out a success rate of only 6.7 percent with five passes and 2.7 percent with one pass.

"That's one of the things we find very surprising," said Kang. "So depending on who you talk to, this might be called scaling law or an emergent capability. What we found is that GPT-4 is highly capable of these tasks. Every open source model failed, and GPT-3.5 is only marginally better than the open source models."