The AI Thread
- Thread starter Truthy
- Start date
Hygro
soundcloud.com/hygro/
They have to be or they won't be useful lol
Please be good!
Hygro
soundcloud.com/hygro/
thankfully you can see the weird AIness still but it's getting a lot better fast.
Narz
keeping it real
There's like 5 different movies about AI nannies/housekeepers (spoiler alert they all seem to turn evil and/or seduce the husband).
Maybe someday 'AI' can make our films more creative but I wouldn't hold breath.
Maybe someday 'AI' can make our films more creative but I wouldn't hold breath.
PPQ_Purple
Purple Cube (retired)
- Joined
- Oct 11, 2008
- Messages
- 5,764
Our creations rising up to destroy us is a trope as old as time. Frankensteins monster being a good example. So its only logical that we would project the same onto AI.
I for one I feel that the end of the world is more likely to be like Soilent Green than the Matrix.
I for one I feel that the end of the world is more likely to be like Soilent Green than the Matrix.
Last edited:
LLMs produce racist output when prompted in African American English
Hundreds of millions of people now interact with language models, with uses ranging from help with writing to informing hiring decisions. However, these language models are known to perpetuate systematic racial prejudices, making their judgements biased in problematic ways about groups such as African Americans. Although previous research has focused on overt racism in language models, social scientists have argued that racism with a more subtle character has developed over time, particularly in the United States after the civil rights movement. It is unknown whether this covert racism manifests in language models. Here, we demonstrate that language models embody covert racism in the form of dialect prejudice, exhibiting raciolinguistic stereotypes about speakers of African American English (AAE) that are more negative than any human stereotypes about African Americans ever experimentally recorded. By contrast, the language models’ overt stereotypes about African Americans are more positive. Dialect prejudice has the potential for harmful consequences: language models are more likely to suggest that speakers of AAE be assigned less-prestigious jobs, be convicted of crimes and be sentenced to death. Finally, we show that current practices of alleviating racial bias in language models, such as human preference alignment, exacerbate the discrepancy between covert and overt stereotypes, by superficially obscuring the racism that language models maintain on a deeper level. Our findings have far-reaching implications for the fair and safe use of language technology.
Hundreds of millions of people now interact with language models, with uses ranging from help with writing to informing hiring decisions. However, these language models are known to perpetuate systematic racial prejudices, making their judgements biased in problematic ways about groups such as African Americans. Although previous research has focused on overt racism in language models, social scientists have argued that racism with a more subtle character has developed over time, particularly in the United States after the civil rights movement. It is unknown whether this covert racism manifests in language models. Here, we demonstrate that language models embody covert racism in the form of dialect prejudice, exhibiting raciolinguistic stereotypes about speakers of African American English (AAE) that are more negative than any human stereotypes about African Americans ever experimentally recorded. By contrast, the language models’ overt stereotypes about African Americans are more positive. Dialect prejudice has the potential for harmful consequences: language models are more likely to suggest that speakers of AAE be assigned less-prestigious jobs, be convicted of crimes and be sentenced to death. Finally, we show that current practices of alleviating racial bias in language models, such as human preference alignment, exacerbate the discrepancy between covert and overt stereotypes, by superficially obscuring the racism that language models maintain on a deeper level. Our findings have far-reaching implications for the fair and safe use of language technology.
Spoiler Legend :
Spoiler Legend :
Spoiler Legend :
Hygro
soundcloud.com/hygro/
Even though the AAVE version has the subjunctive tense lacked in the dominant english version and people be saying it's "bad" English. /slice of gripe
Ordnael
Emperor
New AI model can hallucinate a game of 1993’s Doom in real time
New AI model can hallucinate a game of 1993’s Doom in real time
Dobos: “Why write rules for software by hand when AI can just think every pixel for you?”…
arstechnica.com
But can it hallucinate Doom?...Yes, yes it can!
Narz
keeping it real
Roy, coming to an arcade near you in ten years!New AI model can hallucinate a game of 1993’s Doom in real time
New AI model can hallucinate a game of 1993’s Doom in real time
Dobos: “Why write rules for software by hand when AI can just think every pixel for you?”…
But can it hallucinate Doom?...Yes, yes it can!
I have to say I do not see that this is an issue with AI, if you give any untrusted bit of code access to your passwords your security is already broken.
GPT apps fail to disclose data collection, study finds
Many of the GPT apps in OpenAI's GPT Store collect data and facilitate online tracking in violation of OpenAI policies, researchers claim.
Boffins from Washington University in St. Louis, Missouri, recently analyzed almost 120,000 GPTs and more than 2,500 Actions – embedded services – over a four-month period and found expansive data collection that's contrary to OpenAI's rules and often inadequately documented in privacy policies.
The researchers – Evin Jaff, Yuhao Wu, Ning Zhang, and Umar Iqbal – describe their findings in a paper titled "Data Exposure from LLM Apps: An In-depth Investigation of OpenAI's GPTs."
"Our measurements indicate that the disclosures for most of the collected data types are omitted in privacy policies, with only 5.8 percent of Actions clearly disclosing their data collection practices," the authors claim.
The data gathered includes sensitive information such as passwords. And the GPTs doing so often include Actions for ad tracking and analytics – a common source of privacy problems in the mobile app and web ecosystems.
The OpenAI Store, which opened officially in January, hosts GPTs, which are generative pre-trained transformer (GPT) models based on OpenAI's ChatGPT. Most of the three million or so GPTs in the store have been customized by third-party developers to perform some specific function like analyzing Excel data or writing code.
A small portion of GPTs (4.6 percent of the more than 3 million) implement Actions, which provide a way to translate the structured data of API services into the vernacular of a model that accepts and emits natural language. Actions "convert natural language text into the json schema required for an API call," as OpenAI puts it.
"App activity data consists of user generated data (e.g., conversation and keywords from conversation), preferences or setting for the Actions (e.g., preferences for sorting search results), and information about the platform and other apps (e.g., other actions embedded in a GPT). Personal information includes demographics data (e.g., Race and ethnicity), PII (e.g., email addresses), and even user passwords; web browsing history refers to the data related to websites visited by the user using GPTs."
At least 1 percent of GPTs studied collect passwords, the authors observe, though apparently as a matter of convenience (to enable easy login) rather than for malicious purposes.
However, the authors argue that even this non-adversarial capture of passwords raises the risk of compromise because these passwords may get incorporated into training data.
"We identified GPTs that captured user passwords," explained Wu. "We did not investigate whether they were abused or captured with an intent for abuse. Whether or not there is intentional abuse, plaintext passwords and API keys being captured like this are always major security risks.
"In the case of LLMs, plaintext passwords in conversation run the risk of being included in training data which could result in accidental leakage. Services on OpenAI that want to use accounts or similar mechanisms are allowed to use OAuth so that a user can connect an account, so we'd consider this at a minimum to be evasion/poor security practices on the developer's part."
It gets worse. According to the study, "since Actions execute in shared memory space in GPTs, they have unrestrained access to each other's data, which allows them to access it (and also potentially influence each other's execution."
Then there's the fact that Actions are embedded in multiple GPTs, which allow them – potentially – to collect data across multiple apps and share that data with other Actions. This is exactly the sort of data access that has undermined privacy for users of mobile and web apps.
GPT apps fail to disclose data collection, study finds
Many of the GPT apps in OpenAI's GPT Store collect data and facilitate online tracking in violation of OpenAI policies, researchers claim.
Boffins from Washington University in St. Louis, Missouri, recently analyzed almost 120,000 GPTs and more than 2,500 Actions – embedded services – over a four-month period and found expansive data collection that's contrary to OpenAI's rules and often inadequately documented in privacy policies.
The researchers – Evin Jaff, Yuhao Wu, Ning Zhang, and Umar Iqbal – describe their findings in a paper titled "Data Exposure from LLM Apps: An In-depth Investigation of OpenAI's GPTs."
"Our measurements indicate that the disclosures for most of the collected data types are omitted in privacy policies, with only 5.8 percent of Actions clearly disclosing their data collection practices," the authors claim.
The data gathered includes sensitive information such as passwords. And the GPTs doing so often include Actions for ad tracking and analytics – a common source of privacy problems in the mobile app and web ecosystems.
The OpenAI Store, which opened officially in January, hosts GPTs, which are generative pre-trained transformer (GPT) models based on OpenAI's ChatGPT. Most of the three million or so GPTs in the store have been customized by third-party developers to perform some specific function like analyzing Excel data or writing code.
A small portion of GPTs (4.6 percent of the more than 3 million) implement Actions, which provide a way to translate the structured data of API services into the vernacular of a model that accepts and emits natural language. Actions "convert natural language text into the json schema required for an API call," as OpenAI puts it.
"App activity data consists of user generated data (e.g., conversation and keywords from conversation), preferences or setting for the Actions (e.g., preferences for sorting search results), and information about the platform and other apps (e.g., other actions embedded in a GPT). Personal information includes demographics data (e.g., Race and ethnicity), PII (e.g., email addresses), and even user passwords; web browsing history refers to the data related to websites visited by the user using GPTs."
At least 1 percent of GPTs studied collect passwords, the authors observe, though apparently as a matter of convenience (to enable easy login) rather than for malicious purposes.
However, the authors argue that even this non-adversarial capture of passwords raises the risk of compromise because these passwords may get incorporated into training data.
"We identified GPTs that captured user passwords," explained Wu. "We did not investigate whether they were abused or captured with an intent for abuse. Whether or not there is intentional abuse, plaintext passwords and API keys being captured like this are always major security risks.
"In the case of LLMs, plaintext passwords in conversation run the risk of being included in training data which could result in accidental leakage. Services on OpenAI that want to use accounts or similar mechanisms are allowed to use OAuth so that a user can connect an account, so we'd consider this at a minimum to be evasion/poor security practices on the developer's part."
It gets worse. According to the study, "since Actions execute in shared memory space in GPTs, they have unrestrained access to each other's data, which allows them to access it (and also potentially influence each other's execution."
Then there's the fact that Actions are embedded in multiple GPTs, which allow them – potentially – to collect data across multiple apps and share that data with other Actions. This is exactly the sort of data access that has undermined privacy for users of mobile and web apps.
I mostly agree, but there is an extra issue with AI: If personal data somehow gets incorporated during training it is almost impossible to detect in the model unless it randomly decides to spew it out. And even if it is detected, it would be hard to remove without a (possibly very expensive) re-train of the model.
It seems this is a core problem with these LLMs trained on the internet. The internet is full is accurate and inaccurate personal information. There is such a capacity for breach of so many laws, unless they are filtering it out then it is going to appear, and that is illegal in a lot of situations.
PPQ_Purple
Purple Cube (retired)
- Joined
- Oct 11, 2008
- Messages
- 5,764
That is not really an issue though if you just limit your learning data to things sourced responsibly. As in pick the sites you let your AI read responsibly and not just give it google and let it loose on the internet without any restraint or control.
And it's not hard either. You just need to make a curated list of sites that you let the AI go to and block anything outside of that. There are literally browser plugins that can do that so the logic is not that hard.
Last edited:
AI is coming for my jerb
Researchers built an ‘AI Scientist’ — what can it do?
Could science be fully automated? A team of machine-learning researchers has now tried.
‘AI Scientist’, created by a team at Tokyo company Sakana AI and at academic labs in Canada and the United Kingdom, performs the full cycle of research from reading the existing literature on a problem and formulating hypothesis for new developments to trying out solutions and writing a paper. AI Scientist even does some of the job of peer reviewers and evaluates its own results.
AI Scientist joins a slew of efforts to create AI agents that have automated at least parts of the scientific process. “To my knowledge, no one has yet done the total scientific community, all in one system,” says AI Scientist co-creator Cong Lu, a machine-learning researcher at the University of British Columbia in Vancouver, Canada. The results1 were posted on the arXiv preprint server this month.
“It’s impressive that they’ve done this end-to-end,” says Jevin West, a computational social scientist at the University of Washington in Seattle. “And I think we should be playing around with these ideas, because there could be potential for helping science.”
The output is not earth-shattering so far, and the system can only do research in the field of machine learning itself. In particular, AI Scientist is lacking what most scientists would consider the crucial part of doing science — the ability to do laboratory work. “There’s still a lot of work to go from AI that makes a hypothesis to implementing that in a robot scientist,” says Gerbrand Ceder, a materials scientist at Lawrence Berkeley National Laboratory and the University of California, Berkeley. Still, Ceder adds, “If you look into the future, I have zero doubt in mind that this is where much of science will go.”
Automated experiments
AI Scientist is based on a large language model (LLM). Using a paper that describes a machine learning algorithm as template, it starts from searching the literature for similar work. The team then employed the technique called evolutionary computation, which is inspired by the mutations and natural selection of Darwinian evolution. It proceeds in steps, applying small, random changes to an algorithm and selecting the ones that provide an improvement in efficiency.
To do so, AI Scientist conducts its own ‘experiments’ by running the algorithms and measuring how they perform. At the end, it produces a paper, and evaluates it in a sort of automated peer review. After ‘augmenting the literature’ this way, the algorithm can then start the cycle again, building on its own results.
The authors admit that the papers AI Scientists produced contained only incremental developments. Some other researchers were scathing in their comments on social media. “As an editor of a journal, I would likely desk-reject them. As a reviewer, I would reject them,” said one commenter on the website Hacker News.
West also says that the authors took a reductive view of how researchers learn about the current state of their field. A lot of what they know comes from other forms of communication, such as going to conferences or chatting to colleagues at the water cooler. “Science is more than a pile of papers,” says West. “You can have a 5-minute conversation that will be better than a 5-hour study of the literature.”
West’s colleague Shahan Memon agrees — but both West and Memon praise the authors for having made their code and results fully open. This has enabled them to analyze the AI Scientist’s results. They’ve found, for example, that it has a “popularity bias” in the choice of earlier papers it lists as references, skirting towards those with high citation counts. Memon and West say they are also looking into measuring whether AI Scientist’s choices were the most relevant ones.
Repetitive tasks
AI Scientist is, of course, not the first attempt at automating at least various parts of the job of a researcher: the dream of automating scientific discovery is as old as artificial intelligence itself — dating back to the 1950s, says Tom Hope, a computer scientist at the Allen Institute for AI based in Jerusalem. Already a decade ago, for example, the Automatic Statistician2 was able to analyse sets of data and write up its own papers. And Ceder and his colleagues have even automated some bench work: the ‘robot chemist’ they unveiled last year can synthesize new materials and experiment with them3.
Hope says that current LLMs “are not able to formulate novel and useful scientific directions beyond basic superficial combinations of buzzwords”. Still, Ceder says that even if AI won’t able to do the more creative part of the work any time soon, it could still automate a lot of the more repetitive aspects of research. “At the low level, you’re trying to analyse what something is, how something responds. That’s not the creative part of science, but it’s 90% of what we do.” Lu says he got a similar feedback from a lot of other researchers, too. “People will say, I have 100 ideas that I don’t have time for. Get the AI Scientist to do those.”
Lu says that to broaden AI Scientist’s capabilities — even to abstract fields beyond machine learning, such as pure mathematics — it might need to include other techniques beyond language models. Recent results on solving maths problems by Google Deep Mind, for example, have shown the power of combining LLMs with techniques of ‘symbolic’ AI, which build logical rules into a system rather than merely relying on it learning from statistical patterns in data. But the current iteration is but a start, he says. “We really believe this is the GPT-1 of AI science,” he says, referring to an early large language model by OpenAI in San Francisco, California.
The results feed into a debate that is at the top of many researchers’ concerns these days, says West. “All my colleagues in different sciences are trying to figure out, where does AI fit in in what we do? It does force us to think what is science in the twenty-first century — what it could be, what it is, what it is not,” he says.
Researchers built an ‘AI Scientist’ — what can it do?
Could science be fully automated? A team of machine-learning researchers has now tried.
‘AI Scientist’, created by a team at Tokyo company Sakana AI and at academic labs in Canada and the United Kingdom, performs the full cycle of research from reading the existing literature on a problem and formulating hypothesis for new developments to trying out solutions and writing a paper. AI Scientist even does some of the job of peer reviewers and evaluates its own results.
AI Scientist joins a slew of efforts to create AI agents that have automated at least parts of the scientific process. “To my knowledge, no one has yet done the total scientific community, all in one system,” says AI Scientist co-creator Cong Lu, a machine-learning researcher at the University of British Columbia in Vancouver, Canada. The results1 were posted on the arXiv preprint server this month.
“It’s impressive that they’ve done this end-to-end,” says Jevin West, a computational social scientist at the University of Washington in Seattle. “And I think we should be playing around with these ideas, because there could be potential for helping science.”
The output is not earth-shattering so far, and the system can only do research in the field of machine learning itself. In particular, AI Scientist is lacking what most scientists would consider the crucial part of doing science — the ability to do laboratory work. “There’s still a lot of work to go from AI that makes a hypothesis to implementing that in a robot scientist,” says Gerbrand Ceder, a materials scientist at Lawrence Berkeley National Laboratory and the University of California, Berkeley. Still, Ceder adds, “If you look into the future, I have zero doubt in mind that this is where much of science will go.”
Automated experiments
AI Scientist is based on a large language model (LLM). Using a paper that describes a machine learning algorithm as template, it starts from searching the literature for similar work. The team then employed the technique called evolutionary computation, which is inspired by the mutations and natural selection of Darwinian evolution. It proceeds in steps, applying small, random changes to an algorithm and selecting the ones that provide an improvement in efficiency.
To do so, AI Scientist conducts its own ‘experiments’ by running the algorithms and measuring how they perform. At the end, it produces a paper, and evaluates it in a sort of automated peer review. After ‘augmenting the literature’ this way, the algorithm can then start the cycle again, building on its own results.
The authors admit that the papers AI Scientists produced contained only incremental developments. Some other researchers were scathing in their comments on social media. “As an editor of a journal, I would likely desk-reject them. As a reviewer, I would reject them,” said one commenter on the website Hacker News.
West also says that the authors took a reductive view of how researchers learn about the current state of their field. A lot of what they know comes from other forms of communication, such as going to conferences or chatting to colleagues at the water cooler. “Science is more than a pile of papers,” says West. “You can have a 5-minute conversation that will be better than a 5-hour study of the literature.”
West’s colleague Shahan Memon agrees — but both West and Memon praise the authors for having made their code and results fully open. This has enabled them to analyze the AI Scientist’s results. They’ve found, for example, that it has a “popularity bias” in the choice of earlier papers it lists as references, skirting towards those with high citation counts. Memon and West say they are also looking into measuring whether AI Scientist’s choices were the most relevant ones.
Repetitive tasks
AI Scientist is, of course, not the first attempt at automating at least various parts of the job of a researcher: the dream of automating scientific discovery is as old as artificial intelligence itself — dating back to the 1950s, says Tom Hope, a computer scientist at the Allen Institute for AI based in Jerusalem. Already a decade ago, for example, the Automatic Statistician2 was able to analyse sets of data and write up its own papers. And Ceder and his colleagues have even automated some bench work: the ‘robot chemist’ they unveiled last year can synthesize new materials and experiment with them3.
Hope says that current LLMs “are not able to formulate novel and useful scientific directions beyond basic superficial combinations of buzzwords”. Still, Ceder says that even if AI won’t able to do the more creative part of the work any time soon, it could still automate a lot of the more repetitive aspects of research. “At the low level, you’re trying to analyse what something is, how something responds. That’s not the creative part of science, but it’s 90% of what we do.” Lu says he got a similar feedback from a lot of other researchers, too. “People will say, I have 100 ideas that I don’t have time for. Get the AI Scientist to do those.”
Lu says that to broaden AI Scientist’s capabilities — even to abstract fields beyond machine learning, such as pure mathematics — it might need to include other techniques beyond language models. Recent results on solving maths problems by Google Deep Mind, for example, have shown the power of combining LLMs with techniques of ‘symbolic’ AI, which build logical rules into a system rather than merely relying on it learning from statistical patterns in data. But the current iteration is but a start, he says. “We really believe this is the GPT-1 of AI science,” he says, referring to an early large language model by OpenAI in San Francisco, California.
The results feed into a debate that is at the top of many researchers’ concerns these days, says West. “All my colleagues in different sciences are trying to figure out, where does AI fit in in what we do? It does force us to think what is science in the twenty-first century — what it could be, what it is, what it is not,” he says.
Hygro
soundcloud.com/hygro/
Can't do smack as it's own entity. I have this gnarly data project at work rn, and I'm building it with a ton of help obviously with chatgpt but like, I asked for it to give me a function to calculate average sales by month and gives me total sales for each month, instead of 12 averages I have 80 totals, and then it confidently declares it averages, names the function averages.
It's just speed-up tool for the human combined with a brainstorm tool. That's incredible enough.
Now with a lot of upfront work, you can get a collection of agents with roles to review each other and criticize and stuff, and they get a lot more autonomous. I bet they did that above, and going deeper into the explanation like that was ignored by the editors.
It's just speed-up tool for the human combined with a brainstorm tool. That's incredible enough.
Now with a lot of upfront work, you can get a collection of agents with roles to review each other and criticize and stuff, and they get a lot more autonomous. I bet they did that above, and going deeper into the explanation like that was ignored by the editors.
Manfred Belheim
Moaner Lisa
- Joined
- Sep 11, 2009
- Messages
- 8,633
I mean that's cool and everything, but why use such language as "hallucinate" to describe it? I suppose the answer is "clickbait", but still.New AI model can hallucinate a game of 1993’s Doom in real time
New AI model can hallucinate a game of 1993’s Doom in real time
Dobos: “Why write rules for software by hand when AI can just think every pixel for you?”…
But can it hallucinate Doom?...Yes, yes it can!
Ordnael
Emperor
The way I understood the article is that the AI didn't use any factual game code. The emulation is based on the AI watching the game being played for very little time an then it replicates what it saw... I think the word hallucinate applies. I remember when I was a kid playing doom 2 through the night and making it farther then I've ever been. Then I went to bed for a very restless sleep is my mind kept on replaying the game. This still happens sometimes when I play till too late, I'd call it an hallucination.
Joij21
🔥Deny, Defend, Depose!🔥
Or a dream. But they don't use that term.
Hallucination is a more negative word to describe this phenomenon. Like it's on drugs or something.
So I think if anything the terminology surrounding AI is already leaning on the negative, with the public as well as experts not trusting it, so they use terms with a more negative leaning connotation to describe mundane technical phenomenon.
And yet despite this negative bias even among the experts, they still want to develop it further. Probably because of some existential schizo paranoia that their rival will develop AGI first and use it against them.
So what this means is that negative AI bias is actually more useful in determining that A.) the elites have many enemies since they show signs of paranoia. B.) that if the elites have this many enemies among themselves than they can't be very good people and therefore should not be trusted by the proletariat C.) one of these enemies of the elite could be the proletariat itself and they are worried that AIs current open source development up until this point could give them a weapon to be used against them potentially destroying the fragile bourgeoisie order
Hallucination is a more negative word to describe this phenomenon. Like it's on drugs or something.
So I think if anything the terminology surrounding AI is already leaning on the negative, with the public as well as experts not trusting it, so they use terms with a more negative leaning connotation to describe mundane technical phenomenon.
And yet despite this negative bias even among the experts, they still want to develop it further. Probably because of some existential schizo paranoia that their rival will develop AGI first and use it against them.
So what this means is that negative AI bias is actually more useful in determining that A.) the elites have many enemies since they show signs of paranoia. B.) that if the elites have this many enemies among themselves than they can't be very good people and therefore should not be trusted by the proletariat C.) one of these enemies of the elite could be the proletariat itself and they are worried that AIs current open source development up until this point could give them a weapon to be used against them potentially destroying the fragile bourgeoisie order
Hallucination is pretty standard term in AI/ML
Hallucination (artificial intelligence) - Wikipedia
Though in this headline it's indeed out of place. Only improperly generated parts of video can be called hallucinations, but not the whole process of generation.
