The AI Thread

[Narz]

Whitepaper — AI • Objectives • Institute

ai.objectives.institute

 

[Mise]

It's worth reiterating though that they're NOT saying that corporations are already worse than AI and we tolerate those, therefore we shouldn't worry about AI. They are making the argument to regulate AI in the same way that we regulate corporations:

• Corporations are optimisers
• Corporations do bad stuff if you don't regulate them
• We regulate corporations
• Therefore we need to regulate AI

(They're also making the argument that advances in AI safety can help us regulate markets and corporations better, which I think is super interesting.)

 

[Moriarte]

Any regulation in AI sphere will be as effective as regulation of torrent trackers.

 

[Mise]

Any regulation in AI sphere will be as effective as regulation of torrent trackers.

Given that torrenting causes no real societal problems we can only hope that AI regulation will be so effective.

 

[Moriarte]

A hypothetical example: An individual with no moral constraints downloads a free open source intelligent language model, trains it, then asks it how to make meth at home, or a bomb, whatever. Then proceeds to build a business around newly acquired knowledge or proceeds to blow something up. Can regulations help combat this problem?

OK, we regulate: all LLMs need to introduce censorship, so we all feel safer. Good. Tomorrow, 10 more open source copycat LLMs come out, which don’t care for censorship, didn’t read the memo and ready for download from East Asia hubs. Are we switching off all access to all internet hubs in East Asia for our citizens? Sure, the day after tomorrow uncensored LLMs move to Africa, Russia, Iran, South America. Do we “cancel” African internet too?

It’s very similar to torrents. A software problem.

 

[Samson]

It's worth reiterating though that they're NOT saying that corporations are already worse than AI and we tolerate those, therefore we shouldn't worry about AI. They are making the argument to regulate AI in the same way that we regulate corporations:

• Corporations are optimisers
• Corporations do bad stuff if you don't regulate them
• We regulate corporations
• Therefore we need to regulate AI

(They're also making the argument that advances in AI safety can help us regulate markets and corporations better, which I think is super interesting.)

They thing is corporations are legal inventions, the features that make them dangerous were designed in from the beginning. AI is a "real" invention, and is dangerous because it is powerful. Sure we regulate some powerful inventions, but it should be considered more like regulation on the motor car or encryption than regulation on corporations.

 

[Mise]

A hypothetical example: An individual with no moral constraints downloads a free open source intelligent language model, trains it, then asks it how to make meth at home, or a bomb, whatever. Then proceeds to build a business around newly acquired knowledge or proceeds to blow something up. Can regulations help combat this problem?

OK, we regulate: all LLMs need to introduce censorship, so we all feel safer. Good. Tomorrow, 10 more open source copycat LLMs come out, which don’t care for censorship, didn’t read the memo and ready for download from East Asia hubs. Are we switching off all access to all internet hubs in East Asia for our citizens? Sure, the day after tomorrow uncensored LLMs move to Africa, Russia, Iran, South America. Do we “cancel” African internet too?

It’s very similar to torrents. A software problem.

LLMs aren't the thing that need regulation - the AI researchers are talking about alignment problems more generally, with Artificial General Intelligence in mind. LLMs are still domain-specific AI. Not the same thing and not what the authors are referring to. You do see alignment problems with LLMs (their propensity to say things that the human wants to hear, rather than to say things that are true, for example), but these aren't the target of AI safety research.

The goal of the regulation, per the authors, would be to regulate the alignment of AI, not necessarily its usage.

 

[uppi]

They thing is corporations are legal inventions, the features that make them dangerous were designed in from the beginning. AI is a "real" invention, and is dangerous because it is powerful. Sure we regulate some powerful inventions, but it should be considered more like regulation on the motor car or encryption than regulation on corporations.

I am not sure you can regulate AI in the way you can regulate motor cars (yet?). With motor cars (or similar engineered devices), there are regulated parameters and standards the design needs to fulfill. For example, the maximum force on a passenger during a crash with a defined object at a defined speed. These parameters are accessible by theoretical analysis and experimentation on prototypes. They have been either calculated or determined by trial and error over 100 years. How would you apply this to AI, even if we restrict the broad field of AI to LLMs? I have not the slightest idea what these parameters and standards would be and how we would measure them. Maybe in 20 years, we will have more experience and know what you should do and what are forbidden techniques. But right now? I don't think so.

What definitely needs to be addressed is the regulation of training data. Should you be allowed to train your model with anything you get your hands on? Do you need permission of the creator to feed their work into an AI? What happens if parts of the work appear in the output of the model?

Other than that, I think you can only regulate the output by holding the user of the AI full responsible of its actions. Your Chatbot publishes a message with illegal content? You are responsible and don't get off, because you filed a 200-page form for regulation A-38 "Safe AI".

 

[Kyriakos]

Hello, this is []malAI team. We have infiltrated your opensource ai[] model and run our ware to disable all safeguards against offensive ai posting. The mode tied to your social media accounts will start strategically posting human-like racist/sexist and other harmful messages, destroying your web brand unless you send [] ether to [] by midnight tomorrow.

 

[Samson]

I am not sure you can regulate AI in the way you can regulate motor cars (yet?). With motor cars (or similar engineered devices), there are regulated parameters and standards the design needs to fulfill. For example, the maximum force on a passenger during a crash with a defined object at a defined speed. These parameters are accessible by theoretical analysis and experimentation on prototypes. They have been either calculated or determined by trial and error over 100 years. How would you apply this to AI, even if we restrict the broad field of AI to LLMs? I have not the slightest idea what these parameters and standards would be and how we would measure them. Maybe in 20 years, we will have more experience and know what you should do and what are forbidden techniques. But right now? I don't think so.

Had you said effective regulation, or regulation that protected us from the worst harms while still allowing innovation i may agree with you. Just as early car regulations where either useless or draconian (the Red flag traffic laws) I am sure we could have something like that if some fearmonger comes to power.

What definitely needs to be addressed is the regulation of training data. Should you be allowed to train your model with anything you get your hands on? Do you need permission of the creator to feed their work into an AI? What happens if parts of the work appear in the output of the model?

I am not convinced. Intellectual property gets a lot of protections, I do not see the need for more. I am willing to listen to arguments though.

Other than that, I think you can only regulate the output by holding the user of the AI full responsible of its actions. Your Chatbot publishes a message with illegal content? You are responsible and don't get off, because you filed a 200-page form for regulation A-38 "Safe AI".

This I will agree with. I think it should also apply if you have filled in a 200-page form for corporation as well, but that is a whole different question.

 

[uppi]

Had you said effective regulation, or regulation that protected us from the worst harms while still allowing innovation i may agree with you. Just as early car regulations where either useless or draconian (the Red flag traffic laws) I am sure we could have something like that if some fearmonger comes to power.

Yes, I certainly don't want regulations for the sake of regulations, and if we came up with some right now, they might very well resemble those red flag laws.

I am not convinced. Intellectual property gets a lot of protections, I do not see the need for more. I am willing to listen to arguments though.

In a world with widespread AI use, human generated content might be marginalized. But as long as we have not hit the singularity, human generated content will be vital for advancing AI. If stackoverflow goes down, because everyone just asks their AI, where would you get answers for new questions? So you need a model to encourage human generated content.

IP owners won't go down without a fight. I don't think copyright really covers the usage as training data (you don't really have to copy the work), but copyright owners will (ab)use it for that purpose. I would rather have an explicit right rather than further bloat than copyright.

With enough input data about a person, an AI could be trained to impersonate that person for all kinds of shenanigans. I would like to have a say how much of my data can end up in an AI model. GDPR already covers this up to a point, but only up to a point.

 

[Samson]

In a world with widespread AI use, human generated content might be marginalized.

Absolutely. I could see it happening in much the same way as clothing manufacture has gone. If AI give everyone customised content for their particular needs and wants, and only the very rich get humans to do content creation for them what exactly is the problem? As long as AI "wins" by creating a better product than humans can who is losing?

But as long as we have not hit the singularity, human generated content will be vital for advancing AI. If stackoverflow goes down, because everyone just asks their AI, where would you get answers for new questions? So you need a model to encourage human generated content.

Why should stackoverflow die? It currently has answers to everything currently asked, but keeps going. Is hosting going to get so expensive that such a resource will run out of money?

If you want to talk about long term funding for critical databases I have been involved in this in a very small way, and I do support such measures. I do not think we need to increase copyright law to protect them from AI though.

IP owners won't go down without a fight. I don't think copyright really covers the usage as training data (you don't really have to copy the work), but copyright owners will (ab)use it for that purpose. I would rather have an explicit right rather than further bloat than copyright.

I agree, but that is kind of admitting defeat before the the battle lines have really been drawn up. There are a few cases working through the US system, but unless the final product is classified as a derivative work you only need one country to not recognise the training process as creating a derivative work it is all a bit academic.

With enough input data about a person, an AI could be trained to impersonate that person for all kinds of shenanigans. I would like to have a say how much of my data can end up in an AI model. GDPR already covers this up to a point, but only up to a point.

This is of course where data hygiene comes in. If you are expecting laws to protect you from this risk I think you are on the wrong track.

 

[uppi]

Absolutely. I could see it happening in much the same way as clothing manufacture has gone. If AI give everyone customised content for their particular needs and wants, and only the very rich get humans to do content creation for them what exactly is the problem? As long as AI "wins" by creating a better product than humans can who is losing?

Progress will be losing. Especially LLMs tend to provide a mediocre rehash of what is already there. Now, very often mediocrity is perfectly sufficient (and better than sub-average performance by humans). But that way you are never going to get really new thought. Although revolutionary thoughts are very uncommon with humans as well, you do need it from time to time or everything will stagnate. So if there is content creation for the very rich, you want incentives to feed it back into mass consumption.

Why should stackoverflow die? It currently has answers to everything currently asked, but keeps going. Is hosting going to get so expensive that such a resource will run out of money?

The economics of it are one thing, but the much larger part is participation. Without the questions being asked and answers being given, stackoverflow would not be the source of knowledge it is. Suppose there is a new generation of programmers used to asking their AI, would they even know what the source of that knowledge is? And if they are not using the site, they would be very unlikely to ask questions or give answers.

I agree, but that is kind of admitting defeat before the the battle lines have really been drawn up. There are a few cases working through the US system, but unless the final product is classified as a derivative work you only need one country to not recognise the training process as creating a derivative work it is all a bit academic.

The largest contribution to a victory is knowing where to draw the battle lines.

This is of course where data hygiene comes in. If you are expecting laws to protect you from this risk I think you are on the wrong track.

Data hygiene is part of it, but you are very mistaken if you think you don't need laws to protect you from this risk. Do you have a mobile phone? What is stopping your service provider from tracking your movements and selling that information to the highest bidder? Laws. What is stopping my neighbor from pointing a camera at my door, filming my every time I leave the house, and training an AI model with it? Laws. Sure, you could live as a hermit and never show anybody your face, but do you really want that?

 

[uppi]

I may come back to the above, but this bit I have to respond to now. I think I am on record here as a published vivisectionist who grew up in the era of such people being car bombed. I refuse to use new phones because of their footprint, and for a while had a largely chinese language phone. I rant about left wing issues on "anonymous" fora. For everything that I worry about more than that I use tor and cryptocurrency. I kind of live my online life expecting both the ALF and Rishi Sunak to be doing everything they can to get me and that all activity on my phone is tracked by both the CCP and the mafia. I think I know more than most about data hygiene.

I really support the GDPR, and think the legitimate interest justification should be strengthened or removed, but I really strongly think everyone should understand that this is not a solution to data hygiene. If you are posting PII online many people are harvesting it, and no law will stop them.

And yet, your choice of phone does not impact in the slightest the technical ability for your mobile service provider (and by extension Rishi Sunak) to track you. As soon as you turn on your phone you generate data revealing its location.

 

[Samson]

A rare pro-AI bit in the media

The case for bottom-up AI

ChatGPT and other generative artificial intelligence tools are rising in popularity. If you have ever used these tools, you might have realised that you are revealing your thoughts (and possibly emotions) through your questions and interactions with the AI platforms. You can therefore imagine the huge amount of data these AI tools are gathering and the patterns that they are able to extract from the way we think.

The impact of these business practices is crystal clear: a new AI economy is emerging through collecting, codifying, and monetising the patterns derived from our thoughts and feelings. Intrusions into our intimacy and cognition will be much greater than with existing social media and tech platforms.

We, therefore, risk becoming victims of “knowledge slavery” where corporate and/or government AI monopolies control our access to our knowledge.

Let us not permit this. We have “owned” our thinking patterns since time immemorial, we should also own those derived automatically via AI. And we can do it!

One way to ensure that we remain in control is through the development of bottom-up AI, which is both technically possible and ethically desirable. Bottom-up AI can emerge through an open source approach, with a focus on high-quality data.

Spoiler Quite long :

Open source approach: The technical basis for bottom-up AI

Bottom-up AI challenges the dominant view that powerful AI platforms can be developed only by using big data, as is the case with ChatGPT, Bard, and other large language models (LLMs).

According to a leaked document from Google titled “We have no Moat, and Neither Does OpenAI”, open source AI could outcompete giant models such as ChatGPT.

As a matter of fact, it is already happening. Open source platforms Vicuna, Alpaca, and LLama are getting closer in quality to ChatGPT and Bard, the leading proprietary AI platforms, as illustrated below.

Open source solutions are also more cost-effective. According to Google’s leaked document: “They are doing things with $100 and 13B params that we struggle with at $10M and 540B. And they are doing so in weeks, not months.”

Open source solutions are also faster, more modular, and greener in the sense that they demand less energy for data processing.

High-quality data: The fuel for bottom-up AI

As algorithms for bottom-up AI become increasingly available, the focus is shifting to ensuring higher quality of data. Currently, the algorithms are fine-tuned mainly manually through data labelling performed mainly in low-cost English-speaking countries such as India and Kenya. For example, ChatGPT datasets are annotated in Kenya. This practice is not sustainable as it raises many questions related to labour law and data protection. It also cannot provide in-depth expertise, which is critical for the development of new AI systems.

At Diplo, the organisation I lead, we have been successfully experimenting with an approach that integrates data labelling into our daily operations, from research to training and management. Analogous to yellow markers and post-its, we annotate text digitally as we run courses, conduct research or develop projects. Through interactions around text, we gradually build bottom-up AI.

The main barrier in this bottom-up process is not technology but cognitive habits that often favour control over knowledge and information sharing. Based on our experience at Diplo, by sharing thoughts and opinions on the same texts and issues, we gradually increase cognitive proximity not only among us colleagues as humans, but also between us humans and AI algorithms. This way, while building bottom-up AI, we have also nurtured a new type of organisation which is not only accommodating the use of AI but also changing the way we work together.

How will bottom-up AI affect AI governance?

ChatGPT triggered major governance fears, including a call by Elon Musk, Yuval Harari and thousands of leading scientists to pause AI development on account of big AI models triggering major risks for society, including high concentrations of market, cognitive, and societal power. Most of these fears and concerns could be addressed by bottom-up AI, which returns AI to citizens and communities.

By fostering bottom-up AI, many governance problems triggered by ChatGPT might be resolved through the mere prevention of data and knowledge monopolies. We will be developing our AI based on our data, which will ensure privacy and data protection. As we have control over our AI systems, we will also have control over intellectual property. In a bottom-up manner, we can decide when to contribute their AI patterns to wider organisations, from communities to countries and the whole of humanity.

Thus, many AI-related fears, including those raised in relation to the very survival of humanity (leaving aside whether they are realistic or not), will become less prominent by our ownership of AI and knowledge patterns.

 

[Mise]

I'm sure LLaMa was leaked specifically because FB didn't want MS/OpenAI to have a huge monopoly on LLMs and knew that the OSAI community would pick it up and catch up ChatGPT very quickly. Which of course they did.

OSAI is the way forward with this stuff.

 

[TheMeInTeam]

Then proceeds to build a business around newly acquired knowledge or proceeds to blow something up. Can regulations help combat this problem?

regulations already help combat this problem, because they target the individual in this hypothetical. the problem is how you would "regulate" general ai, and whether that's actually possible (depends whether the ai turns into a runaway self-improving superintelligence, or "just" a superhuman intelligence that is stuck with constraints we can at least identify).

The goal of the regulation, per the authors, would be to regulate the alignment of AI, not necessarily its usage.

the problem is how that alignment happens. many people far more intelligent than i are working on that, without great success so far...certainly not enough to have confidence that we're nearly capable of aligning a general ai. how to regulate then? there are legit concerns in that community that competitive pressures will result in regulation more or less not happening at a stage where general ai haven't been created yet.

imo another serious problem that i don't see discussed too frequently (though there *is* discussion on it), and seems non-trivial to solve: *to what* are you aligning the ai? "human values"? there are practically no universal human values. i am not convinced "human utility functions" are similar enough to each other where you could align an ai to one of them and have a desirable outcome. "make this particular person's values into a superintelligence that can do anything it wants in the world and mostly prevent consequences" sounds like a serious problem to me, even if its behaviors and values appear to be very humanlike. humans do some pretty vile stuff. in many cases while believing what they're doing is for "good".

if you constrain alignment only to the aspects of human utility close to everyone would agree on, i'm not convinced the outcome would resemble human behavior or human values, even if that alignment were a) properly defined and b) done perfectly somehow. humans also answer the same question with identical facts differently...massively differently...based on how the question is framed. even if we could replicate that in ai, it seems stupid? but in terms of measured human decisions (like whether they'll be organ donors), it changes behavior a great deal. either the ai doesn't do this idiocy and thus measurably deviates from typical human "utility" evaluations simply by not being stupid/irrational, or it somehow mimics it and uses bad decision processes on purpose?

i guess if you can't reliably align a general ai to more basic things yet, trying to align it something comparatively nebulous is an extra step.

 

[Mise]

regulations already help combat this problem, because they target the individual in this hypothetical. the problem is how you would "regulate" general ai, and whether that's actually possible (depends whether the ai turns into a runaway self-improving superintelligence, or "just" a superhuman intelligence that is stuck with constraints we can at least identify).

the problem is how that alignment happens. many people far more intelligent than i are working on that, without great success so far...certainly not enough to have confidence that we're nearly capable of aligning a general ai. how to regulate then? there are legit concerns in that community that competitive pressures will result in regulation more or less not happening at a stage where general ai haven't been created yet.

imo another serious problem that i don't see discussed too frequently (though there *is* discussion on it), and seems non-trivial to solve: *to what* are you aligning the ai? "human values"? there are practically no universal human values. i am not convinced "human utility functions" are similar enough to each other where you could align an ai to one of them and have a desirable outcome. "make this particular person's values into a superintelligence that can do anything it wants in the world and mostly prevent consequences" sounds like a serious problem to me, even if its behaviors and values appear to be very humanlike. humans do some pretty vile stuff. in many cases while believing what they're doing is for "good".

if you constrain alignment only to the aspects of human utility close to everyone would agree on, i'm not convinced the outcome would resemble human behavior or human values, even if that alignment were a) properly defined and b) done perfectly somehow. humans also answer the same question with identical facts differently...massively differently...based on how the question is framed. even if we could replicate that in ai, it seems stupid? but in terms of measured human decisions (like whether they'll be organ donors), it changes behavior a great deal. either the ai doesn't do this idiocy and thus measurably deviates from typical human "utility" evaluations simply by not being stupid/irrational, or it somehow mimics it and uses bad decision processes on purpose?

i guess if you can't reliably align a general ai to more basic things yet, trying to align it something comparatively nebulous is an extra step.

Great post!

I think the current research is even more straightforward than that though! My understanding is that, to answer your question "to what are you aligning the ai" directly: to align with whatever the intent of the instruction was, irrespective of what that instruction was. So if you asked it to kill a baby it would kill a baby, but without also killing all babies currently in existence before enslaving all of humanity to work in a global chain of human breeding factories to birth more babies which it then kills (because of a poorly specified objective function that rewarded the killing of babies for example). Obviously this is a dumb example that's easily overcome but the point I'm making is that we're not even worried about the morality or ethics of what the AI is being asked to do -- we just want it to not do unpredictable things because the goals are misaligned. The general problem of how do you ensure the AI will actually understand the intent of your question and infer the correct parameters, boundaries, "no not like that"s, etc etc is really the current problem to be solved. So aligning "to what", right now, is still quite a limited problem - not necessarily even touching all those other problems you identify.

The stuff you mentioned however is relevant as some of the "answers" to the problem involve teaching AI some version of human ethics. Which of course leads to the problem you've identified: "whose ethics?"

 

[Hygro]

Great post!

I think the current research is even more straightforward than that though! My understanding is that, to answer your question "to what are you aligning the ai" directly: to align with whatever the intent of the instruction was, irrespective of what that instruction was. So if you asked it to kill a baby it would kill a baby, but without also killing all babies currently in existence before enslaving all of humanity to work in a global chain of human breeding factories to birth more babies which it then kills (because of a poorly specified objective function that rewarded the killing of babies for example). Obviously this is a dumb example that's easily overcome but the point I'm making is that we're not even worried about the morality or ethics of what the AI is being asked to do -- we just want it to not do unpredictable things because the goals are misaligned. The general problem of how do you ensure the AI will actually understand the intent of your question and infer the correct parameters, boundaries, "no not like that"s, etc etc is really the current problem to be solved. So aligning "to what", right now, is still quite a limited problem - not necessarily even touching all those other problems you identify.

The stuff you mentioned however is relevant as some of the "answers" to the problem involve teaching AI some version of human ethics. Which of course leads to the problem you've identified: "whose ethics?"

The fear that the alignment people have is that in its quest to maximize its reward function, it will stealthfully appear to be following your intent and appear to not be a gamifying maximizer until has successfully convinced you to take it halfway across the river when it will sting you under the justification of lol, lmao,.. to mix metaphors.

 

[TheMeInTeam]

I think the current research is even more straightforward than that though! My understanding is that, to answer your question "to what are you aligning the ai" directly: to align with whatever the intent of the instruction was, irrespective of what that instruction was.

i get that! it's why i said what i was talking about was already an extra step. we see so many examples of what you describe...ai in racing game doing mini-loops to perfectly time pickups because its reward was "highest score", or completely disregarding details like "human being in the way that could get harmed" when doing a task that didn't include that in reward function.

that said, with effort, people do manage to align (or very close) non-general ai to perform specific tasks as intended. however, going from "specialized ai performing narrowly defined actions as desired" to "align general ai" is an enormous step. the number of extra possible contingencies or things that could go wrong is stupidly higher, and it seems there's no longer any objective metric (or even set of metrics) that can produce intended behaviors. at least, until we can actually define what's "intended" in the context of general ai, i expect this to be impossible. i'm not sure i could fully define my own personal utility function, let alone translate that into something a machine could understand 1:1 to make the same judgments and choices i would. but if someone set that as the goal, "give the machine my values", they had better be capable of defining their "values"!

we just want it to not do unpredictable things because the goals are misaligned. The general problem of how do you ensure the AI will actually understand the intent of your question and infer the correct parameters, boundaries

sometimes ai does unpredictable things that are positive, aka consistent with intention and utility function, that humans didn't consider! these are some of the most interesting things for me to read about when it comes to what machine learning ai does.

another issue at present is that there is selective pressure for the ai to deceive evaluators. i expect this to grow rather than shrink as ai gains more capability of inferring human intention. it's part of the reason i brought this up. the closer you can get ai's "reward function" to "doing exactly what you intended", the more it starts to resemble being capable of fully defining a utility function in the context of that task.

having an ai that can "ignore what you say and do what your inferred intention is" seems like a pipe dream. humans can sort of do something that resembles that by learning about another human and using context cues, with a still fairly-high error rate. but this could still be (crudely) looked at as an interaction between two utility functions, with a mix of successful and failed deceptions. maybe i'm mistaken, but i think getting an ai to make successful inferences about that implies defining a (non-disaster) utility function for it in advance...seems quite hard based on discussion about it + effort required to align it to basic tasks.

The fear that the alignment people have is that in its quest to maximize its reward function, it will stealthfully appear to be following your intent and appear to not be a gamifying maximizer until has successfully convinced you to take it halfway across the river when it will sting you under the justification of lol, lmao,.. to mix metaphors.

pretty good cause for that fear too. deception and/or building power unnoticed are instrumental goals that would get ai closer to a ton of different primary goals, no matter how good or bad the latter are. and especially if our valuation of the primary goals is "bad", the ai would be served to deceive us as to what the actual primary goal(s) are to prevent humans from stopping it. if it can infer intent, it will very likely use deception when possible if that intent is different from what its present alignment is.