The AI Thread

[Thorgalaeg]

Wow, i managed to make a large language models up to 70B paramaters as LLama 2 70B, which is roughly equivalent to ChatGPT 3.5, to work in my PC fairly well. (here ´large´means really large, in fact it takes +50GB in my hard disk).

Llama.cpp is magic. It uses a combination of RAM and VRAM, the more VRAM the faster logically. In my system with 3900ti+1080ti it is faster than chatgpt in a middly bussy hour. But 70B is huge, obviously smaller models work faster. Thanks to llama.cpp anybody can run a Llama-based model in his computer, even without a decent graphic car. It is amazing to think a few months ago only running any LLM at home was unthinkable, lets no say one the size of ChatGPT.

 

[Samson]

Can we design our way out the environmental impact of deep learning? Digital computer have been king for a while, perhaps analog computers will be the way we replicate the analog computer we keep between our ears,

An analog-AI chip for energy-efficient speech recognition and transcription

Models of artificial intelligence (AI) that have billions of parameters can achieve high accuracy across a range of tasks, but they exacerbate the poor energy efficiency of conventional general-purpose processors, such as graphics processing units or central processing units. Analog in-memory computing (analog-AI) can provide better energy efficiency by performing matrix–vector multiplications in parallel on ‘memory tiles’. However, analog-AI has yet to demonstrate software-equivalent (SWeq) accuracy on models that require many such tiles and efficient communication of neural-network activations between the tiles. Here we present an analog-AI chip that combines 35 million phase-change memory devices across 34 tiles, massively parallel inter-tile communication and analog, low-power peripheral circuitry that can achieve up to 12.4 tera-operations per second per watt (TOPS/W) chip-sustained performance. We demonstrate fully end-to-end SWeq accuracy for a small keyword-spotting network and near-SWeq accuracy on the much larger MLPerf8 recurrent neural-network transducer (RNNT), with more than 45 million weights mapped onto more than 140 million phase-change memory devices across five chips.

Spoiler Legend :

a, Speech recognition has improved markedly over the past 10 years, driving down the WER for both the Librispeech and SwitchBoard (SWB) datasets, thanks to substantial increases in model size and improved networks, such as RNNT or transformer. For comparison with our results, the MLPerf RNNT full-precision WER is shown for two Librispeech datasets (‘test-clean’ and ‘dev-clean’)8, along with this work’s WER, which was computed on Librispeech dev-clean. For model size: B, 1 billion; M, 1 million. b, Inference models are trained using popular frameworks such as PyTorch or TensorFlow. Further optimization for analog AI can be achieved with the IBM analog HW acceleration kit (https://aihwkit.readthedocs.io/en/latest/). c, Trained model weights are then used on a 14-nm chip with 34 analog tiles, two processing elements (PE, not used for this work) and six ILP–OLP pairs. Tiles are labelled as north (N), centre (C) or south (S) followed by west (W) or east (E). d, Each ILP converts 512 8-bit inputs into 512 element vectors of pulse-modulated durations, which are then routed to the analog tiles for integration using a fully parallel 2D mesh that allows multi-casting to multiple tiles. After MAC, the charge on the peripheral capacitors is converted into durations4 and sent either to other tiles, leading to new MACs, or to the OLP, where durations are reconverted into 8-bit representations for off-chip data-processing. e, Transmission Electron Microscopy (TEM) image of one PCM. f, Each tile contains a crossbar array with 512 × 2,048 PCMs, programmed using a parallel row-wise algorithm4. g, PCMs can be organized in a 4-PCM-per-weight configuration, with G+, g+ adding and G−, g− subtracting charge from the peripheral capacitor, with a significance factor F (which is 1 in this paper). h, Alternatively, they can have a 2-PCM-per-weight configuration, which achieves a higher density. By reading different input frames through weights WP1 or WP2, a single tile can map 1,024 × 512 weight layers. i, Finally, two adjacent tiles can share their banks of 512 peripheral capacitors, enabling integration in the analog domain across 2,048 input rows.

 

[Samson]

If anyone here has opinions on AI and copyright and is american, then your government is actually asking you what they should do.

AGENCY: U.S. Copyright Office, Library of Congress.
ACTION: Notice of inquiry and request for comments.

SUMMARY: The United States Copyright Office is undertaking a study of the copyright law and policy issues raised by artificial intelligence (“AI”) systems. To inform the Office’s study and help assess whether legislative or regulatory steps in this area are warranted, the Office seeks comment on these issues, including those involved in the use of copyrighted works to train AI models, the appropriate levels of transparency and disclosure with respect to the use of copyrighted works, and the legal status of AI generated outputs.

DATES: Written comments are due no later than 11:59 p.m. Eastern Time on Wednesday, October 18, 2023. Written reply comments are due no later than 11:59 p.m. Eastern Time on Wednesday, November 15, 2023.

 

[Thorgalaeg]

https://twitter.com/x/status/1697204841813070261

Dark side of AI.

 

[Samson]

Did ChatGPT scrape LibGen?

That would screw any fair use argument they are making, you have to come by the original legally to claim fair use on making a copy.

Pulitzer Prize winning author Michael Chabon and others sue OpenAI

Pulitzer Prize winning US novelist Michael Chabon and several other writers have filed a proposed class action accusing OpenAI of copyright infringement for allegedly pulling their work into the datasets used to train the models behind ChatGPT.

The suit claims that OpenAI "cast a wide net across the internet" to capture the most comprehensive set of content available to better train its GPT models, allegedly "necessarily" leading it "to capture, download, and copy copyrighted written works, plays and articles."

One of the more interesting parts of the lawsuit is an allegation about how the authors believe the AI business got its hands on "two internet-based book corpora," which it notes that OpenAI simply refers to as "Books1" and "Books2." The filing alleges that in the July 2020 paper introducing GPT-3, "Language Models are Few-Shot Learners," OpenAI disclosed that in addition to "Common Crawl" and "WebText" web page datasets, "16 percent of the GPT3 training dataset came from... 'Books1' and 'Books2'."

The writers lawsuit goes on to allege that there are only a few places on the public internet that contain this much material, claiming that OpenAI's Books1 dataset "is based on either the Standardized Project Gutenberg Corpus or Project Gutenberg itself" and accusing the AI biz of sourcing Books2 from:

infamous "shadow library" websites, like Library Genesis ("LibGen"), Z-Library, Sci-Hub, and Bibliotik, which host massive collections of pirated books, research papers, and other text-based materials. The materials aggregated by these websites have also been available in bulk through torrent systems.​

 

[Hygro]

yarrr mateys I be pickin me side

 

[Samson]

There has been a lot of talk about AI leading to misinformation. I have been doubtful, as it does not look like any of the major disinformation threats are that short of content creation that AI would help. It seems to me that AI should be a solution to misinformation, as it will put powerful reference heckers at everyone's fingertips. We should all have browser plugins that validate what we read in the same way that the AI did for wikipedia in this case:

AI tidies up Wikipedia’s references — and boosts reliability

Wikipedia lives and dies by its references, the links to sources that back up information in the online encyclopaedia. But sometimes, those references are flawed — pointing to broken websites, erroneous information or non-reputable sources.

A study published on 19 October in Nature Machine Intelligence suggests that artificial intelligence (AI) can help to clean up inaccurate or incomplete reference lists in Wikipedia entries, improving their quality and reliability.

Fabio Petroni at London-based company Samaya AI and his colleagues developed a neural-network-powered system called SIDE, which analyses whether Wikipedia references support the claims they’re associated with, and suggests better alternatives for those that don’t.

“It might seem ironic to use AI to help with citations, given how ChatGPT notoriously botches and hallucinates citations. But it’s important to remember that there’s a lot more to AI language models than chatbots,” says Noah Giansiracusa, who studies AI at Bentley University in Waltham, Massachusetts.

When SIDE’s results were shown to a group of Wikipedia users, 21% preferred the citations found by the AI, 10% preferred the existing citations and 39% did not have a preference.

Writeup Paper

Spoiler Legend :

The decision flow of SIDE from a claim on Wikipedia to a suggestion for a new citation is as follows: (1) the claim is sent to the Sphere retrieval engine, which produces a list of potential candidate documents from the Sphere corpus; (2) the verification engine ranks the candidate documents and the original citation with respect to the claim; (3) if the original citation is not ranked above the candidate documents, then a new citation from the retrieved candidates is suggested. Note that the score of the verification engine can be indicative of a potential failed verification, as the one reported in the example.

 

[Farm Boy]

yarrr mateys I be pickin me side

Authors replaced by trillions of imaginary monkeys with imaginary typewriters. Such a brave new post human world.

 

[EvaDK]

We hold courses in mostly Adobe software at my work; the newly started AI introduction course (chatGPT, Leonardo, Canva, Firefly Adobe, newest versions of Photoshop & Illustrator) are constantly fully booked. There is definitely a demand out there and it's increasing. The AI assisted functionality I've played around with - mostly in Photoshop - is quite impressive. Until you notice that the AI generated George Clooney image has 6 fingers instead of 5 on one of his hands

 

[Hygro]

Authors replaced by trillions of imaginary monkeys with imaginary typewriters. Such a brave new post human world.

Yarr mateys I want me million monkeys sailing the high seas of accuracy plundering scientific booty

 

[Farm Boy]

Think of the poop on the decks.

 

[Hygro]

There’s going to be monkeys either way.

 

[Farm Boy]

You could push them off the deck. Not sure how long monkeys swim.

 

[Hygro]

I use AI. I want it to be trained on scientific papers. I do not want copyright to prohibit the training of AI.

I definitely don’t want copyright to prohibit my own citing of information which is taking that to its conclusion.

 

[Klaus Hergersheimer]

then you got this whole information superhighway thing

 

[Farm Boy]

I use AI. I want it to be trained on scientific papers. I do not want copyright to prohibit the training of AI.

I definitely don’t want copyright to prohibit my own citing of information which is taking that to its conclusion.

I am not so sure the interaction falls under understood patterns of quid pro quo. If it's parasitic, does being new and strong excuse that? Or maybe just make it a sort of imevitable?

 

[Hygro]

I am not so sure the interaction falls under understood patterns of quid pro quo. If it's parasitic, does being new and strong excuse that? Or maybe just make it a sort of imevitable?

I don’t know how you decide a jump off point. Are we better as hunter gatherers using sticks only? Or do we arbitrarily draw the technology line at present day and say “what happened before was progress/whatever, but more happens to be bad”? Or is there a way to filter?

I don’t find technology in general to be parasitical, and certainly not large language models aka the ai in question. I regard the internet as a good thing.

Nor do I think it can be stopped. But it can be altered, in so far as we can choose if openai has to pay out the nose for its models being able to answer scientific questions and therefore pass that cost on to us, or limit that value of these things.

 

[Lexicus]

Nor do I think it can be stopped. But it can be altered, in so far as we can choose if openai has to pay out the nose for its models being able to answer scientific questions and therefore pass that cost on to us, or limit that value of these things.

If OpenAI makes itself open-source and freely available then I'd say there's a moral argument to allow it to train on anything without regard to copyright. If OpenAI wants to sell access to its model to make profits then it can pay to use stuff like anyone else.

 

[Samson]

If OpenAI makes itself open-source and freely available then I'd say there's a moral argument to allow it to train on anything without regard to copyright. If OpenAI wants to sell access to its model to make profits then it can pay to use stuff like anyone else.

What is "like everyone else"? There are allegations of using pirated content, but generally it is a case of consuming internet content for free like everyone else.

There are also OS AI models.

 

[Kyriakos]

Ai videos are already taking off.