• Civilization 7 has been announced. For more info please check the forum here .

The AI Thread

Does this make any sense to anyone else?

Why do we need a fourth colour [Traffic Light] now?

Humans and autonomous cars use different sets of visual cues when it comes to interpreting lighting systems. Different colours – sometimes flashing to indicate that a change is imminent – work best for the human brain, while a single light works better for autonomous cars.​
Therefore, a fourth light – most likely white – would be added for the benefit of self-driving cars. The white light would be interpreted by a self-driven car as an instruction to “keep going unless instructed otherwise”.​
Hajbabaie, the NCSU professor, explained: “If the white light is active, you just follow the vehicle in front of you.”​

Of all the complexities that an autonomous cars has to deal with, how is a flashing light a complex input? How can they safely navigate a modern city but not understand traffic lights such that they have to be redesigned?
 
Does this make any sense to anyone else?

Why do we need a fourth colour [Traffic Light] now?

Humans and autonomous cars use different sets of visual cues when it comes to interpreting lighting systems. Different colours – sometimes flashing to indicate that a change is imminent – work best for the human brain, while a single light works better for autonomous cars.​
Therefore, a fourth light – most likely white – would be added for the benefit of self-driving cars. The white light would be interpreted by a self-driven car as an instruction to “keep going unless instructed otherwise”.​
Hajbabaie, the NCSU professor, explained: “If the white light is active, you just follow the vehicle in front of you.”​

Of all the complexities that an autonomous cars has to deal with, how is a flashing light a complex input? How can they safely navigate a modern city but not understand traffic lights such that they have to be redesigned?
Maybe it is (underwhelming) for the same reason all computers (dumb or dumber) can follow specific code. One wonders how the white in the light will be picked up so as to avoid other white lights being interpreted as the same.
 
Maybe it is (underwhelming) for the same reason all computers (dumb or dumber) can follow specific code. One wonders how the white in the light will be picked up so as to avoid other white lights being interpreted as the same.
A flashing light as a signal cannot be the most complicated bit about driving around a city.
 
A flashing light as a signal cannot be the most complicated bit about driving around a city.
Yes, but a white light can be roughly the least complicated, and apparently what the (=> not intelligent) system can handle ^^
I won't be surprised if it can't even handle that, tbh.
The "do as the car in front of you does" is itself potentially extremely dangerous as it is.
 
Honestly I do not understand why you would want to use a light for that sort of purpose. Not only is it going to be confusing to human drivers who now have to mentally deal with yet another thing but its just a completely stupid way to transmit information to vehicles as well. Having to worry about keeping the light clean, maintaining it and making sure the cars sensors don't get confused by other lights (potentially malicious ones as well) etc. It's just not worth it in my view.

If it were me I'd just use a wireless signal to transmit information to the vehicles computer directly. That would be invisible to people, far less intrusive to install, more secure and reliable and overall just better in every way. And on top of it all it would be much more future proof as the only thing you'd need to do in order to add functionality is update the software.
 
Of all the complexities that an autonomous cars has to deal with, how is a flashing light a complex input? How can they safely navigate a modern city but not understand traffic lights such that they have to be redesigned?

I suspect that the image recognition in those cars works on individual images. This has many advantage, but there is one disadvantage: You cannot identify a flashing light in a static image. So the image recognition will identify it as something else (with a different meaning) and then there needs to be some post-processing that takes multiple images and changes the identification to a flashing light. This is a solvable problem, of course, but you need to have specific code for that and cannot just throw more machine learning at it, like you could do for almost any other feature.


Honestly I do not understand why you would want to use a light for that sort of purpose. Not only is it going to be confusing to human drivers who now have to mentally deal with yet another thing but its just a completely stupid way to transmit information to vehicles as well. Having to worry about keeping the light clean, maintaining it and making sure the cars sensors don't get confused by other lights (potentially malicious ones as well) etc. It's just not worth it in my view.

If it were me I'd just use a wireless signal to transmit information to the vehicles computer directly. That would be invisible to people, far less intrusive to install, more secure and reliable and overall just better in every way. And on top of it all it would be much more future proof as the only thing you'd need to do in order to add functionality is update the software.

One does not simply update the software. The problem is that signal software and car software would need to communicate, so this means you would need to update the software of the signal and the software of every car model. And then you need to have a frequency range in the spectrum to transmit the signal and this might not the same in all countries and it would be a mess. A light signal would be much less complicated. Use NIR light, so drivers don't see it and they won't be confused.
 
I suspect that the image recognition in those cars works on individual images. This has many advantage, but there is one disadvantage: You cannot identify a flashing light in a static image. So the image recognition will identify it as something else (with a different meaning) and then there needs to be some post-processing that takes multiple images and changes the identification to a flashing light. This is a solvable problem, of course, but you need to have specific code for that and cannot just throw more machine learning at it, like you could do for almost any other feature.
It does not seem to be that rare to need some sense of time to really make sense of what is going on around a car. The most obvious one being the speed and direction of movement. The difference between interpreting a traffic light in the off phase of a flash and a child moving towards the road does not seem that big.
 
It does not seem to be that rare to need some sense of time to really make sense of what is going on around a car. The most obvious one being the speed and direction of movement. The difference between interpreting a traffic light in the off phase of a flash and a child moving towards the road does not seem that big.

Speed and direction of movement of surrounding objects is afaik typically found by radar and/or lidar and not through ML-based analysis of the camera frames.
 
Speed and direction of movement of surrounding objects is afaik typically found by radar and/or lidar and not through ML-based analysis of the camera frames.
I do not know anything about this, but it seems there must be a model above the level of detecting objects. I assume it has things like "That thing is to my left, is moving right and so may be in my path in the future.". Surely it also has things like "That is a traffic light, it is currently in state A but it may be in state B when I get to it". Having "That is a traffic light, it is currently in the off phase of a flash but it may not be in the future" does not seem that hard, surely there are harder problems that need some sense of time when it come to driving around a city. It is hard, people fail loads, and they can do stuff like that.
 
I do not know anything about this, but it seems there must be a model above the level of detecting objects. I assume it has things like "That thing is to my left, is moving right and so may be in my path in the future.". Surely it also has things like "That is a traffic light, it is currently in state A but it may be in state B when I get to it". Having "That is a traffic light, it is currently in the off phase of a flash but it may not be in the future" does not seem that hard, surely there are harder problems that need some sense of time when it come to driving around a city. It is hard, people fail loads, and they can do stuff like that.

Almost all of the time-related stuff the decision layer needs to deal with is movement. And for this, you don't actually need to look at the past, you just need vectors. Those vectors may have come from some sensor and that sensor needs to have a concept of time, but once you have the vectors, there is usually no need to look at the passing of time anymore. Flashing things are the exception, because they are time-related but have no movement. Detecting that is not a hard problem in itself, but it does not fit the model. And writing extensions to a model that was not supposed to handle that is always causing trouble
 
But an obstacle in the road (not having moved there while you were monitoring) isn't a vector either. Nor is a fallen body blocked by the car in front of you, and then that car moving to the side and leaving you with that body which you shouldn't run over just because it's not a vector.
 
But an obstacle in the road (not having moved there while you were monitoring) isn't a vector either. Nor is a fallen body blocked by the car in front of you, and then that car moving to the side and leaving you with that body which you shouldn't run over just because it's not a vector.

The zero-vector is also a vector. The point is that an unmoving obstacle does not have any time dependence.
 
I am not sure it matters, if it was so, the flashing light can be a zero-vector too, and then be on (say) a different plane when "off". My own point was that time-dependence is not at all the only problem with the AI in the car not identifying what it needs to so as to be safe.
 
Meta faces multiple complaints in Europe over plans to train AI on user data

Meta's plans to use customer data in AI training have resulted in complaints to data protection authorities in 11 European countries.

The complaints were filed by privacy activist group noyb following updates to Meta's privacy policy. The updates are due to take effect on June 26.

The main issue, according to noyb, are proposals by Meta to use years of posts – including images – "to develop and improve AI at Meta." Private messages between the user and friends and family are not used to train the corporation's AIs.

Rather than Meta asking for consent, users must opt out of the default data slurp. Once the data has been entered into the models, there appears to be no option to scrub it. The Register asked Meta how a user would go about removing their data, and the mega-biz has yet to respond to that point.
 
This sounds really big to me. It is getting really close to AI doing medicine.

A Multimodal Generative AI Copilot for Human Pathology

The field of computational pathology has witnessed remarkable progress in the development of both task-specific predictive models and task-agnostic self-supervised vision encoders. However, despite the explosive growth of generative artificial intelligence (AI), there has been limited study on building general purpose, multimodal AI assistants and copilots tailored to pathology. Here we present PathChat, a vision-language generalist AI assistant for human pathology. We build PathChat by adapting a foundational vision encoder for pathology, combining it with a pretrained large language model and finetuning the whole system on over 456,000 diverse visual language instructions consisting of 999,202 question-answer turns. We compare PathChat against several multimodal vision language AI assistants and GPT4V, which powers the commercially available multimodal general purpose AI assistant ChatGPT-4. PathChat achieved state-of-the-art performance on multiple-choice diagnostic questions from cases of diverse tissue origins and disease models. Furthermore, using open-ended questions and human expert evaluation, we found that overall PathChat produced more accurate and pathologist-preferable responses to diverse queries related to pathology. As an interactive and general vision-language AI Copilot that can flexibly handle both visual and natural language inputs, PathChat can potentially find impactful applications in pathology education, research, and human-in-the-loop clinical decision making.


Spoiler Legend :
Figure 1: Instruction-following dataset curation and PathChat overview. a. We curated the currently largest instruction finetuning dataset specialized for the domain of pathology, consisting of 456,916 instructions and corresponding responses covering various formats (e.g. multi-turn conversations, multiple-choice questions, short answers; see Extended Data Figure 1 for complete examples) from diverse sources. b. To build an MLLM-based vision language AI assistant that can reason over visual and natural language inputs, we begin with a SOTA vision-only self-supervised pretrained foundation encoder model, UNI, and perform further vision language pretraining analogous to CONCH. The resulting vision encoder is subsequently connected to a 13 billion parameter, pretrained Llama 2 LLM via a multimodal projector module (not shown) to form the complete MLLM architecture. The MLLM is finetuned via the curated instructionfollowing dataset to build PathChat, a visual language AI assistant specialized for human pathology. More details about data curation and model training can be found in PathChat dataset curation and PathChat model design and training section of Methods respectively. Scale bars are 200 μm
 
Many jobs consist basically on reading a lot of stuff and then applying the relevant parts to a given problem. AI can do that pretty well right now, or has the potential to do so in the near future. Medicine, at least diagnosis, is one of those jobs. Anyway a good medic able to feed the AI with good information about symptoms, analysis results and such is going to be necessary always.
 
One does not simply update the software. The problem is that signal software and car software would need to communicate, so this means you would need to update the software of the signal and the software of every car model. And then you need to have a frequency range in the spectrum to transmit the signal and this might not the same in all countries and it would be a mess. A light signal would be much less complicated. Use NIR light, so drivers don't see it and they won't be confused.
None of that is true.

Firstly you do not need a new frequency range for this thing. Just use the existing mobile telephony/internet infrastructure. The amount of data required to transmit bursts of information to individual vehicles such as broadcasting a red light or speed limit or similar information is less than even a text message. So cars can slot right in without issues. And you get the additional benefit of being able to piggyback off existing infrastructure.

As for updates again, just treat it as your phone and download any update over the internet. All modern cars have computers anyway and I refuse to believe a self driving vehicle won't have one. So if a new law passes tomorrow saying that all cars must accept some alteration to their software you can just have all phone companies send a mandatory update at midnight and force everyone to update and restart or else their car won't start. Simple as.

And yes, you will need bespoke hardware for this in the form of essentially a stripped down cheap mobile phone bolted onto the main computer. But compared to all the other costs of a self driving car that one is just trivial. And all the transmission infrastructure is already there in the form of your existing telephone network.
 
Firstly you do not need a new frequency range for this thing. Just use the existing mobile telephony/internet infrastructure. The amount of data required to transmit bursts of information to individual vehicles such as broadcasting a red light or speed limit or similar information is less than even a text message. So cars can slot right in without issues. And you get the additional benefit of being able to piggyback off existing infrastructure.
I am not convinced existing connection times are fast enough for this. Broadcasting a red light and reacting to it seems like the sort of thing that needs to connect to multiple nodes really fast, and I do not think the existing tech does that.

One also has to ask how one deals with bad actors on the network if everyone is going to transmitting safety critical information. It all sounds a nightmare to me, but perhaps it could be made to work.
 
None of that is true.

Firstly you do not need a new frequency range for this thing. Just use the existing mobile telephony/internet infrastructure. The amount of data required to transmit bursts of information to individual vehicles such as broadcasting a red light or speed limit or similar information is less than even a text message. So cars can slot right in without issues. And you get the additional benefit of being able to piggyback off existing infrastructure.
The mobile phone industry has been trying to sell this for years. So far they have not had much success with it. The problem is that the existing infrastructure is not really sufficient. The first problem is coverage. There are plenty of roads and intersections without adequate cell phone coverage and what would the cars do when they reach those? Sure, one could put a picocell at every intersection and the mobile phone industry would love to do that - if someone paid for it. Second problem is reliability. A typical cell is available like 99% of the time, which may sound high but means that it is down 3 days in a year. Third problem is latency which I'll address below.

Having dedicated vehicle-to-vehicle and vehicle-to-infrastructure communication system avoids most of these issues, so it would be way more preferrable in my opinion.

As for updates again, just treat it as your phone and download any update over the internet. All modern cars have computers anyway and I refuse to believe a self driving vehicle won't have one. So if a new law passes tomorrow saying that all cars must accept some alteration to their software you can just have all phone companies send a mandatory update at midnight and force everyone to update and restart or else their car won't start. Simple as.
Uhh, no it is not that simple. First, there will never be "an update". Any update needs to be compatible with the car software and each car manufacturer does their own thing. So you would need updates for every software variant and every version of those. Just look at how well (hint: not) Android versions get pushed to cell phones. And those have at least a common base software and a much shorter lifetime than cars.

Seconds, yes you could distribute it over the internet, maybe even a MPN of the car manufacturer (if they are still in business, that is). But for that to work, the car must be in coverage. It could be in a rural area or in undergound parking with no signal. And if the car would not start, you could not get it back into coverage.

And yes, you will need bespoke hardware for this in the form of essentially a stripped down cheap mobile phone bolted onto the main computer. But compared to all the other costs of a self driving car that one is just trivial. And all the transmission infrastructure is already there in the form of your existing telephone network.
That is the least part. Many modern cars already have that. Maintaining the cell phone service for those is a bigger part.

I am not convinced existing connection times are fast enough for this. Broadcasting a red light and reacting to it seems like the sort of thing that needs to connect to multiple nodes really fast, and I do not think the existing tech does that.
Depends a bit what you mean with "existing tech". The concepts and the technology certainly exists. 5G with edge computing is supposed to deliver latencies below 1 ms, which is certainly fast enough for car traffic. But you would need to roll this out on a large scale. Again, the mobile phone industry would love to build all of it (and have coloful powerpoints depicting all of that) - if someone paid them for it.

One also has to ask how one deals with bad actors on the network if everyone is going to transmitting safety critical information. It all sounds a nightmare to me, but perhaps it could be made to work.

Certainly possible to some degree - but only to some degree. No security system is ever perfect, so this is a question of how much risks we want to accept.
 
The mobile phone industry has been trying to sell this for years. So far they have not had much success with it. The problem is that the existing infrastructure is not really sufficient. The first problem is coverage. There are plenty of roads and intersections without adequate cell phone coverage and what would the cars do when they reach those? Sure, one could put a picocell at every intersection and the mobile phone industry would love to do that - if someone paid for it. Second problem is reliability. A typical cell is available like 99% of the time, which may sound high but means that it is down 3 days in a year. Third problem is latency which I'll address below.

Having dedicated vehicle-to-vehicle and vehicle-to-infrastructure communication system avoids most of these issues, so it would be way more preferrable in my opinion.
The thing you fail to realize is that any self driving vehicle is absolutely going to have to possess a mobile internet connection anyway, This is not negotiable.

The reason for it is navigation. A vehicle does not just have to dodge cones on a test track. It needs to be capable of knowing its current location and navigating to a destination. Otherwise its just a cool toy with no practical utility. To do this the vehicle has to posses a map of the general area it is in.

Now, as you rightly point out this map does have to be internal. It can't just be an app like google maps because you do want the car to be capable of functioning and not turn into a brick the moment the internet cuts out or goes out of range. But at the same time this map needs to be updated regularly to account for changing conditions on the roads. Streets change names, they get closed for road works or turned one way because some official wanted to make a mark. New roads are added and old ones closed. Etc. Etc.

You don't want to end up wasting three hours driving down the highway only to discover the exit point you need to take is closed for renovation and you need to take the long way around. And as a manufacturer if that happens to a whole column of your cars it's a good way to ruin your brand.

And while gross updates and map changes can be done periodically like at night the reality is that you really, really want as much of these updates as possible to be as close to real time as possible too because road conditions change in real time and not in convenient nightly builds. Pipes burst, crashes happen, streets get closed etc. That's the sort of thing you spot on google maps on your phone or hear announced on the radio as you drive. And as a human you know how to process that and get around it. A self driving car has to have the same capability or else it is just objectively worse and people won't transition to it.

So while you will not need a constant internet connection bottom line is that if you are giving your navigation over to a computer that computer is going to have to at the very least have the capability to match the real time monitoring aspects of a driver looking things up on his phone using google maps or listening to the radio.

And the reality is that the only practical way to keep it updated is via the internet. And in particular mobile internet.

Yes, you could develop a whole new set of standards and build a whole bespoke network of communications just for these things and than deploy them all over the planet everywhere and get everyone to standardize on a single world standard. But good luck with that. People can't even standardize what side of the road you drive on. And you absolutely need standardization because you don't want cars manufactured in France to be undrivable in Germany. So any proposal that is not at least market-zone wide is a non starter.

The internet meanwhile is an already established technology that has been proven and developed. It's a standard that is well known and accepted by everyone across the planet and the infrastructure is there as well. And when it comes to internet on the go there simply is no practical alternative to mobile. Regular home wireless won't do because most people in most cities park on the street some distance away from their homes and destinations and not in convenient reach of a wireless router. And even if they were, updating at night is only fine for software patches and gross map updates but not by the minute road changes.

And while yes, coverage is a problem it is going to remain a problem no matter what technology you use. And paying to expand mobile internet is far cheaper and more efficient than paying to duplicate all the coverage for your bespoke system AND than add more on top to cover everything. So there really is no economically viable alternative.

Finally, when it comes to reliability and coverage it is simply not as much of an issue as you think because of the nature of the requirements to update.

Put simply these things are only an important issue in areas where the coverage is not overlapping and duplicated. In other words, out in really small far away places like villages of 50 people or on the open highway. But these are precisely the areas where real time updates are going to be least important. New stretches of highways opening or parts being closed off for works is something that can and is announced a long time in advance. And there are not many traffic lights in a small village of 50 people or on the open highway to keep track of. So if there are no updates in a day that's not a big deal for these zones.

And conversely in a city where traffic is constantly shifting and something is always going on and you really need those by the minute updates in order for your car that you hear on the radio or see on your satnav or phone running google maps you also to match the navigation efficiency of a human driver you also have plenty of redundant overlapping phone coverage.

Uhh, no it is not that simple. First, there will never be "an update". Any update needs to be compatible with the car software and each car manufacturer does their own thing. So you would need updates for every software variant and every version of those. Just look at how well (hint: not) Android versions get pushed to cell phones. And those have at least a common base software and a much shorter lifetime than cars.
That is not the states problem though. I mean, it's not like any country is going to suddenly create a whole new set of traffic laws that radically change how driving works to the point that you require manufacturers to rework their entire software system. As I said, most updates are going to be of the traffic information kind because that is just the baseline minimum that the vehicle needs in order to be competitive to a human driver.

And for those you don't care what the manufacturers do internally as long as you get them to agree to standardize on the same JSON packet format for the data. Something that is very easy to do.

Seconds, yes you could distribute it over the internet, maybe even a MPN of the car manufacturer (if they are still in business, that is). But for that to work, the car must be in coverage. It could be in a rural area or in undergound parking with no signal. And if the car would not start, you could not get it back into coverage.
As I explained above, that is not really an issue as the vehicle is going to update as soon as it connects to a signal and areas with little or no coverage are also those with the least population density and thus the least chance of needing rapid continuous update.

So it all boils down to this.

You have a vehicle that has to maintain an internal map of the local area anyway in order to navigate. That vehicle also has to maintain near continuous internet service to receive traffic updates and other important information in order to compete with human drivers. And the lag time tolerance on these updates is conveniently such that it favors requiring more updates precisely in areas where coverage conditions are most favorable. So at that point why not just decorate the map with additional information about traffic signs lights and have updates be streamed to the car as well?

Sure, you will want an optical backup such that the vehicle can tell if a light is red even if the signal goes out. But that is going to be a backup. Something you use out in the village of 50 people on the 3 days of the year when their only cell tower goes out. And for the city of 3 million where every centimeter is covered by 5 competing companies it just becomes a box to tick in order to make the regulators happy.
 
Top Bottom