"On October 28, 2021, after turning right onto Fremont Blvd from Cushing Pkwy, the Pony.ai Autonomous Vehicle ("Pony.ai AV") performed a left lane change maneuver in autonomous mode. While performing the lane change, the Pony.ai AV came into contact with a center divider on Fremont Blvd. and the traffic sign that was posted on the divider. The Pony.ai AV suffered moderate damage to the front of the vehicle and the undercarriage. There were no injuries and no other vehicles involved. Fremont Police Department were called to report the incident and the damaged street sign. Pony.ai has subsequently worked with local authorities to resolve all issues related to the damaged sign."
It's not a serious incident, but it is a fully autonomous vehicle under good conditions going somewhere it was absolutely not supposed to go. So DMV revoked their license for full autonomous operation without a safety driver. They have to go back to testing with a safety driver.
Could Tesla have put up adversarial triggers nearby their own facilities to test, and it overlapped on whatever opencv or vision package pony.ai is using?
Using invisible-to-humans paint on signs for adversarial recognition attacks on autonomous driving systems would also be an anti-Tesla tactic not entirely unreasonable to consider.
"Tesla autopilot crashes right outside Tesla factory!" would be a desired headline for a certain crowd.
It'll be interesting to see what caused this, at any rate, probably less than 1% chance the above is what happened.
This one gives a better idea of why this might have happened, in the direction of travel, the white back of the sign is easy to miss with a white-sided box truck behind it:
The pole at the end of the divider with the sign facing away, stretched-arrow street paint left-turn symbol, and the enormous expanse of intersection pavement are all a little disorienting.
It’s not a serious incident for a human, there are lots of explanations for such a thing which are innocent. It is a serious incident for a robot for which running into a street sign in the middle of the road is a “never” kind of problem and indicative of a significant issue.
And perhaps that's part of the problem. "Oh, this person crashed his heavy machine into something...it happens" seems to significantly undervalue the responsibility and risks of driving. It should really be a "never" problem for people, too.
FSD isn't even close to human capability yet so its a moot point. When FSD is as good as a human driver we can then think about quantifying how seriously to take its occasional malfunctions / mistakes.
“Do not drive into stationary objects” should be one of the more basic requirements for a self driving car, any company that messes that up makes me nervous about how they might handle more complicated situations.
That was part of spec of the traction/break of a vehicle I worked on.
Passagers life was prioritized as the premise behind the decision was : "Outside the vehicle is an unknown, we don't know if it's a human, an animal, an object ; we know that inside we have human life (at least the driver, at most the hundred of passengers), so we give priority to what we know)
That sounds bad. Yes, you know that there’s a person in the car, but you also know that if the object in front of you is a person then they are multiple orders of magnitude more likely to be killed than the occupants. You have to weigh the uncertainty and the disparity of outcomes.
Sounds like a continuation of the automotive trend that pretends like the only person whose life matters is the occupant.
>Why is Tesla “autopilot” still allowed on Californian roads?
Because Pony.ai is level 4 autonomy while Autopilot is level 2. The human driver is ultimately responsible for any accidents in a level 2 system. There is no driver to intercede or to blame in the case of a level 4 system. This crash is the sole responsibility of Pony.ai.
That particular spot had been crashed into before and the crash buffers hadn't been properly replaced. This led to a normal crash turning into a fatality.
The driver of the tesla had complained in the past of the tesla swerving into the wall when passing that spot, yet still had his guard down on the day he died.
That the tesla swerved is down to the poor striping on the road. The stripe along the right side of the left fork was brand new and very sharp, while the stripe along the left side of the right fork was extremely dim to non existant due to excessive wear sustained during construction of the left fork and the bridge it leads to (connection from the fast lane of 101s to the fast lane of 85s).
The crash barrier has since been properly replaced and the striping redone with diagonal lines hashing off the gore so it's abundantly clear even to a camera not to drive there.
That said, the tesla self driving made a very dumb error no human driver would make.
> That the tesla swerved is down to the poor striping on the road.
Certainly not. It's not appropriate for a self driving car to make any assumptions about striping or any road markings being correct. It crashed because it wasn't capable of self-driving after all.
The article goes on to say that two other drivers who said they had autopilot on also crashed into fire trucks… How many fire trucks do Teslas need to crash into for it to be considered a problem?
One could argue that normal cruise control on normal cars would also crash into fire trucks. Ultimately drivers are responsible in both cases, it's their fault 100% end of story.
PS Tesla rolled a software update to detect emergency vehicles and slow down.
The cruise control on my Subaru probably wouldn't hit a fire truck since it has pre-collision braking.. but in any case, it's not sold as "Autopilot" or "Full Self Driving" so the limitations are pretty clear. There's a reason several countries stopped Tesla from marketing their nonsense like they do in the US - it turns out when you say things are self driving, consumers believe you and don't read the fine print.
Most TACC systems won’t brake for objects going less than vehicle_speed-threshold, due to how radar works. You don’t get relative speeds and distance for a set of objects, you get a spectrum showing how much of what the radar sees is moving at each relative speed.
That’s also why most TACC doesn’t work below 30km/h or so. It can’t tell the difference between the car in front and a stationary object next to the road.
Edit: This is actually an interesting rabbit hole to go down, because on the face of it, it seems like a fairly straightforward system to implement. Then the more you think about how a 'simple' system should handle various scenarios, the more you realise that to actually do it properly, 'just keep a good distance from the car in front' requires most of the perception required for general free-form driving.
This isn't true (on Subaru at least, likely due to their binocular eyesight cameras...) The primary purpose of their suite of driver aides is to prevent accidents, so it's actually really good at that. There's plenty of videos to attest to that but here's an example of a 60kph stop when the system detects a cone:
I guess you did say braking with TACC is complicated -- but I think that's a bit of an edge case that only applies to Tesla. The Subaru version of TACC is ~fine but it never gives you the impression it will handle 100% of the driving for you, so you'd never leave it to navigate around emergency vehicles by itself. Maybe the solution to rarely dangerous self-driving is to make them only ~75% good so driver's are always aware?
That's pretty cool! Much better than the radar based TACC on the Hilux I rented a couple of weeks ago. I came away from that one with a strong impression that it was inconsistent in a scary way and shouldn't have shipped like that.
Do you have one (or experience driving one)? If so:
- How does it go with phantom braking and various objects on the side of the road?
- Does it distinguish between various on-road things? (eg. does it brake for a dog? a plastic bag? a puff of smoke? an anvil falling off the back of the truck in front?)
- How good's its understanding of lanes (eg. a very slow car that's in another lane but currently directly in front of you due to the road curving)?
- Does it brake for objects outside the lane entering it (cross traffic, lane cut-ins, pedestrians, vegetation moving around in high winds)?
It is important to note that suddenly, and against all probability, a sperm whale had been called into existence, several miles above the surface of an alien planet. But since this is not a naturally tenable position for a whale, this innocent creature had very little time to come to terms with its identity.
1) Most consumers have absolutely no idea how airplane autopilot works. Pulling a “well actually” on this does not remove the issue of misleading marketing.
2) Yes, and the videos I’ve seen from those beta testers are terrifying.
No, I don’t think people believe that at all. A certain meme entered pop culture years ago that either the pilot is there for takeoff and landing, or the pilot is there in order to make the passengers feel safer. I certainly recall a lot of “planes these days basically fly themselves” articles. Neither of these are strictly correct about how autopilot works, but we are arguing about cultural understanding here.
Even taking the lesser of these two scenarios, this would imply that Tesla autopilot should be capable of doing most of the driving, with the driver just there for whatever bits you might argue are equivalent to a takeoff and landing. Given the stories of Teslas accelerating into stationary objects or totally losing track of the road lane, I don’t think even this limited understanding of what “autopilot” means is correct.
> "The person in the driver's seat is only there for legal reasons. He is not doing anything. The car is driving itself."
> “Full Self-Driving Capability All new Tesla cars have the hardware needed in the future for full self-driving in almost all circumstances. The system is designed to be able to conduct short and long distance trips with no action required by the person in the driver’s seat. All you will need to do is get in and tell your car where to go. If you don’t say anything, the car will look at your calendar and take you there as the assumed destination or just home if nothing is on the calendar. Your Tesla will figure out the optimal route, navigate urban streets (even without lane markings), manage complex intersections with traffic lights, stop signs and roundabouts, and handle densely packed freeways with cars moving at high speed. When you arrive at your destination, simply step out at the entrance and your car will enter park seek mode, automatically search for a spot and park itself. A tap on your phone summons it back to you. The future use of these features without supervision is dependent on achieving reliability far in excess of human drivers as demonstrated by billions of miles of experience, as well as regulatory approval, which may take longer in some jurisdictions. As these self-driving capabilities are introduced, your car will be continuously upgraded through over-the-air software updates.”
Yeah, Musk blatantly lying about these things really doesn’t help their case one bit. It’s hard to hide behind “autopilot is just marketing” when their popular CEO is lying about its capabilities left and right.
Cruise control is literally just for (at least in a traditional sense) keeping your speed constant.
You're still responsible for watching traffic and using the brake or gas as needed, especially if there is some issue up ahead that requires a safety maneuver of any kind.
i.e. an emergency vehicle, like a California Highway Patrol vehicle, begins a traffic break 15 car lengths ahead of you. As long as you're observant, you'll notice that in time and either cancel the cruise control or make the right adjustment so you slow down enough to the break speed.
By the same line of reasoning, you could say that my car doesn't have an emergency oxygen mask, therefore airliners don't need them either. If the airplane goes too high for passengers to breathe comfortably, it's 100% the pilot's fault, end of story.
The pre-collision avoidance should have kicked in either way. My Subaru has saved me once or twice when a car in front of me braked harder than I realized, and they’re not even claiming anything close to full self driving.
The Subaru wouldn't stop for a stationary vehicle though. Most of those systems either don't work at all for stationary vehicles, or work far less reliably. They're mostly designed for the case you described - car ahead slows suddenly and the driver isn't paying attention.
Care to clarify what you mean here? Had a subaru outback with eyesight a few years ago and I tested it with stationary objects in path multiple times (cardboard boxes and such). The subaru braked and stopped short of the objects 100% of the time.
When the vehicle you're about to collide with is moving slowly, there are good radar returns, so you can use both vision and radar to make the decison.
When the vehicle is stationary, the radar data has to be heavily filtered because radar has poor angular resolution and there are a lot of other stationary objects, many that you don't want to stop for at similar angles, like overhead road signs. That means you're relying purely on vision, which alone has a lot of false positives and negatives, so detection thresholds need to be set much lower.
Doesn't mean it won't work. Just means it's less likely to work, especially in the dark and the rain, where radar would normally be a big part of the signal.
> Most of those systems either don't work at all for stationary vehicle
Citation needed. Subaru claims their eye sight system will detect and attempt to brake for a stationary vehicle. Whether or not they’ll managed to halt the vehicle is obviously speed dependent, but obviously slowing down before the crash is vastly superior to maintaining speed.
>Why is Tesla “autopilot” still allowed on Californian roads?
Because Tesla suckered some idiots to take the blame if for example the car will instantly change direction and throw you of a bridge or in from of a truck, Tesla is saying that their customers are supposed to be ready for this cases .
For now only some Tesla alpha testers suckers got killed and no innocent person, but when an innocent will be killed I am waiting to see if the driver and hist lawyer will accept the blame or will fight the ToS claims.
I interviewed at a self driving car startup in 2016 that was testing their early prototypes on public roads! They were also using their investor money to rent out some ten million dollar home in the Silicon Valley hills as their office. Big red flags all around. I declined their request for a second round interview.
>They were also using their investor money to rent out some ten million dollar home in the Silicon Valley hills as their office
Is this supposed to be a bad thing? Why?
The focus on the price of the house seems pretty strange in an area with a particularly high price-to-rent ratio. But I guess $20k/mo just doesn’t sound as spicy.
That would be a very strange detail to leave out. But sure, it’s the only way their comment makes any sense.
Otherwise, spending investor money on office space seems like a perfectly normal thing to do. The $ amount described doesn’t seem too unusual for the area, and a house might offer significant advantages over traditional office space for a startup that is working on cars.
Most of these seem like separate issues. Founders living in the office is weird regardless of it being a “ten million dollar house”.
> Is it centrally located for commuting workers and testing sites or the personal preferences of the founders?
The house in question was probably the comma.ai office in West Portal, SF (right next to the West Portal Station). Draw your own conclusions regarding commutes, it’s a pretty central location.
> Is it scalable for a growing company?
Offices are rarely very scalable, you just move somewhere else once it no longer works.
Seems like setting up in houses is a common pattern for Bay Area self driving car startups then. I’d guess that the availability of decent office space with workshop bays is essentially nonexistent
Not in the South Bay where this company was. Plenty of machine shops, mechanics, etc with workshop bays imo. What was true at the time was that investors were throwing money hand over fist at self driving car startups, even to young founders fresh out of college. I would not discount the psychological appeal to a 20-something founder with millions of dollars a to burn to set up shop in a mansion.
The point of my original comment is not to say that doing so is objectively bad, but I had just left an abusive startup run by a young and inexperienced startup and I wanted something more stable. I ended up getting a job at a Toyota research branch in a boring office park and I played with a cute little robot. It was just what I needed at the time.
> * Are the founders or employees living in their office?
This is common for startups in the seed stage. I don't personally recommend it but I wouldn't consider it to be a red flag for a pre-series-A startup.
> * Is there adequate parking or public transportation?
There is practically no public transportation in the bay area. No, there is never enough parking.
> * Is it legal? Does it conform to HOA and city requirements?
Nobody takes a shit about HOAs. Startups exist to drill square pegs into circular holes. Hell, I'd love to create a startup to help people get rid of HOAs.
You get it. It felt like they were inexperienced and burning cash unnecessarily. I had just left a very unprofessional and in hindsight abusive startup and I didn’t want to walk in to another “boys clubhouse” situation.
Anther explanation could be that many startups try to give out the vibe of "just look at how much investor cash we can burn, come join in on the fun!", and are very open about that. I can understand why you wouldn't want to join such a company, even though that is exactly what many others are looking for.
Sure, but now we’re just guessing. We could go on all day listing bad things founders could do, but the original comment simply doesn’t go beyond implying that it’s bad to use a $10M house in Silicon Valley as an office.
The price mentioned isn’t anything unusual even for fairly early stage startups in the Bay Area.
Well, when humans try to explain a culture that was alien to them they tend to use examples from that encounter. I can very well see the founder proudly saying "Look at this cool 10 million dollar office we have!", and then this guy latched on to that statement as being representative of their culture. When I work at an office I don't know what it costs, I doubt he would have looked it up if they didn't tell him, so this explanation seems likely to me. Of course to you that is not a good representation of the culture, so you protest here. Which is good, now we all know a bit more how to state things so others can understand it.
We're not guessing, at least not randomly, we're using intuition. Applying experiences and subtle cues to create something more educated than a random guess.
I see you have something like a half dozen comments now trying to rationalize this all cross the comment section, but like yeah renting out a mansion for your startup does send out a "vibe".
If you don't realize that a) you're probably who they're looking for, and b) you probably have some room to grow when it comes to picking up on subtext.
-
Not everything in life can be quantified in a neat little bullet pointed list. But part of being an experienced person is being able to pick out the more subtle aspects of situations.
To me a mansion as a startup HQ screams tech-bro 24-7 party culture with some major WLB issues.
And again, it's not like using a mansion magically causes that means that, but it strongly implies it.
It's like walking into an office and seeing everyone in suits and ties vs seeing everyone in sweatpants and baggy tees.
Technically the choice of clothes does not directly force you to behave a certain way, but your intuition should tell you that those are two very different cultures, and it should tell you a little bit about each.
tl;dr Ignoring intuition because it's not backed by unequivocal fact is not a super-power, and it's not really productive.
> If you don't realize that a) you're probably who they're looking for, and b) you probably have some room to grow when it comes to picking up on subtext.
Don’t get me wrong, I’d never work with the manchildren at comma.ai. I’ve interacted extensively with George Hotz in the past and he’s certainly not a person I’d want to spend any time in the same room with.
I just don’t see any issue with their choice of office space, it just seems like a perfect fit for their business.
> To me a mansion as a startup HQ screams tech-bro 24-7 party culture with some major WLB issues
Where else would you put your car tech startup in SF? Software engineers aren’t going to work out of a garage, and good luck finding a traditional office space with decent facilities for working on cars.
> Where else would you put your car tech startup in SF? Software engineers aren’t going to work out of a garage
At the same time as the $10m mansion place I interviewed for a self driving truck company and they had their office located in an industrial park in a shop that looks like it used to be an auto mechanic with upstairs offices. That place was fine but I didn’t feel like working for a 22 year old CEO who had just left college. Back in 2016 investors were dumping money on self driving car startups but I wanted something stable.
> Where else would you put your car tech startup in SF? Software engineers aren’t going to work out of a garage, and good luck finding a traditional office space with decent facilities for working on cars.
In…a garage. I’m serious - there are spaces big enough in SOMA and other places that can fit a garage space plus open office space plus some conference rooms (I’ve worked in several of them).
To me it seemed more expensive than renting a boring office building with few workshop bays with roll top doors. So to me, they were burning unnecessary cash and this made it seem like they were inexperienced. The CEO was some university researcher who had done some self driving car work and then had investors throw I think more than $10m at him. I just got the sense that this guy was feeling flush with cash and wanted to burn some on a luxurious multi level mansion with a pool and a hot tub.
I had just gotten out of a startup where the guy treated the place like his personal clubhouse and had a really controlling and abusive attitude. I wanted to steer clear of any company that seemed inexperienced.
It apparently doesn’t look that way to you, but that’s how it looked to me and I see other people here can relate.
No, I just meant that they were using some extravagant home in the hills as an office rather than doing the less exciting thing of renting an office in one of the many office parks. It felt like they were wasting money on a lavish lifestyle. To me it seemed like an indicator of a lack of prudent business sense.
I had also just left a startup where the CEO treated the company like some boys clubhouse and I did not want to get in to a situation like that again.
If it does that more than people then it's not ready. People hit signs and dividers all the time. It's hard to come to realistic conclusions based on one case.
A bad driver risks his own life so he has good reasons to try to improve. A bad self driving car program doesn't risk the founders life so he has no reason to improve it if regulations doesn't force him to.
It risks the founder's net worth since we ban their cars from driving if they aren't safe. If we didn't then they would continue to run unsafe programs.
It may be much better. This is a single case and we get every single case reported. The problem is that computer vision is not the same as human vision and at some point we may end up with a situation close to "this car drives itself perfectly apart from running over a completely obvious, well visible person once a year". And it's going to be a really hard one to process for many people.
Well, Tesla's autopilot has been slamming into emergency vehicles on the sides of highways for years and the feds have finally gotten around to investigating things.
Their full self driving beta requires constant vigilance because it likes to randomly veer at pedestrians and light poles.
As a sidenote, it's irritating that Reuters keeps calling it an "accident." It was a crash.
After they got a permit to drive without a safety driver? I don't think so. The permit should be for when the technology is mature enough to never do these kinds of mistakes.
It's awe-inspiring how much more quickly a human brain can learn versus a computer. These companies have expended tens of thousands of engineer-hours of some one the best minds in the industry to build models derived from tens of thousands of hours of training data, and still I'd place money on a novice human driver with just ~100 hours of experience besting any AI in the majority of "strange" driving circumstances.
Of course, if that human brain is tired, stressed, and checking their phone, it's a whole different story :)
The driving part is usually normal algorithm code and not machine learning though. Machine learning is for object detection, what the software does with that knowledge is regular programming.
That's not quite right. There are learned models inside the planning systems of most of the sophisticated self-driving car companies. Here's a public video that gives a few high-level examples: https://vimeo.com/618478174
Full disclosure, I work for Aurora in the Planning group.
How do you know those models are safe? I know Google said they didn't use machine learning for the driving part.
Edit: To clarify, I wonder how you do this part:
> make safe, predictable decisions on the road.
ML models doesn't make predictable outputs. So you always need a system that can handle the ML model going haywire, since you have no way to prove that it wont.
This is probably a miscommunication issue. Waymo absolutely uses machine learning in the perception and prediction layers of the stack. Underneath that are a few layers of trajectory planning and controls code that don't necessarily need ML.
As for stack accuracy and predictability, they're active areas of research for everyone, and part of the secret sauce. There's lots of little mitigations scattered throughout everyone's stacks for various particular issues, but no silver bullet.
Very much so. The tens of thousands of hours of training is like the learning babies do while looking at jangling keys. Then the driving part is more static -- I'm not sure that it's entirely regular programming, there may be some training of models there. But my understanding is it's certainly not learning/training _while_ driving, and _almost_ certainly it's not even learning to drive. It's just doing more learning for object detection and tracking, like a baby watching the world from a pram.
So I feel like no matter how many autonomous hours these cars accumulate on the road, they have effectively 0 hours of "learning to drive".
I think the only (but very major) difference is that is that animals and humans learn general knowledge. Someone with 100 hours of driving also has 16+ years of additional general learning.
A driving AI has no general knowledge. It doesn’t actually understand what it is looking at.
It’s not possible to make up reasonable responses consistently when you don’t know what you are looking at most of the time, machine or human. Have you ever tried to solve a technical problem before you read the documentation covering the basic concepts? You’re basically throwing shit at the wall until something sticks.
> I'm not sure that it's entirely regular programming, there may be some training of models there
I doubt it, not anything serious at least. Machine learning isn't at the level where it can be used to navigate environments better than human coded algorithms can. Maybe if you had a datacenter worth of computing power to do it, like AlphaStar, but you don't have that in a car. And even if you had that I still doubt it, Alphastar had a perfect representation of the game and could simulate the effects of moves perfectly.
I had about 16 years of walking and 14 years of cycling around gradually increasingly complex environments before I drove a car.
The car controls are not the challenge. It’s understanding what everyone else is doing and how to handle it safely while making progress and not surprising other people that is tough. And those skills are partially - only partially - transferable from your pre-car experience.
>I had about 16 years of walking and 14 years of cycling
There are young kids that drive carts and race and they don't confuse walls and solid objects. The reason we don't allow younger people to drive is because young people are bad at managing risk(I done some very stupid risky things with my bike as a teen, I could have broke my bones or neck),
Autopilot from Tesla that uses mostly or only cameras has the issue of identifying obstacles which is a much generic problem(that young animals and kids have it solved) so you don't need to appeal to the "age of driving" excuse for this guys.
Not to mention, it's not just driving that makes you a good driver. It's 16 years of life experience. You spend that time growing up riding around in cars and busses. Living in the world, knowing how things work, learning how people behave. Driving is so much more than just pointing your wheel the way you want to go and pressing the gas and avoiding objects.
So many people don't even know all the rules of the road, but still manage to drive just fine without accidents. There are different unknown rules of the road in all different places around the world, but most people figure out how to adapt to them in a very short time.
> I had about 16 years of walking and 14 years of cycling around gradually increasingly complex environments before I drove a car.
A lot of that has to do with humans not being born with fully developed brains. Animals can walk and navigate environments as soon as they are born, so there is no reason to believe that humans actually learns these things rather than those systems slowly maturing as we grow up.
The other part is that our pre-trained movement system is made for human bodies, not cars. So learning to drive a car is learning to use another mode of moving yourself. Also you have to learn traffic rules. But the other things like understanding environments etc you get for free for being a human, every single large animal can do the same. They wont learn the traffic laws or how to drive the car, but they know how to navigate environments without hitting things.
I feel like driving a vehicle is fundamentally different from "natural" activities, because it involves a large difference in how you deal with things ahead versus the sides.
Things in front are highly compressed, while things to the sides are not, and your ability to move sideways is reduced by mass and speed.
When I was a teenager, I went on a road trip where I drove like 12 or more hours in a day, which I could never do now. At the end, I got home and for a while I had a weird sort of tunnel vision that I don't know how to describe. It was almost like I was looking through a fish-eye lens or something; everything seemed distorted because of spending so much time concentrating on small lateral motions and things coming towards me at highway speeds.
Sometimes I walk routes that I also frequently drive and it reminds me how motor vehicles compress time and space.
Birds can fly through thick forests without colliding with trees, and flying also is very clunky and you can't change your direction quickly etc. If they collide they die, so the selective pressure is really strong for them to not hit anything. Cheetahs can also run at highway speeds on uneven ground and still do just fine.
Horses (actually, all the equines) are able to stand within minutes of birth, and run with the herd within hours. This indicates how much is built in before any learning, a useful data point.
(Amusingly, while standing up is clearly built in - foals usually stand on the first try - lying down is not. I've seen a newborn foal try to lie down for the first time, and after moving a few legs, just collapsing to the ground. This makes sense - there's a survival advantage to being able to stand up quickly and reliably, while being able to lie down and rest quickly need not work as well.
>> This indicates how much is built in before any learning, a useful data point.
Animals have a heck of a lot of background knowledge like that. Statistical machine learning algorithms can't incorporate background knowledge [edit: not easily] and have to learn everything end-to-end. Hence why they remain dumb as bricks, compared to even simple animals (like insects, which are "simple" only compared to mammals, say).
This sounds like a start of a dystopian novel, where animal brains are being shoved into various robots. And that a researcher who's trying to stop it, ultimately fails and he's also turned into a robot.
I wonder what comes first, engineered biological brains that we can put in things like cars or that machine learning has progressed to the point we no longer need biological brains.
Biological brains are super cheap and efficient though. The brain of a fly can still do object recognition and navigate environments for basically free, weighs less than a milligram including cameras, and the factory to produce those brains is just putting a cell in a wet environment and it builds itself, the brain of an ant can build complex structures etc. It is possible metal brains will never beat those.
I seem to remember a short story by James Tiptree Jr. who had a somewhat similar theme. In the story it was children born with deformities who were given robotic bodies and then spaceships to navigate, if I remember correctly.
I occasionally drive in the NYC metro area, and I'm pretty sure a 100 hour old infant could drive better than some of the folks I'm sharing the road with.
>> It's awe-inspiring how much more quickly a human brain can learn versus a computer.
Do you think we live in StarTrek future already? Human brain is the most complex thing in the entire universe, as far as we know so far. You compare it to a what, a computer?
The end result is that Uber ended their self driving car program and paid the family money, while the grunt worker who was put in the car without proper training got charged with negligent homicide and that trial is still going.
The person responsible for that program should be charged, not the safety driver, but that is how things goes. You can't just put random people in a car and call them safety drivers, and charging such a person with homicide doesn't really get to the root of the problem.
The problem is the justice system assumes 1 person kills 1 other person.
In reality there were 100s of people in the chain of events, from the safety driver being bored and watching their phone, to the person who hired them and didn't train them properly, to the person who invented the safety driver system and didn't think people would get bored.
Then you have the people who made the decision to turn off some of the subsystems of the car and then test it at night with no extra precautions, and every programmer who let that happen.
The the management of the company who provided the wrong incentives to their employees.
And that's before you get to the people who designed the roads, and the lack of social care to look after people who for some reason are homeless
The thought that drivers will be put in jail for killing someone (especially a pedestrian or cyclist) already seems optimistic. Often they aren't even charged, even in cases of extreme road rage where the collision is deliberate.
In my part of aviation (not in the US) we have a person responsible for each subsystem, and they very much risk going to jail if it turns out they've been negligent. If it's not safe, you don't sign the papers saying it is. People wouldn't dream of doing the equivalent of putting a prototype car on public roads with an aircraft here.
Aren't there entire abandoned cities, and yet-to-be-filled suburb developments in the world? Are autonomous driving companies not using those, stuffed full of "stunt" drivers who are employed to do nothing but simulate regular city traffic?
You really under estimate how much externalized risks some people are willing to accept. If people can save millions by risking others lives then many will do so. This is why we have regulations, such as this permit process which is basically the government saying "your technology needs further testing".
Heck, many industries are based on pollution or addiction: tobacco, alcohol, fossil fuels, casinos, sports gambling, etc.
Basically stuff the owners don't do, but they're perfectly fine if it happens to others (for the pollution bit, I'm reasonably sure the stakeholders and executives live in nice, low pollution areas).
The current state of self driving cars can't handle basic, well behaved, standard traffic.
There's no way it's ready for the sort of adversarial stuff you'd get hiring a bunch of people to drive around simulating traffic for a car. I guarantee it's a great way to find out how your car is at dealing with being cut off, at people running stop signs, etc - and there's certainly value in doing that.
Just, not while you still can't avoid hitting stationary objects along the edge of the road.
> The current state of self driving cars can't handle basic, well behaved, standard traffic.
Some self-driving cars can't.
At one end, you have the Uber self-driving vehicles which were caught committing numerous traffic offenses by dashcams and cyclist helmet cams within the first days of their illegally testing their fleet on public roads, and then one of their cars ran over a pedestrian that the vehicle's sensors detected but the computers misclassified, somehow - and Uber had disabled the SUV's own auto-braking systems that probably would have saved the woman's life.
At the other end you have Google's self-driving cars have had, I think one at-fault crash across the entire fleet in many, many years and millions of miles of testing? They've been involved in a lot of crashes, but every single one until their one slow-speed scrape with a bus was the fault of another driver. That was 5 years ago or so.
Somewhere in-between is Tesla's FSD "Beta" which, as has been shown by numerous youtube videos (at least, the ones that Tesla doesn't abuse DMCA to take down), behaves extremely erratically repeatedly in the space of less than half an hour in an urban environment. This includes veering off a turn at pedestrians on a curb corner, and in another incident veering toward a light pole. In general, it seems to have a lot of problems making turns.
What would that accomplish? The goal is for these cars to self-drive in locations where people live, and that does include mapping those locations over many drives. Autonomous vehicles still have a good safety record, and California's willingness to suspend a permit when nobody was injured does signal to me that they'll take it very seriously when a company's safety record falters.
Here's one of them. It's a modern, former company town now used by DHS for counterterrorism training, but AFAIK it's not used for self-driving car testing. Maybe because it's too far from the Bay Area or it's just not well-known.
I believe the first generations of driverless cars have to be overly cautious. If it can't be 100% sure that it's model corresponds to reality, and that nothing is going to jump out of the bush, then it has to stop. A safe car is going to stop or slow down at least once per trip, for the forseeable future.
I think the car should also talk about what it thinks, to get people to understand the limitations of the technology.
"There could be a child between those parking cars, slowing down"
"I cannot see the road markings, emergency stop"
"Road signs don't match my data, slowing down"
"Train mode activated, following virtual track"
Then, and only then, you can do the second step. Automated cars have the potential to be much safer than cars with human drivers. You can accept some accidents, in order to prevent many more. But you can only do that if the public and lawmakers have confidence in the technology. Right now it is way too much of a black box with wierd failure modes.
This reminds me of a situation I saw with another autonomous car company's vehicle in SF a couple weeks back. I truly didn't know what the correct move by the vehicle should have been.
The situation: The vehicle was on Vallejo crossing west to east over Sansome Street, a one-way, two-lane thoroughfare. For Sansome, there is no stop sign here, but there is for Vallejo. The vehicle stopped at its sign, then began to proceed across the intersection, until two pedestrians entered the crosswalk from south to north along the far side of Sansome. The vehicle stopped in the middle of the street to wait for the pedestrians to cross. There was no cross traffic coming, but what if there were? Seems like a situation ripe for a side collision, especially because Sansome is brisk at this point.
I really don't know what the capabilities of the sensors are, or how situations are assessed, so I'm assuming it did what it did based on its situational awareness. Wondered if the vehicle could have backed up to the stop sign it came from if there was cross traffic, or if it could have broken the rules and gone around the pedestrians if there was danger of a side collision.
Depends on how you define "viable self driving car". If that means it needs to drive at least as well as humans in every scenario you tend to use a car in, then yeah probably, there are so many places with overgrown roads, avoid getting stuck in sand etc. If however you just want it to navigate highways and city streets, then probably not. The self driving cars wont be perfect around people and human drivers, no, but pedestrians and human drivers can update their own behaviour to compensate. It took a while for pedestrians to understand how to behave around human drivers, the same thing will happen for autonomous cars.
Humans are generally predictable, even crazy ones, since we get to interact with them from day 1, after birth. Plus in many places there are decent tests in place that block some of the crazies.
with all the billions being spent on fully self driving cars why doesn't a company build a fake indoor town where it can simulate various driving conditions, weather, lighting etc and put fake pedestrian everywhere and then have the AI be able to learn in an environment simultaneously more challenging (since you can throw all kinds of shit at it) and forgiving (since mistakes while training aren't hurting people).
"On October 28, 2021, after turning right onto Fremont Blvd from Cushing Pkwy, the Pony.ai Autonomous Vehicle ("Pony.ai AV") performed a left lane change maneuver in autonomous mode. While performing the lane change, the Pony.ai AV came into contact with a center divider on Fremont Blvd. and the traffic sign that was posted on the divider. The Pony.ai AV suffered moderate damage to the front of the vehicle and the undercarriage. There were no injuries and no other vehicles involved. Fremont Police Department were called to report the incident and the damaged street sign. Pony.ai has subsequently worked with local authorities to resolve all issues related to the damaged sign."
It's not a serious incident, but it is a fully autonomous vehicle under good conditions going somewhere it was absolutely not supposed to go. So DMV revoked their license for full autonomous operation without a safety driver. They have to go back to testing with a safety driver.
[1] https://www.dmv.ca.gov/portal/file/pony-ai_102821-pdf/