By Not a Tesla App Staff
Tesla reduces the need for the steering wheel nag with FSD v12.4
Tesla has just rolled out its latest FSD software, v12.4 to employees. Elon Musk announced that this update would be available to employees this past weekend, with plans to release it to the public in small numbers later this week.
Surprisingly, the new update is version 2024.9.5, which is likely based on the earlier 2024.8 branch and not Tesla's latest 2024.14. The spring update (2024.14) brings various new features such as a new media player, a new parked visualization, Audible support, and a Preview of Sentry Mode events, among others.
However, FSD v12.4 brings itsownexcitement with two new major changes.
Vision-Based Attention Monitoring
The release notes show a new Vision-Based Attention Monitoring feature that replaces the steering wheel nag as Musk previously mentioned.
However, as we predicted, Tesla will still leverage the steering wheel to detect attentiveness when the cabin camera is inconclusive.
The car can only rely on the vehicle's cabin camera, and therefore remove the steering wheel nag under certain conditions:
the camera is not occluded
there is sufficient lighting
the driver is looking forward
the driver is not wearing sunglasses
the driver is not wearing a low-brim hat or another object that covers their eyes
If any of these situations occur, or if the vehicle doesn't have a cabin camera, then the vehicle will continue to use the steering wheel to determine driver attention.
Tesla is careful to state that images and video from the cabin camera are not saved or transmitted unless you enable data sharing.
Updated Strike System
With FSD v12.4, Tesla has also updated its Autopilot Suspension feature which is designed to enforce the responsible use of FSD.
The current system lets the driver receive up to five strikes (three strikes for vehicles without a cabin camera)before Autopilot and FSD become unavailable. If that happens, then FSD is unavailable for one week. Strikes are only removed once the driver has accrued five strikes, or when Tesla wipes out strikes for everyone, which happens about twice a year.
The new system is more gracious about removing strikes. The vehicle will continue to issue strikes whenever the driver isn't paying attention, however, now the vehicle will gradually remove strikes for the driver after a certainperiod of time.
Tesla states that one strike will be removed foreach 7-day periodthe driver goes without receiving a strike. So if FSD gets disabled due to strikes, the driver will still go one week without FSD, although now strikes are removed on an ongoing basis. This new strike system is expected to apply tovehicleswith and without a cabin camera.
Other New Features
Other new features areexpected in FSD 12.4 as well, which we outlined in our look at Tesla's FSD v12.4 article.
They include a focus on improved driver comfort by reducing the amount of hard braking, automatically seeking a parking spot when arriving at a destinationandmore. Driver interventions are also expected to be drastically reduced with Musk stating that Tesla expects to see a 5-10x improvementin interventions.
Eligibility
Unfortunately, due to FSD v12.4 being on branch 2024.9, it's expected to only be available to owners on update 2024.8 and below, which includes everyone currently on update 2024.3.25.
If no major issues are found with FSD v12.4, we could see it start to roll out to the public later this week.
Subscribe
Subscribe to our newsletter to stay up to date on the latest Tesla news, upcoming features and software updates.
By Karan Singh
Tesla has been improving its FSD features since introducing it in 2016
Tesla
Previously, the Tesla X account shared a video where FSD V12 appeared to consider a pedestrian’s hand gestures when navigating. Elon Musk also confirmed on X that hand gesture recognition would be improved in V12.4, and even better by V12.5.
Tesla has had plans for gesture recognition for many years, with Musk mentioning hand gesture recognition in 2021. FSD is supposed to be able to adapt to new and unique circ*mstances in a safe manner, whether following instructions from a police officer, or understanding the hand signals from a cyclist.
Boris Johnson and FSD
Boris Johnson, the ex-Prime Minister of the UK, recently got to experience FSD V12 in the crowded streets of LA, with his wife and child in the backseat. Tesla provided a vehicle and assistant for his self-driven experience through 5-road intersections, heavy traffic, and pedestrians.
At the end of about 45 minutes I feel like a driving test examiner – except that I want to tell the car that it has passed, with flying colours. -Boris Johnson
Gestures coming soon
Johnson had a single experience with hand gestures in his drive – being waved at a hotel. While the car didn’t recognize the gesture – even on FSD V12.3.6 – the Tesla official in the vehicle confirmed that “‘It doesn’t yet recognise that gesture, but we are fixing it for the next iteration. It should be done next month.”.
It’s not immediately clear whether the Tesla employee is referring to FSD v12.4, which is expected to start rolling out to the public later this week, or Tesla’s next major FSD release, V12.5. According to Musk, V12.5 is expected to handle much more complicated situations and even start vehicle-to-fleet communication.
Yeah, I wouldn’t count on this in 12.3.x, but 12.4 is a big improvement and 12.5 might nail it
— Elon Musk (@elonmusk) May 10, 2024
Solving FSD
When you’re tackling a problem like FSD, which has never been solved before, you sometimes have to pivot and realize there’s a better approach. That’s what Tesla has done with FSD and AI. Just a few years ago where people thought cars would need to be able to read signs to achieve autonomy, but with the latest approach, there’s no need to. As long as the vehicle has seen a specific sign before and has enough examples, it’s already trained on what humans do when such a sign is found. The same will be true for hand gestures, the vehicle won’t need to know what each hand gesture means, it’ll just recognize that when a current gesture is made, this is how the vehicle should react.
Tesla does this by feeding it millions of examples of very specific situations and AI recognizes patterns. This is an example where Tesla leverages fleet data. They can easily capture millions of examples of hand gestures and how humans reacted, and feed that to their AI training model.
As Tesla processes more video, FSD will continue to improve. Tesla is expected to spend $10 billion on AI this year alone, most of it going toward improving FSD.
By Not a Tesla App Staff
Coming to Rivian vehicles over a software update is Google Cast and YouTube support, which will allow for seamless, high-quality content streaming to Rivian vehicles.
This is a big step forward for Rivian’s software integration, especially compared to Tesla, which has yet to integrate native apps or release the long-rumored Tesla native app store.
Google Cast
Google Cast is Google’s version of Apple AirPlay which is supported in a variety of audio and video streaming apps such as Netflix, Disney+, YouTube, and more. By using a supported app, a user will be able to cast content directly to Rivian’s main 15.6” center screen.
Like Casting to a TV or Smart Device at home, this will allow you to play, pause, or skip back and forth using the vehicle’s touchscreen. Google Casting automatically pauses when the vehicle shifts into Drive.
Native YouTube App
Alongside Google Cast, Rivian is also bringing a native YouTube app to vehicles, enabling users to access YouTube’s video and music library from anywhere with a data connection. These features will be integrated right into Rivian’s vehicle operating system, which means they will receive regular software updates, and not be reliant on browser-based capabilities.
Similar to casting content, YouTube content will also automatically pause when the vehicle is shifted into Drive.
Tesla Equivalents
For the time being, there is no way to AirPlay or Cast video from a phone or tablet to Tesla’s front or rear screens. However, for some apps such as YouTube, you can send the video URL to the vehicle and have it open the YouTube app to that video.
To do this, open the YouTube app and find a video. Once you’d like to share it with the vehicle you can tap on share. You’ll then need to share it with the Tesla app which will then send it to your vehicle. On iPhones this requires you to scroll to the right and tap on More, which brings up the native sharing screen and has the Tesla app.
However, this is far more limited than including AirPlay or Google Cast support which will work for practically and video, even personal videos in your photo library.
Elon Musk mentioned back in 2016 that Tesla would add mirroring of phone apps to the center screen, but that has yet to materialize. More recently Musk mentioned a native X App coming to Tesla vehicles.
Tesla Theater
Tesla Theater allows users to stream videos from various platforms, including YouTube, but these are web-based applications, with no native applications available. This leaves some marked limitations on usability.
Tesla most recently announced in their 2024.14 update that the browser will be able to be expanded to full-screen mode when parked, enabling more access to streaming services on the web. Meanwhile, Google Cast and Apple Airplay continue to remain highly requested features.
