How do predictive touchscreens work?

Touchscreens have something in common with Marmite: we either love them or hate them. Except, it’s not all that clear whether anyone actually does love them. They seemed a good idea at the time, for decluttering instrument panels, but lots of drivers find them a dodgy distraction.

The latest idea to emerge for reducing the need to glance quite so intently at the icon you’re trying to poke is Predictive Touch, a new technology cooked up by Jaguar Land Rover’s Human Machine Interface Research Group and the University of Cambridge. The system predicts what your finger is aiming for before it even touches the screen and makes the selection without the need for contact. Doing the job without the need to take such accurate aim could go a long way to reducing distraction, but how can it reliably know the driver’s choice and avoid making false selections?

As the driver or passenger moves to touch the screen, the system tracks the movement of their hand using a time of flight (ToF) camera to keep track of its position relative to the screen. The ToF camera uses the infrared light spectrum to track hand movement in 3D. The camera can be used solo or in conjunction with a vision-based system that works a bit like radar by measuring the time it takes for the light it emits to bounce back from the subject – in this case, the hand.

Advanced ‘time of flight’ camera tech combined with machine learning algorithms has made the Predictive Touch screen possible.

So, how does the system let the user know it’s about to make the right choice without causing more distraction? Theoretical analysis earmarked four suitable methods of preselecting items on the screen: highlighting, changing the size, glowing colour and haptics. The last method can include using ultrasound to create feel in a fingertip while it’s still in mid-air.

Following the analysis, there were live experiments with human testers to focus in on the best choice. Using advanced cognitive load and distraction techniques developed by the Massachusetts Institute of Technology (MIT), combined with customer preferences, it became clear that the best solution was a combination of increasing the icon size and making it glow.

Probably the greatest benefit of the system will be reducing driver distraction. This is no gimmicky flash in the pan, either. Jaguar Land Rover and the University of Cambridge published an academic paper on the early work way back in 2016.

Even then, data collected from 20 participants in various driving conditions showed “that a predictive display can reduce workload, effort and duration of completing on-screen selection tasks in vehicles”.

Four years on and it could be that, at last, there’s a solution – other than buttons, touchscreens or weird conversations with robotic voice recognition systems – which could make driving modern, complex cars much safer. The latest laboratory results and on-road trials have shown that driver interaction with a touchscreen could be reduced by as much as 50 percent.

A Breath Of Fresh Air 

Predictive Touch is part of improving the wellbeing of drivers, and so is improved air con. Hyundai’s new system features After-Blow to dry out the system’s passageways, after the engine turns off, to avoid mouldy smells. Multi-Air mode pumps the air out of multiple vents instead of a few, avoiding uncomfortable icy blasts, while Fine Dust Indicator gives details of harmful ultra-fine particles in the cabin and cleans them up.