Several analysts and tech experts believe that Apple is working on an AR headset. A new patent describes how Cupertino would use gaze tracking to give video AR experiences on its ‘head-mounted- display,’ which we commonly refer to as the Apple Glasses.
“Head-mounted displays are used to provide virtual reality, augmented reality, and/or mixed reality experiences for users. Video from a virtual reality or mixed reality experience may be recorded for later playback or analysis. Gaze-tracking sensors may be used to provide an indication of where a subject person is currently looking […]
The subject matter described in this specification can be embodied in systems that include one or more gaze-tracking sensors, one or more image sensors, and a processing apparatus configured to: access gaze data captured using the one or more gaze-tracking sensors; apply a temporal filter to the gaze data to obtain a smoothed gaze estimate; determine a region of interest based on the smoothed gaze estimate, wherein the region of interest identifies a subset of a field of view of the one or more image sensors; access a frame of video captured using the one or more image sensors; and apply signal processing to the frame of video based on the region of interest to obtain an enhanced frame of video.”
According to Apple’s latest patent application, the new Apple Glasses would use video to complete the AR experience, by laying over AR content over the real-world environment. It would also use multiple image sensors and a mix of sensor-selecting and digital zoom to capture what the user is looking at.
“Moving the recording box with the gaze center point can result in a jittery video. Some implementations run a smoothing or stabilization algorithm on top of gaze center points to avoid jitter in recorded video and get a more pleasant recording.
It is believed that these Apple Glasses may be an iPhone accessory since the AR content would be wirelessly transmitted from the Apple Phone, even though the patent suggests that it may also work with more Apple products. Some rumors also say that this headset may launch until 2022, or maybe even later than that.
Via Patently Apple