Algorithm Opens Up Multimillion Opportunity
Time of Flight AGI Graduates from Analog Garage to Consumer Business Unit
| “It’s often been said that if we only start to work on a challenge when the customer points it out to us, we are already too late. But the enterprising combination of algorithm know-how at the Analog Garage and circuit and applications know-how in the Consumer Business Unit (BU) led to a faster time to market. Together, they solved an important customer problem and added a distinguishing advantage to the product.” |
Time-of-flight (ToF) is an emerging sensing modality with significant potential. Have you seen gamers control play by waving their hands in midair? Time-of-flight (ToF) sensors make this possible. In luxury cars and long-distance trucking, ToF sensors are used to detect a tired or sleepy driver. With real-time distance measurements of the scene collected by ToF sensors, these applications can distinguish facial expressions and gestures. Drones and robots can navigate through complex environments. And augmented reality and virtual reality become more lifelike.
| The Time of Flight AGI Team from left: (front row) Evan Semle, Tomo Funabashi, Nicolas Le Dortz, Dhruvesh Gajaria, Charles Mathy, (middle row) Erik Barnes, Kainan Wang, Atulya Yellepeddi, Sefa Demirtas, Tao Yu, (back row) Bin Huo, Brian Donnelly, Rick Haltmaier, Yuzo Shida. |
Moving from Basic to Complex
The basic science behind ToF is straightforward. The system pulses light and measures the time it takes the photons to reflect off an object and back to the sensor. Because this time and the speed of light are known, distance can be calculated.
| The point cloud is a 3D visualization of the depth at each pixel as computed from a time-of-flight (ToF) camera. ToF is a depth sensing technology where an infrared light source illuminates the scene with multiple precisely-timed bursts of light. An image sensor co-located with the light source receives the returned photons from all the surfaces in the scene in the same order as the proximity of the surface to the sensor. The distance, or depth, associated with each pixel is calculated using the known speed of light and the time it takes the photons to arrive back at each pixel of the sensor. |
To precisely synchronize bursts of light, make these calculations for millions of pixels every second, and adjust for operating conditions is a significant mixed-signal circuit design and applications challenge. ADI is among a short list of companies with the expertise to enable high-performance, cost-effective ToF solutions and the Analog Devices solution has been in the market for several years. However, when faced with the challenge of operating multiple ToF cameras together – at least 10 with a roadmap to 64 cameras operating simultaneously in close proximity – the marketing and applications team turned to Analog Garage.
“A lot of trial and error would be involved in solving the interference challenge and there was a strong chance that existing technology simply was not going to be adequate,” said Bin Huo, systems design engineer for the Consumer Segment. “We thought an algorithmic approach would allow for iterations and experimentation much faster than circuit design.”
The Ideation Process
Many of the disruptive ideas Analog Garage incubates are new ventures so identifying customers and forecasting revenue can be risky, but the ToF Analog Garage Initiative (AGI) was different. There was a clear alignment between customers’ needs and a potential solution.
“We had been working with the BU for several months. They had a very deep understanding of the application and the hardware,” explained Sefa Demirtas, lead scientist and head of signal processing algorithm development at Analog Garage. “On the one hand, the BU had tremendous insight and was relaying real customer feedback continuously. This was extremely valuable. But, on the other hand, they were raising a variety of issues from interference to power consumption to dynamic range. We needed these to be prioritized.”
Atulya Yellepeddi, research scientist at Analog Garage, added, “There was a clear constraint on the solution side in that we knew we had to use the sensor and camera module with the ADI AFE. But we needed to similarly constrain the problem to be solved. Following the Analog Garage process provided the structure our teams needed to effectively collaborate.”
The Analog Garage process also helped the team think through the staffing strategy. The team proposed a group of five people working at 50%, as opposed to 2 or 3 people full time. “Brainstorming is essential in algorithm discovery,” said Sefa. “With more people critiquing, good ideas surface faster and bad ideas sink faster.” Sefa also noted that gaining the commitment of the individuals in advance was equally important. Research scientists Charles Mathy, Nicolas Le Dortz, Tao Yu, Atulya and Sefa each committed to spend 50% of their time on the AGI. And the team jelled from the start.
Interference Reduction Algorithms
To enable experimentation and fast learning using the ADI-equipped sensor and camera modules, Tao and Nicolas dove deep into the BU’s test-and-debug GUI and created a tool for conducting experiments in the ToF lab at Analog Garage. This tool was essential to the team, enabling them to test ideas, simulate multi-camera environments, and demonstrate the impact of the algorithms on interference and power consumption.
The team came up with two approaches to address the need: interference avoidance and interference cancellation. The two approaches were combined in the last part of the project to allow for many more cameras to operate simultaneously than possible with one algorithm alone.
Creating and Capturing Value
According to Pat O’Doherty, vice president of emerging business and head of Analog Garage, “The interference reduction algorithms that resulted from the ToF AGI are a shining example of going beyond silicon to create and capture value. It’s important to note that the innovation was the result of combining the BU’s deep understanding of the application and the device architecture, and the algorithm expertise of the Analog Garage scientists.”