Toshiba's Image Recognition Chip Enables To Detect Pedestrians At Night

Toshiba's Image Recognition Chip Enables to Detect Pedestrians at NightToshiba Corp developed a new series of its image recognition chips for driving support systems and announced its functions, etc at a press conference of electronica 2014.

The company defines the series, "TMPV760," as its fourth-generation product.

Pedestrian detection at night

Compared with Toshiba's previous products, the TMPV760 has two new major characteristics. It can detect pedestrians during nighttime hours and roughly obtain distance information only with one automotive camera to realize 3D recognition.

In 2018, the European New Car Assessment Programme (Euro NCAP) will start to require the capability to detect pedestrians during both daytime and nighttime hours. And the new product was designed to deal with it. Toshiba aims to start volume production of the series in December 2016.

Toshiba enabled to detect pedestrians at night by making improvements to an image recognition technology called "CoHOG." It can recognize the boundary between a pedestrian and the environment even at night by extracting five kinds of data such as the gradation of the color of the pedestrian and the texture of the pedestrian from video of the pedestrian and using the data.

Compared with the company's existing automotive image recognition chip, the new product's capability to detect pedestrians at night was improved by about 100%, the company said.

3D model created with monocular camera

To realize 3D recognition with a monocular camera, Toshiba used a technology called "SfM (Structure from Motion)," which moves the position of a camera to change its viewpoint while taking multiple images and compares the images to create a 3D model of the target object.

With an automotive camera, it is possible to take images from different viewpoints when the vehicle moves. Therefore, the SfM enables to make a simple 3D model of the environment around the vehicle, Toshiba said.

Automakers, not Toshiba, will decide how to use the obtained 3D model. For example, by combining it with a 360-degree image generated by using images taken by multiple automotive cameras, it is possible to improve the accuracy of driving support functions.

Those image recognition processes can be realized with automotive cameras with a resolution from VGA to full HD (1,920 x 1,080), enabling to use a variety of cameras. The interface to cameras supports MIPI's Camera Serial Interface-2 (CSI-2).

For the image recognition chip, an accelerator for image recognition is used. And the CPU will perform the other control processes.

Traffic lights, signs can be recognized

With the TMPV760, it becomes possible not only to detect pedestrians and generate 3D models but also to recognize preceding vehicles, traffic lights, signs and lanes, Toshiba said.

When developing the new product, Toshiba had difficulty in effectively using the bandwidth of external memory.

"In the case of our previous image recognition chips, we decided functions in accordance with the bandwidth of external memory (DRAM)," Toshiba said. "In other words, they are limited by the memory. This time, however, we decided how to effectively use the bandwidth of the limited memory after deciding to add functions that we considered our users (automakers and electrical component manufacturers) wanted to use."

The TMPV760 is enclosed in a 27 x 27mm BGA package. The number of pins is 800.