To avoid accidents, self-driving cars and drones need to be able to quickly take in their surroundings even in extreme road conditions and bad weather, but conventional optical cameras aren't quite up to the job. To improve the vision of such vehicles, a team of engineers at the Nanyang Technological University Singapore (NTU Singapore), led by Assistant Professor Chen Shoushun, has developed an ultrafast, high-contrast smart camera that records the changes in light intensity between scenes at nanosecond intervals to detect movement and objects in real time.
Autonomous vehicle technology has progressed remarkably in the past decade and has taken to the streets in numerous trials on the way to becoming a regular fixture on our roads. To operate safely they are dependent on sophisticated sensors – and especially cameras, which need to be fast and accurate. But conventional optical cameras can be blinded by bright light and have trouble resolving images quickly in bad weather or darkness.
According to the NUT team, this is because current camera have sensors made of millions of pixels that use light to form an image. A high-speed video camera, such as a self-driving car would use, running at 120 frames a second, generates gigabytes of video data that needs to be analyzed for autonomous systems to "see" the surrounding environment. In very bright light, bad weather, darkness, or where it's just visually complex, it can take some time to process the images, which results in a significant time lag that a fast-moving vehicle or drone can't afford.
Chen and his team started developing the new camera, called Celex, in 2009 and it's now in its final prototype stage. The team claims that it can record the slightest movements and objects in real time, is faster than conventional cameras by operating at nanosecond intervals, and can store images in a new data format allowing for smaller files.
The team says that Celex contains a built-in processor circuit that analyzes changes in light intensity of individual pixels at the sensor. As this avoids analyzing whole images, it reduces the amount data that needs to be handled and speeds up the process. The method also allows the camera to quickly differentiate between foreground and background objects and gives the vehicle's computer more time to react to situations.
NTU says that the new technology has attracted the interest of private industry and that Chen and his team have spun off a company called Hillhouse Tech to handle commercialization of the camera. According to Chen, the camera may be commercially available by the end of 2017.
"Our new camera can be a great safety tool for autonomous vehicles, since it can see very far ahead like optical cameras but without the time lag needed to analyze and process the video feed," says Chen. "With its continuous tracking feature and instant analysis of a scene, it complements existing optical and laser cameras and can help self-driving vehicles and drones avoid unexpected collisions that usually happen within seconds."
A prototype of the camera was shown at the 2017 IS&T International Symposium on Electronic Imaging (EI 2017) and the research was published by the Institute of Electrical and Electronics Engineers (IEEE).
Source: NTU Singapore
To avoid accidents, self-driving cars and drones need to be able to quickly take in their surroundings even in extreme road conditions and bad weather, but conventional optical cameras aren't quite up to the job. To improve the vision of such vehicles, a team of engineers at the Nanyang Technological University Singapore (NTU Singapore), led by Assistant Professor Chen Shoushun, has developed an ultrafast, high-contrast smart camera that records the changes in light intensity between scenes at nanosecond intervals to detect movement and objects in real time.
Autonomous vehicle technology has progressed remarkably in the past decade and has taken to the streets in numerous trials on the way to becoming a regular fixture on our roads. To operate safely they are dependent on sophisticated sensors – and especially cameras, which need to be fast and accurate. But conventional optical cameras can be blinded by bright light and have trouble resolving images quickly in bad weather or darkness.
According to the NUT team, this is because current camera have sensors made of millions of pixels that use light to form an image. A high-speed video camera, such as a self-driving car would use, running at 120 frames a second, generates gigabytes of video data that needs to be analyzed for autonomous systems to "see" the surrounding environment. In very bright light, bad weather, darkness, or where it's just visually complex, it can take some time to process the images, which results in a significant time lag that a fast-moving vehicle or drone can't afford.
Chen and his team started developing the new camera, called Celex, in 2009 and it's now in its final prototype stage. The team claims that it can record the slightest movements and objects in real time, is faster than conventional cameras by operating at nanosecond intervals, and can store images in a new data format allowing for smaller files.
The team says that Celex contains a built-in processor circuit that analyzes changes in light intensity of individual pixels at the sensor. As this avoids analyzing whole images, it reduces the amount data that needs to be handled and speeds up the process. The method also allows the camera to quickly differentiate between foreground and background objects and gives the vehicle's computer more time to react to situations.
NTU says that the new technology has attracted the interest of private industry and that Chen and his team have spun off a company called Hillhouse Tech to handle commercialization of the camera. According to Chen, the camera may be commercially available by the end of 2017.
"Our new camera can be a great safety tool for autonomous vehicles, since it can see very far ahead like optical cameras but without the time lag needed to analyze and process the video feed," says Chen. "With its continuous tracking feature and instant analysis of a scene, it complements existing optical and laser cameras and can help self-driving vehicles and drones avoid unexpected collisions that usually happen within seconds."
A prototype of the camera was shown at the 2017 IS&T International Symposium on Electronic Imaging (EI 2017) and the research was published by the Institute of Electrical and Electronics Engineers (IEEE).
Source: NTU Singapore
As an aside, if this ever came out in an app on the phone it would be amazing for all sorts of applications. Not lacking CPU on modern devices to deal with this.