Sony combines camera sensor with AI engine and memory for image analysis

Sony introduces its Intelligent Vision Sensor. It concerns a camera sensor with an AI engine and its own memory underneath, so that image analysis can be performed on the chip. The chip does not seem to be intended for smartphones for the time being.

The Sony IMX500 is a 12.3-megapixel sensor of the 1/2.3″ format. It is a stacked sensor, in which the light-sensitive layer is mounted on top of an image processing chip. The bottom layer contains an isp and dsp, for processing and interpretation of the images There is also on-chip memory, in which the AI ​​model to be used can be stored.

Sony presents the sensor as an image analysis chip that does not require an external processor or cloud server. The sensor can send the information in different ways: as an image, or just as metadata. The latter saves a lot of bandwidth. If image analysis in the cloud is done like this, all image information would have to be uploaded first, and then the relevant metadata from the images should be extracted. Sony states that the new sensors also bring privacy benefits, because the images do not have to be stored or sent.

Sony makes two versions; a separate sensor and a complete package. The latter is the IMX501. The separate chip has been made available from April and costs 10,000 Japanese yen. The package version will be released in June and will cost 20,000 yen. Converted this concerns amounts of about 87 and 174 euros.

The chip is intended for IoT equipment, security cameras and industrial applications. Whether Sony plans to bring such chips to smartphones is not known. Current smartphones use socs that contain a special neural processing unit for calculations with artificial intelligence. These NPUs are also used for image processing.