Nvidia develops software to accelerate trained neural networks

As part of its GPU Technology Conference, Nvidia has announced software that makes trained neural networks faster in the field of inference, for example recognizing objects.

The software is TensorRT 3, the successor to version 2. The new version is intended for use with Nvidia’s Volta GPUs and, according to the manufacturer, should be forty times faster than a system that only runs on CPUs. The Volta GPU is used in the Tesla V100 accelerator, which Nvidia announced in May. With the new software, the accelerator should be 3.7 times faster than the P100 accelerator with Pascal GPU.

Nvidia itself describes the software as a ‘high performance deep learning inference optimizer’. Neural networks must first be trained to perform a certain task, for example recognizing objects. This is done on the basis of large data sets. Once the neural network has been trained, it can be used for that specific task. According to Nvidia, inference is the application of the ‘learned’ to new data.

TensorRT should accelerate this activity, leading to lower latency, for example, in real-time tasks such as analyzing video streams. Before that, the company introduced the so-called Deepstream SDK. Applications are also conceivable in the field of self-driving cars and robots, according to the manufacturer. Competitor AMD presented its own Radeon Instinct accelerators based on Vega GPUs in the summer.

Image via Nvidia