Synaptics Incorporated : Proliferating AI further into our connected world requires micro-power IoT solutions

Proliferating AI further into our connected world requires micro-power IoT solutions

Jan 22, 2021

By Vineet Ganju

In our connected world, AI-enriched devices and systems have dramatically improved the convenience, enjoyment and safety of how we live, work and move about. These AI capabilities are enabled by powerful processors driving innovative algorithms that embed intelligence on edge devices and process data locally for a wide range of smart applications for home automation, office, industrial, agriculture, security, and entertainment. This is largely possible because they are all powered by access to a consistent and sustained wired energy source where they operate.

The next wave of AI-enriched IoT will untether devices from the power grid and rely on locally-supplied energy - primarily batteries - but perhaps other sources of energy harvesting as well. This type of sensing reach has become even more important in our Covid world as we look for ways to continuously monitor people and places to ensure health and safety.

Overall, the goal with micropower AI is to bring machine learning to the sensor endpoint, where the data is generated, regardless of location or energy source. With the limited power budgets these IoT endpoints have, power consumption of the chip has to be less than a milliwatt.

While this opens up a vast new realm of possibilities for remote intelligence, it significantly changes the requirements from a processing, efficiency, and even physical form factor point of view. Most important is how to efficiently perform the critical AI inferencing that traditionally demands large amounts of power.

Synaptics expands solution offering for Edge AI
As a leader in enabling AI functionality for vision, audio and voice in 'traditional' IoT systems such as set-top boxes, smart speakers and smart displays, Synaptics is now applying our expertise to the world of ultra-low-power IoT, enabling micropower AI. Our efforts will have a real impact on proliferating more, and smarter, sensing capabilities in places that either don't have access to power or where installing plugged-in systems would be too costly. Think temporary people counting to avoid large concentrations of people in offices or restaurants; smart city monitoring to assist with traffic and transportation issues; and real-time inventory monitoring in retail settings.

Our recently-announced Katana Edge AI^™ platform addresses what we see as a growing industry gap for solutions that enable such battery-powered devices for consumer and industrial IoT markets. This platform combines Synaptics' proven low power SoC architecture with energy-efficient AI software, enabled by a partnership with Eta Compute, the leader in energy-efficient endpoint AI solutions for intelligent sensing anywhere.

With this new offering we are taking aim at the needs of ultra-low power use cases in edge devices for office buildings, retail shops, factories, farms, cities and smart homes. Typical applications include people or object recognition and counting, visual, voice or sound detection, asset or inventory tracking and environmental sensing.

Like our more power intensive Edge AI solutions, Katana features a multi-core processor architecture capable of running the demanding AI algorithms required by smart devices. Katana is specifically developed to be optimized for ultra-low-power and low latency voice, audio and vision applications. The full system SoC combines proprietary power and energy-optimized neural network and domain specific processing cores, significant on-chip memory, and extensive use of multiple architectural techniques that save power for each unique mode of operation.

Partnership with Eta Compute for optimal power efficiency
A significant addition to our offering comes through our partnership with Eta Compute. The growing demand for efficiency in battery-operated devices requires software optimization techniques tightly coupled to the underlying silicon. Eta Compute's TENSAI Flow compiler is the crown jewel here: it can get an order of magnitude better energy efficiency than other compilers on the market, or proprietary ones.

This is done through a variety of memory optimization techniques, as well as performance and energy optimization. But the software must be optimized for the specific target processor. In other words, optimization of neural networks for specific applications that can increase power efficiency by an order of magnitude compared to designs from the standard TensorFlow framework.

In addition, Eta Compute's TENSAI Flow software de-risks development by quickly confirming feasibility and proof-of-concept and enables seamless development for machine learning applications in IoT and low power edge devices. It includes a neural network compiler, a neural network zoo, and middleware comprising FreeRTOS, HAL and frameworks for sensors, as well as IoT/cloud enablement. As a result of our collaboration, Synaptics' Katana SoC will be co-optimized with the TENSAI Flow software to create a complete platform that combines the industry's most efficient AI compiler with an extensive set of performance- and power-optimized libraries.

Ted Tewksbury, CEO of Eta Compute, told EE Times: 'What we heard from the market very clearly is that you need a very easy to use, push button approach to interface these complex processors with TensorFlow, PyTorch and other AI frameworks. We're working very closely with Synaptics to do co-optimization of the hardware and software to provide the most efficient code on the Synaptics Katana processor.'

We will also be working with Eta Compute to offer application focused kits that speed development and deployment. The kits will include pre-trained machine learning models and reference designs, while also enabling users to train the models with their own datasets using industry-standard frameworks such as TensorFlow, Caffe and ONNX.

We're excited about the vast potential ultra-low power IoT holds for all of us and believe the Katana platform will play an important role in further proliferating AI-enriched devices to the world.