ONNX (Open Neural Network Exchange) is an open standard for representing machine learning and deep learning models, allowing for interoperability between various frameworks and tools. Many popular machine learning libraries, frameworks, and tools support ONNX, either natively or through additional packages or converters. Some of these tools include:
ONNX Runtime: A cross-platform, high-performance inference engine for ONNX models, developed by Microsoft. It allows you to run ONNX models on various platforms and hardware, including CPUs, GPUs, and edge devices.
PyTorch: A popular deep learning framework that natively supports exporting models to the ONNX format using the torch.onnx.export() function. PyTorch also supports importing ONNX models for inference.
TensorFlow: TensorFlow supports ONNX through the tf2onnx package, which provides a converter to convert TensorFlow models to the ONNX format. Additionally, the ONNX-TensorFlow project provides an ONNX backend for TensorFlow, enabling the execution of ONNX models within the TensorFlow framework.
MXNet: Apache MXNet, another deep learning framework, supports ONNX through the mxnet.contrib.onnx module, which provides utilities to import and export ONNX models.
Caffe2: Caffe2, a deep learning framework that has been merged into PyTorch, supports ONNX natively. You can import and export ONNX models using the Caffe2-ONNX converter.
Apple Core ML: Apple's Core ML framework for running machine learning models on iOS and macOS devices supports ONNX through the onnx-coreml package, which provides a converter for transforming ONNX models into Core ML models.
Microsoft ML.NET: ML.NET, a cross-platform, open-source machine learning framework developed by Microsoft, supports ONNX for model inference, enabling the use of ONNX models in .NET applications.
OpenCV: The popular open-source computer vision library OpenCV supports ONNX models for deep learning inference through its cv::dnn module.
These are just a few examples of the tools and frameworks that support ONNX. The ONNX ecosystem is continuously growing, with more tools and libraries adopting the format to promote interoperability and simplify the deployment of machine learning models across different platforms and hardware.
ONNX (Open Neural Network Exchange) files store machine learning and deep learning models in a binary format that represents the model's architecture, learned parameters, and metadata. The files themselves are not human-readable, as they are serialized using Protocol Buffers, a language-agnostic and platform-neutral binary format developed by Google.
An ONNX file contains information about the model's graph, including the nodes, edges, inputs, and outputs. It also stores the model's parameters (weights and biases) as tensors and provides metadata, such as the model's version, producer information, and domain.
Although the binary ONNX files are not human-readable, there are tools available to visualize and explore the content of an ONNX file. Some of these tools include:
Netron: An open-source, cross-platform viewer for deep learning models, including ONNX, TensorFlow, PyTorch, and many others. Netron provides a graphical representation of the model's structure and lets you inspect the model's nodes, layers, and parameters.
ONNX.js: A JavaScript library for running ONNX models in web browsers. It provides a Web API to load and visualize ONNX models, as well as to perform inference directly in the browser.
ONNX Python API: The ONNX Python API allows you to load ONNX files, inspect their content programmatically, and perform various operations on the model, such as simplifying, optimizing, or converting the model to different formats.
In summary, while ONNX files are binary files that are not directly human-readable, there are tools available to visualize, explore, and manipulate the content of these files to better understand the model's architecture and parameters.
Researchers from Intel Labs have developed an open-source 3D printed smart robot that it is possible to build for less than $50.
Once combined with a modern smartphone, the additive bot is able to navigate autonomously, and its real-time sensing capabilities enable it to conduct recon tasks such as following people. The PLA-based Openbot has also been designed to be easy to build too, and its STL files have been made freely available online. By making their design public, the researchers hope that creators will tinker with and improve their robot to continue developing new applications for it in the future.
“Current robots are either expensive or make significant compromises on sensory richness, computational power, and communication capabilities,” stated the researchers in their paper. “We propose to leverage smartphones to equip robots with extensive sensor suites and access to a thriving software ecosystem, in a small electric vehicle that costs $50.”
The herculean cost of additive smart robotics
The industrial applications of smart robots have created a potentially lucrative market for businesses such as Franka Emika and Clearpath. The advanced tech firms produce highly-sophisticated legged-robots and bionic arms, but these are often well out of reach for academic institutions, costing north of $10,000 each.
Previously developed low-cost four-wheeled alternatives can broadly be sorted into three classes: tiny robots, swarm robotics and larger vehicles based on RC trucks. Swarm bots are more economical to make, but they often feature very limited sensing capabilities. By contrast, larger custom-made creations have more abundant sensors, but it has proved difficult to develop autonomous navigation for them.
“ROBOTS ARE EXPENSIVE. LEGGED ROBOTS AND INDUSTRIAL MANIPULATORS COST AS MUCH AS LUXURY CARS, AND THE CHEAPEST ROBOTS FROM FRANKA EMIKA OR CLEARPATH COST AT LEAST $10K.”
Many modern smartphones come equipped with mobile AI processors, and their ability to run complex AI models in real-time has led researchers to experiment with a smart robotic cross-over. Similar projects to that of the Intel team have previously been launched on Kickstarter, with the Botiful telepresence device and the Romo smartphone-powered drone finding success on the platform.
The Wheelphone design, which was popularized on the open-source software site Github a few years ago, probably represents the most similar layout to that developed by the Intel researchers. Unfortunately the technical limitations of the Wheelphone limited its top speed, and its high cost of around $250 caused it to be shelved, with the last update released around two years ago.
The chassis of the Intel team’s Openbot can be created in its entirety using 3D printing. Image via Intel.
The Intel duo’s 3D printed Openbot
Inspired by projects such as Google’s Cardboard VR device, the researchers hypothesized that a smartphone could play a similar role in robotics. Using off-the-shelf parts to create a cheap physical enclosure, while allowing the mobile to act as its ‘brain,’ has plenty of benefits. In addition to being more cost-effective, the vibrant ecosystem of existing mobile apps offer the potential for numerous applications to be developed using the same hardware.
Using only an Ultimaker S5 system, the team produced the chassis, consisting of a bottom plate which is fastened to a top cover with six screws. The top plate itself features a universal smartphone mount with an adjustable spring to make it compatible with different phones, and an opening for attaching a USB-powered Arduino microcontroller.
The Openbot was programmed using a two-piece custom-designed software stack. In order to provide an interface for the operator to collect datasets and control the bot’s workload, the Intel team developed an android-based app. This was supplemented by an Arduino nano microcontroller, which controls low-level functions, and acts as a bridge between the vehicle body and the smartphone.
The bluetooth and networking capabilities of modern phones could be utilized to connect the Openbot to other devices such as game pads. Image via Intel.
Taking a total of 23 hours to build, the smart robot proved capable of speeds of up to 150cm per second, making it markedly quicker than its predecessors. Aiming to assess the sensing capabilities of their Openbot, the researchers tested to see if it could follow someone independently, or even achieve autonomous navigation. Using mid-range smartphones, testing showed that the bot was able to track people at speeds of 10 fps or higher.
Although performance did prove to be device-dependent, even the cheapest phones were able to follow people for around 50 percent of the time. The researchers also trained a “driving policy” for use on smartphones, which was capable of navigating corridors and turning at intersections. Overall, the bot proved consistently able to deal with previously unseen environments, including novel objects, and dynamic obstacles such as people.
Concluding their tests to be a success, the Intel duo expressed optimism that their creation would make robotics more scalable and accessible than was previously thought possible.
“We have developed a combination of hardware and software that turns smartphones into robots,” they concluded in their paper. “We hope our work will open new opportunities for education and large-scale learning via thousands of low-cost robots deployed around the world.”
For those interested in building the Intel research team’s Openbot themselves, the files can be accessed via Github here.
Previous 3D printed smart robots
A number of other research groups have leveraged 3D printing to fabricate smart robotics, and these additive creations have taken on a vast array of shapes, sizes and functions in recent years.
In July 2020, engineering company Scheurer Swiss GmbH partnered with a group of ETH Zürich students to develop their “Rowesys” 3D printed automated robotic weeding system. The compact weed killing bot is designed to be a sustainable alternative to using herbicides in agricultural farming.
The researchers’ findings are detailed in their paper titled “OpenBot: Turning Smartphones into Robots,” which was co-authored by Matthias Muller and Vladlen Koltun.
OpenBot: Turning Smartphones into Robots
Matthias Muller ¨
Intel Labs
Vladlen Koltun
Intel Labs
Abstract: Current robots are either expensive or make significant compromises on
sensory richness, computational power, and communication capabilities. We propose to leverage smartphones to equip robots with extensive sensor suites, powerful computational abilities, state-of-the-art communication channels, and access
to a thriving software ecosystem. We design a small electric vehicle that costs
$50 and serves as a robot body for standard Android smartphones. We develop a
software stack that allows smartphones to use this body for mobile operation and
demonstrate that the system is sufficiently powerful to support advanced robotics
workloads such as person following and real-time autonomous navigation in unstructured environments. Controlled experiments demonstrate that the presented
approach is robust across different smartphones and robot bodies.
The Intel® Movidius™ Myriad™ X VPU is Intel's first VPU to feature the Neural Compute Engine — a dedicated hardware accelerator for deep neural network inference. The Neural Compute Engine in conjunction with the 16 powerful SHAVE cores and high throughput intelligent memory fabric makes Movidius Myriad X ideal for on-device deep neural networks and computer vision applications.
The Movidius Myriad X VPU is programmable with the Intel® Distribution of the OpenVINO™ toolkit for porting neural network to the edge, and via the Myriad Development Kit (MDK) which includes all necessary development tools, frameworks and APIs to implement custom vision, imaging and deep neural network workloads on the chip.
Open Source Spatial AI From The Biggest Name in Computer Vision.
To celebrate OpenCV’s 20th anniversary we are proud to introduce the OpenCV AI Kit (OAK), an MIT-licensed open source software and Myriad X-based hardware solution for computer vision at any scale.
OAK consists of the OAK API software and two different types of hardware: OAK-1 and OAK-D. They are tiny artificial intelligence (AI) and computer vision (CV) powerhouses, with OAK-D providing spatial AI leveraging stereo depth in addition to the 4K/30 12MP camera that both models share. They are also both absurdly easy to use. Up and running in under 30 seconds, OAK-1 and OAK-D allow anyone to access this power: hobbyists, researchers, and professionals alike. Once you're done tinkering, OAK's modular, FCC/CE-approved, open-source hardware ecosystem affords direct integration into your products.
