As IoT technologies become more mature and widely adopted, the amount of data that devices are typically shipping to the cloud is increasing and becoming more expensive. In order to combat this shift, many are deploying edge computing architectures in order to perform some of the computational work locally at the “edge” of the network. This allows for reduced cloud traffic as well as reduced latency on the edge since pre-analyzed data can be sent to the cloud and feedback sent back to edge devices more quickly.
NVIDIA Jetson BSP challenges when coming to edge AI and machine learning
However, an edge computing device often requires special hardware in order to efficiently and effectively analyze the data coming in, namely a sizeable GPU, TPU (Tensor Processing Unit), or NPU (Neural Processing Unit). As well as having the appropriate capabilities to process data at the edge, the edge computing device must be robust and reliable as it acts as the gateway to the core network. Therefore, it is important that the edge computing device must not only have the appropriate software to properly drive its special hardware (GPU, TPU, or NPU), but the entire BSP must be sufficiently stable in order to continue to perform its integral role in the network.
If you are considering integrating an edge-computing device into your network, chances are you’ve considered using an NVIDIA Jetson platform. The devices in the Jetson family provide powerful hardware for computation, particularly for AI and machine learning tasks, as well as extensive software support that allows for integrating with many existing platforms and libraries, such as TensorFlow and PyTorch. However, you also understand the importance of stability regarding your edge device, so you are likely asking yourself how you can create a stable BSP sufficient for productization for an NVIDIA Jetson platform. The answer to that question is a simple one: Yocto!
Along with the Jetson platforms, NVIDIA provides Jetpack L4T which is an Ubuntu-based BSP with the Jetson kernel and bootloader, Ubuntu filesystem, and all of the CUDA packages which make the Jetson special. Jetpack L4T is a great way to get started developing applications on a Jetson platform. Not only can it make full use of the GPU for machine learning and AI applications off the bat, it also has access to the Canonical aarch64 apt repositories so most tools and libraries can be easily loaded onto the platform.