NVIDIA® JETSON AGX ORIN™ DEVELOPER KIT Next-Level AI Performance for Next-Gen Robotics Reviewer’s Guide 1
Jetson AGX™ Orin Developer Kit Reviewer's Guide Introducing NVIDIA® Jetson AGX Orin™ 3 6 Best in Class Performance Up to 8X Higher AI Performance 6 Significant Performance Speedups for Real-world AI Workloads 6 Performance on Vision AI and Conversational AI Models 8 8 Faster Development with the NVIDIA AI Software Platform Appendix Getting Started with the Jetson AGX Orin Developer Kit 12 12 Booting up your Jetson AGX Orin Developer Kit 12 Exploring the Developer Kit 12 Running Inference Benc
Jetson AGX™ Orin Developer Kit Reviewer's Guide Introducing NVIDIA® Jetson AGX Orin™ NVIDIA Accelerated Computing has revolutionized the growth of the AI industry with high performance GPUs, SoCs, and optimized software stacks that are widely used across industries spanning cloud, data centers, and edge computing. NVIDIA Jetson platform was specifically created to deliver the benefits of AI computing to embedded edge devices.
Jetson AGX™ Orin Developer Kit Reviewer's Guide The Jetson AGX Orin Developer Kit contains everything needed for developers to get up and running quickly. The Jetson AGX Orin Developer Kit is priced at $1999 and is available for purchase through NVIDIA authorized distributors worldwide. Developer kit contents ➢ ➢ ➢ ➢ ➢ Jetson AGX Orin module with heat sink and reference carrier board 802.
Jetson AGX™ Orin Developer Kit Reviewer's Guide Jetson AGX Orin Developer Kit features MODULE: GPU CPU DL Accelerator Vision Accelerator Memory Storage Power NVIDIA Ampere architecture with 2048 NVIDIA® CUDA® cores and 64 Tensor cores 12-core Arm Cortex-A78AE v8.2 64-bit CPU 3MB L2 + 6MB L3 2x NVDLA v2.0 PVA v2.0 32GB 256-bit LPDDR5 204.8 GB/s 64GB eMMC 5.
Jetson AGX™ Orin Developer Kit Reviewer's Guide Best in Class Performance Up to 8X Higher AI Performance The power-efficient Jetson AGX Orin System-on-Module (SoM) delivers up to 275 TOPS1 of AI performance within a 60-Watt power budget, an 8X improvement over the 32 TOPS delivered by Jetson AGX Xavier. For designs requiring lower power profiles, customers can tune their designs for power profiles ranging from 15W to 60W.
Jetson AGX™ Orin Developer Kit Reviewer's Guide Jetson AGX Orin delivers almost 3.5X the performance of Jetson AGX Xavier on popular pre-trained neural networks that are used for object detection, action recognition, pose estimation, and others. The new Sparsity feature introduced in the Ampere GPU architecture and available on Jetson AGX Orin, will help further boost the performance advantage.
Jetson AGX™ Orin Developer Kit Reviewer's Guide Performance on Vision AI and Conversational AI Models Table 3 Jetson AGX Orin Performance on widely used Vision and Conversational AI models Instructions for running the above benchmarks on Jetson AGX Orin and Jetson AGX Xavier are available in Running Inference Benchmarks section of Appendix Faster Development with the NVIDIA AI Software Platform The class-leading performance and energy efficiency of Jetson AGX Orin is backed by the same powerful NVIDIA A
Jetson AGX™ Orin Developer Kit Reviewer's Guide training and quickly create highly accurate production ready models. SDKs that are deployed in powerful datacenters, such as RIVA for accelerated conversational AI and Deepstream for accelerated vision AI, also run on the Jetson AGX Orin. NVIDIA Isaac SDK delivers the GPU-accelerated solutions for Jetson-based robotic applications.
Jetson AGX™ Orin Developer Kit Reviewer's Guide NVIDIA TAO (Train-Adapt-Optimize) is a framework that lets developers create custom, productionready models, in hours rather than months, without AI expertise or large training datasets. The NVIDIA TAO Toolkit abstracts away the AI/deep learning framework complexity, letting you fine-tune on highquality NVIDIA pre-trained AI models with only a fraction of the data compared to training from scratch.
Jetson AGX™ Orin Developer Kit Reviewer's Guide NVIDIA Fleet Command™ is a cloud service that securely deploys, manages, and scales AI applications across distributed edge infrastructure. Purpose-built for AI, Fleet Command is a turnkey solution for AI lifecycle management. It removes the complexity of building and maintaining an edge software platform by offering streamlined deployments, OTA updates, and detailed monitoring capabilities.
Jetson AGX™ Orin Developer Kit Reviewer's Guide Appendix Getting Started with the Jetson AGX Orin Developer Kit You can access this Reviewers Guide from https://developer.nvidia.com/jetson-agx-orin-review Booting up your Jetson AGX Orin Developer Kit The Jetson AGX Orin Developer Kit comes pre-flashed with Jetson Linux BSP. Just connect the power supply, keyboard, mouse, display and power on. Also make sure to connect Jetson to the internet over the built-in Wi-Fi or ethernet before you proceed further.
Jetson AGX™ Orin Developer Kit Reviewer's Guide • • • • • • • • CUDA 11.4 cuDNN 8.3.2 TensorRT 8.4.0 OpenCV 4.5.4 Vulkan 1.3 VPI 2.0 Nsight Systems 2021.5 Nsight Graphics 2021.5 On top right of the desktop, there is a power profile selector. When clicked, it provides a drop-down menu of all software defined power modes for the developer kit.
Jetson AGX™ Orin Developer Kit Reviewer's Guide Running Inference Benchmarks Download the tar ball named “benchmarks.tar.gz” required for this benchmarking from here and move it inside the Review Directory. And untar the tar ball: mkdir $HOME/Review cd $HOME/Review Move benchmarks.tar.gz to inside the $HOME/Review and then untar it: tar -xvf benchmarks.tar.gz Setup the requirements for running the benchmark by doing: cd benchmarks bash installBenchmarks.
Jetson AGX™ Orin Developer Kit Reviewer's Guide Loading eval dataset... filename: /benchdata/1272-135031-0000.wav Done loading 1 files Latencies (ms): Median 90th 95th 9.2084 9.3835 9.5279 99th 14.065 Avg 9.4069 Intermediate latencies (ms): Median 90th 9.2061 9.3615 95th 9.4755 99th 9.6975 Avg 9.3387 Final latencies (ms): Median 90th 95th 14.065 14.131 14.131 99th 14.131 Avg 14.041 Run time: 54.441 sec. Total audio processed: 54.425 sec. Throughput: 0.
Jetson AGX™ Orin Developer Kit Reviewer's Guide First audio - P99: 0.0249245 Chunk - average: 0.00473509 Chunk - P90: 0.00488245 Chunk - P95: 0.00489093 Chunk - P99: 0.0058794 Throughput (RTF): 50.8159 At the end stop benchmarking using: bash stopBenchmark.sh NVIDIA TAO NVIDIA provides pre-trained, production-ready neural network models on the NVIDIA GPU Cloud (NGC). For each model, NVIDIA provides a deployable model and a trainable model.
Jetson AGX™ Orin Developer Kit Reviewer's Guide cd $HOME/Review/TAO-PTM bash installTAOPTM.sh Installation will take couple of minutes. After installation is complete, start the jupyter-lab by following the commands below: cd $HOME/Review/TAO-PTM/ export PATH="$HOME/.local/bin/:$PATH" jupyter-lab Click on the link in the end of the output of the above command and it will open Jupyter notebook on the browser.
Jetson AGX™ Orin Developer Kit Reviewer's Guide Download the tar ball named “Riva.tar.gz” required for this demo from here and move it inside the Review Directory. cd $HOME/Review Move “Riva.tar.gz” to inside the $HOME/Review and then untar it: tax -xvf Riva.tar.gz Install the demo using: cd $HOME/Review/Riva bash installRivaASR.sh Once the installation is done, you can experience the following RIVA transcription demo: Please remove the USB Camera and only have the USB Headset with mic connected.
Jetson AGX™ Orin Developer Kit Reviewer's Guide Additional Resources Pytorch Wheels We have hosted Pytorch wheels here: https://nvidia.box.com/shared/static/19je2l0ppy1fpq4mw1a5gsbb5y9fopy7.whl For installation, please do the following steps: wget https://nvidia.box.com/shared/static/19je2l0ppy1fpq4mw1a5gsbb5y9fopy7.whl -O torch-1.10.0-cp38-cp38-linux_aarch64.whl sudo apt-get install python3-pip libopenblas-base libopenmpi-dev pip3 install Cython pip3 install numpy torch-1.10.0-cp38-cp38-linux_aarch64.
Jetson AGX™ Orin Developer Kit Reviewer's Guide NVIDIA Jetson AGX Orin Specs Jetson AGX Xavier Jetson AGX Orin AI Performance 32 TOPS (INT8) 275 TOPS (INT8 with Sparsity) |138 TOPS (INT8) GPU 512-core NVIDIA Volta GPU with 64 Tensor Cores 2048-core NVIDIA Ampere GPU with 64 Tensor Cores DL Accelerator 2x NVDLA 2x NVDLA v2 Vision Accelerator 2x PVA v1 PVA v2 CPU 8-core NVIDIA Carmel Arm®v8.2 64-bit CPU 12-core NVIDIA Arm® Cortex A78AE v8.
Jetson AGX™ Orin Developer Kit Reviewer's Guide NVIDIA Contact Information Any questions when reviewing the Developer Kit? Email JAOReviewersTeam@nvidia.com NVIDIA North/Latin America Public Relations David Pinto PR Manager, Autonomous Machines Office: 408 566 6950 dpinto@nvidia.com Sridhar Ramaswamy Director, Enterprise and SHIELD Technical Marketing Cell: 510 545 3774 sramaswamy@nvidia.com Michael Lim Director, Analyst Relations Office: 408 486 2376 mlim@nvidia.
Jetson AGX™ Orin Developer Kit Reviewer's Guide NVIDIA Europe Public Relations Jens Neuschäfer Senior PR Manager, Enterprise Europe Office: +49 89 6283 50015 Mobile: +49 173 6282912 jneuschafer@nvidia.com NVIDIA GmbH Haus 1 West, 3rd Floor Flössergasse 2 81369 Munich, Germany Rick Napier Senior Technical Product Manager Northern Europe Office: +44 (118) 9184378 Mobile: +44 (7917) 630172 rnapier@nvidia.
Jetson AGX™ Orin Developer Kit Reviewer's Guide NVIDIA Asia/Pacific Public Relations Jeff Yen Director, Technical Marketing, APAC Melody Tu Senior PR / Marketing Manager, APAC Office : +886 987 263 193 jyen@nvidia.com Office: +886 2 6605 5856 metu@nvidia.com NVIDIA 8, Kee Hu Road, Neihu Taipei 114 TAIWAN NVIDIA TASA (TW/AU/SEA) 8, Kee Hu Road, Neihu Taipei 114 TAIWAN Searching Shi Sr.
Jetson AGX™ Orin Developer Kit Reviewer's Guide John Gillooly Technical Marketing Manager, Asia Pacific South Titus Su Technical Marketing Manager, TASA Office: +65 8286 8727 Email: jgillooly@nvidia.com SINGAPORE Office: +886 2 6605 5430 Email: tisu@nvidia.
Jetson AGX™ Orin Developer Kit Reviewer's Guide Notice ALL INFORMATION PROVIDED IN THIS REVIEWER’S GUIDE, INCLUDING COMMENTARY, OPINION, NVIDIA DESIGN SPECIFICATIONS, REFERENCE BOARDS, FILES, DRAWINGS, DIAGNOSTICS, LISTS, AND OTHER DOCUMENTS (TOGETHER AND SEPARATELY, “MATERIALS”) ARE BEING PROVIDED “AS IS.
