White Papers

9 CheXNet – Inference with Nvidia T4 on Dell EMC PowerEdge R7425
Table 1 shows the summary of the project design below:
Table 1:Project Design Summary
Element
Description
Use Case:
Optimized Inference Image Classification with
TensorFlow and TensorRT™
Models:
Custom Model CheXNet and base model ResnetV2_50
Framework:
TensorFlow 1.0
TensorRT™
version:
TensorRT™ 5.0
TensorRT™
implementations:
TensorFlow-TensorRT Integration (TF-TRT) and
TensorRT C++ API (TRT)
Precision
Modes:
▪ Native TensorFlow FP32 – CPU Only
▪ Native TensorFlow FP32 - GPU
▪ TF-TRT-FP32
▪ TF-TRT-FP16
▪ TF-TRT-INT8
▪ TRT-INT8
Performance:
Throughput (images per second) and the Latency (msec)
Dataset:
NIH Chest X-ray Dataset from the National Institutes of
Health
Samples code:
TensorRT™ samples provided by Nvidia included on its
container images, and adapted to run the optimized
inference of the custom model
Software stack
configuration:
Tests conducted using the docker container
environment
Server:
Dell EMC PowerEdge R7425
Table 2 lists the tests conducted to train the model, and inferences in different precision
modes with the TensorRT™ implementations. The script samples can be found within the
Nvidia container image.
Table 2. Tests Conducted
Model/Inference Mode
TensorRT™ Implementation
Test script
Custom Model
n/a
chexnet.py
Native TensorFlow CPU FP32
n/a
tensorrt_chexnet.py
Native TensorFlow GPU
FP32
n/a
tensorrt_chexnet.py
Integration TF-TRT5 FP32
TF-TRT Integration
tensorrt_chexnet.py
Integration TF-TRT5 FP16
TF-TRT Integration
tensorrt_chexnet.py
Integration TF-TRT5 INT8
TF-TRT Integration
tensorrt_chexnet.py
Native TRT5 INT8 – C++ API
C++ API
trtexec.cpp