White Papers

33 CheXNet – Inference with Nvidia T4 on Dell EMC PowerEdge R7425
Figure 12. CheXNet Inference –TF-TRT 5.0 Integration in INT8int8 Precision Mode
Command line to execute the benchmark:
python3 tensorrt_chest.py
--savedmodel_dir=/home/dell/chest-x-ray/chexnet_saved_model/1541777429/ \
--image_file=image.jpg \
--int8 \
--output_dir=/home/dell/chest-x-ray/output_tensorrt_chexnet_1541777429/
--batch_size=1
Docker image for TensorFlow-GPU: nvcr.io/nvidia/tensorflow:18.10-py3
Where: --int8: Benchmark the model with TensorRT™ using int8 precision
Script Output sample:
==========================
network: tftrt_int8_frozen_graph.pb, batchsize 1, steps 100
fps median: 282.2, mean: 315.2, uncertainty: 6.8, jitter: 5.6
latency median: 0.00354, mean: 0.00329, 99th_p: 0.00371, 99th_uncertainty: 0.00008
==========================