White Papers

25 CheXNet – Inference with Nvidia T4 on Dell EMC PowerEdge R7425
Converting A Frozen Graph To UFF:
An existing model built with TensorFlow can be used to build a TensorRT™ engine.
Importing from the TensorFlow framework requires to convert the TensorFlow model into the
intermediate format UFF file. To do so, we used the tool convert_to_uff.py located at the
directory /usr/lib/python3.5/dist-packages/uff/bin, which uses as an input a frozen model,
below the command to convert .pb TensorFlow frozen graph to .uff format file:
python3 convert_to_uff.py \
input_file /home/chest-x-ray/chexnet_frozen_graph_1541777429.pb
Create the builder, network, and UFF parser
//1-Create the builder and network:
IBuilder* builder = createInferBuilder(gLogger);
INetworkDefinition* network = builder->createNetwork();
//2-Create the UFF parser:
IUFFParser* parser = createUffParser();
//3-Declare the network inputs and outputs to the UFF parser:
parser->registerInput("input_tensor", DimsCHW(3,256,256), UffInputOrder::kNCHW);
parser->registerOutput("chexnet_sigmoid_tensor");
//Parse the imported model to populate the network:
parser->parse(uffFile, *network, nvinfer1::DataType::kFLOAT);
For the network definition, it is important to directly specify to TensorRT™ which tensors are
inputs and their dimensions, as well as specify which tensors are outputs for inference (inputs
and output tensors must also be given names); the rest of the tensors are transient values
that may be optimized by the builder.
UFF Parser is used to parse a network in UFF format. For more details on the C++ UFF
Parser, see NvUffParser or the Python UFF Parser [16].
TensorRT™ C++ API expects the input tensor to be in channel first order (CHW). When
importing from TensorFlow, the input tensor is required to be in this format in order to achieve