How can I visualize TensorRT network graph?

Question

Our current flow: Conversation of tf2->onnx->TensorRT (all16 and 32 and 8 bits options)

Is there an existing tool like https://github.com/lutzroeder/netron (or any other way) to see the output model of TensorRT?

Propagandist

Answer 1

You can now visualize tensorrt engine graphs using https://github.com/NVIDIA/TensorRT/tree/main/tools/experimental/trt-engine-explorer

It also offers other info about the engine.

Lulalulea

Answer 2

Is the goal to visualize it after the compression from onnx/TensorRT? Or just to visualize it in general? You can always just use the keras tool.

I believe you can use a TensorRT model as well with netron, based on this youtube video.

Chuipek

Hey, thanks for the info. But I'm looking for a way to visualize the graph in TensorRT format. We do the optimization inside TensorRT. We do not use the tf_trt binding any more – Propagandist

Answer 3

There is no way to do this, because the TensorRT model (engine) is optimized for certain hardware (A specific NVIDIA GPU architecture). It is already "compiled" (similar to coreML mlmodel's are compiled to mlmodelc, where as TensorFlow (or TFLite) models are not "compiled" for specific hardware.

From the NVIDIA forums: https://forums.developer.nvidia.com/t/visualizing-tensorrt-engine/69032

I assume you are asking for something analogous to tensorboard or netron? Unfortunately, I’m not aware a way for TRT.

From the creator of Netron (lutzroeder):

Unless this format is documented not sure much can be done here. Netron has a browser-only version so even if a Python or C++ API existed for supporting non-inference scenarios it wouldn't be useful.

---

You can still of course visualize the ONNX model right before you create the TensorRT one. TensorRT just needs an optimized model, so I don't expect it to be different.

Zoes

Hey, although its compiled. Nvidia still has the information about the layers themself. Actually when compiling it does brouteforcing to explore different ways of "complication". We already use Nertron to see the output of ONNX. But when converting to int8 it wont be exactly the same. – Propagandist