Option object to configure TensorRT backend. More...

#include <option.h>

Public Member Functions
void	SetShape (const std::string &tensor_name, const std::vector< int32_t > &min, const std::vector< int32_t > &opt, const std::vector< int32_t > &max)
	Set shape range of input tensor for the model that contain dynamic input shape while using TensorRT backend. More...

Public Attributes
size_t	max_batch_size = 32
	`max_batch_size`, it's deprecated in TensorRT 8.x

size_t	max_workspace_size = 1 << 30
	`max_workspace_size` for TensorRT

bool	enable_log_info = false
	Enable log while converting onnx model to tensorrt.

bool	enable_fp16 = false
	Enable half precison inference, on some device not support half precision, it will fallback to float32 mode.

std::string	serialize_file = ""
	Set cache file path while use TensorRT backend. Loadding a Paddle/ONNX model and initialize TensorRT will take a long time, by this interface it will save the tensorrt engine to `cache_file_path`, and load it directly while execute the code again.

Detailed Description

Option object to configure TensorRT backend.

Member Function Documentation

void fastdeploy::TrtBackendOption::SetShape	(	const std::string &	tensor_name,
		const std::vector< int32_t > &	min,
		const std::vector< int32_t > &	opt,
		const std::vector< int32_t > &	max
	)

inline

Set shape range of input tensor for the model that contain dynamic input shape while using TensorRT backend.

Parameters

[in]	tensor_name	The name of input for the model which is dynamic shape
[in]	min	The minimal shape for the input tensor
[in]	opt	The optimized shape for the input tensor, just set the most common shape, if set as default value, it will keep same with min_shape
[in]	max	The maximum shape for the input tensor, if set as default value, it will keep same with min_shape

The documentation for this struct was generated from the following file: