Class PaddlePredictor¶

Defined in File paddle_api.h

Nested Relationships¶

Nested Types¶

Struct PaddlePredictor::Config

Class Documentation¶

class paddle::PaddlePredictor¶

A Predictor for executing inference on a model. Base class for AnalysisPredictor and NativePaddlePredictor.

Public Functions

PaddlePredictor() = default¶

PaddlePredictor(const PaddlePredictor&) = delete¶

PaddlePredictor &operator=(const PaddlePredictor&) = delete¶

virtual bool Run(const std::vector<PaddleTensor> &inputs, std::vector<PaddleTensor> *output_data, int batch_size = -1) = 0¶

This interface takes input and runs the network. There are redundant copies of data between hosts in this operation, so it is more recommended to use the zecopyrun interface.

Parameters

inputs – [in] An list of PaddleTensor as the input to the network.
output_data – [out] Pointer to the tensor list, which holds the output paddletensor
batch_size – [in] This setting has been discarded and can be ignored.

Returns

Whether the run is successful

inline virtual std::vector<std::string> GetInputNames()¶

Used to get the name of the network input. Be inherited by AnalysisPredictor, Only used in ZeroCopy scenarios.

Returns: Input tensor names.

inline virtual std::map<std::string, std::vector<int64_t>> GetInputTensorShape()¶

Get the input shape of the model.

Returns: A map contains all the input names and shape defined in the model.

inline virtual std::vector<std::string> GetOutputNames()¶

Used to get the name of the network output. Be inherited by AnalysisPredictor, Only used in ZeroCopy scenarios.

Returns: Output tensor names.

inline virtual std::unique_ptr<ZeroCopyTensor> GetInputTensor(const std::string &name)¶

Get the input ZeroCopyTensor by name. Be inherited by AnalysisPredictor, Only used in ZeroCopy scenarios. The name is obtained from the GetInputNames() interface.

Parameters: name – The input tensor name.
Returns: Return the corresponding input ZeroCopyTensor.

inline virtual std::unique_ptr<ZeroCopyTensor> GetOutputTensor(const std::string &name)¶

Get the output ZeroCopyTensor by name. Be inherited by AnalysisPredictor, Only used in ZeroCopy scenarios. The name is obtained from the GetOutputNames() interface.

Parameters: name – The output tensor name.
Returns: Return the corresponding output ZeroCopyTensor.

inline virtual bool ZeroCopyRun()¶

Run the network with zero-copied inputs and outputs. Be inherited by AnalysisPredictor and only used in ZeroCopy scenarios. This will save the IO copy for transfering inputs and outputs to predictor workspace and get some performance improvement. To use it, one should call the AnalysisConfig.SwitchUseFeedFetchOp(true) and then use the GetInputTensor and GetOutputTensor to directly write or read the input/output tensors.

Returns: Whether the run is successful

virtual std::unique_ptr<PaddlePredictor> Clone() = 0¶

Clone an existing predictor When using clone, the same network will be created, and the parameters between them are shared.

Returns: unique_ptr which contains the pointer of predictor

virtual ~PaddlePredictor() = default¶: Destroy the Predictor.

inline virtual std::string GetSerializedProgram() const¶

struct Config¶

Base class for NativeConfig and AnalysisConfig.

Subclassed by paddle::NativeConfig

Public Members

std::string model_dir¶: path to the model directory.