GpuDelegateFactory.Options

public static class GpuDelegateFactory.Options
Known Direct Subclasses

Delegate options.

Nested Classes

enum GpuDelegateFactory.Options.GpuBackend Which GPU backend to select. 

Constants

int INFERENCE_PREFERENCE_FAST_SINGLE_ANSWER Delegate will be used only once, therefore, bootstrap/init time should be taken into account.
int INFERENCE_PREFERENCE_SUSTAINED_SPEED Prefer maximizing the throughput.

Public Constructors

Options()

Public Methods

boolean
areQuantizedModelsAllowed()
GpuDelegateFactory.Options.GpuBackend
getForceBackend()
int
getInferencePreference()
String
getModelToken()
String
getSerializationDir()
boolean
isPrecisionLossAllowed()
GpuDelegateFactory.Options
setForceBackend(GpuDelegateFactory.Options.GpuBackend forceBackend)
Sets the GPU Backend.
GpuDelegateFactory.Options
setInferencePreference(int preference)
Sets the inference preference for precision/compilation/runtime tradeoffs.
GpuDelegateFactory.Options
setPrecisionLossAllowed(boolean precisionLossAllowed)
Sets whether precision loss is allowed.
GpuDelegateFactory.Options
setQuantizedModelsAllowed(boolean quantizedModelsAllowed)
Enables running quantized models with the delegate.
GpuDelegateFactory.Options
setSerializationParams(String serializationDir, String modelToken)
Enables serialization on the delegate.

Inherited Methods

Constants

public static final int INFERENCE_PREFERENCE_FAST_SINGLE_ANSWER

Delegate will be used only once, therefore, bootstrap/init time should be taken into account.

Constant Value: 0

public static final int INFERENCE_PREFERENCE_SUSTAINED_SPEED

Prefer maximizing the throughput. Same delegate will be used repeatedly on multiple inputs.

Constant Value: 1

Public Constructors

public Options ()

Public Methods

public boolean areQuantizedModelsAllowed ()

public GpuDelegateFactory.Options.GpuBackend getForceBackend ()

public int getInferencePreference ()

public String getModelToken ()

public String getSerializationDir ()

public boolean isPrecisionLossAllowed ()

public GpuDelegateFactory.Options setForceBackend (GpuDelegateFactory.Options.GpuBackend forceBackend)

Sets the GPU Backend.

Parameters
forceBackend

public GpuDelegateFactory.Options setInferencePreference (int preference)

Sets the inference preference for precision/compilation/runtime tradeoffs.

Parameters
preference One of `INFERENCE_PREFERENCE_FAST_SINGLE_ANSWER` (default), `INFERENCE_PREFERENCE_SUSTAINED_SPEED`.

public GpuDelegateFactory.Options setPrecisionLossAllowed (boolean precisionLossAllowed)

Sets whether precision loss is allowed.

Parameters
precisionLossAllowed When `true` (default), the GPU may quantify tensors, downcast values, process in FP16. When `false`, computations are carried out in 32-bit floating point.

public GpuDelegateFactory.Options setQuantizedModelsAllowed (boolean quantizedModelsAllowed)

Enables running quantized models with the delegate.

WARNING: This is an experimental API and subject to change.

Parameters
quantizedModelsAllowed When true (default), the GPU may run quantized models.

public GpuDelegateFactory.Options setSerializationParams (String serializationDir, String modelToken)

Enables serialization on the delegate. Note non-null serializationDir and modelToken are required for serialization.

WARNING: This is an experimental API and subject to change.

Parameters
serializationDir The directory to use for storing data. Caller is responsible to ensure the model is not stored in a public directory. It's recommended to use Context.getCodeCacheDir() to provide a private location for the application on Android.
modelToken The token to be used to identify the model. Caller is responsible to ensure the token is unique to the model graph and data.