ObjectDetector

public final class ObjectDetector

Performs object detection on images.

The API expects a TFLite model with TFLite Model Metadata..

The API supports models with one image input tensor and four output tensors. To be more specific, here are the requirements.

  • Input image tensor (kTfLiteUInt8/kTfLiteFloat32)
    • image input of size [batch x height x width x channels].
    • batch inference is not supported (batch is required to be 1).
    • only RGB inputs are supported (channels is required to be 3).
    • if type is kTfLiteFloat32, NormalizationOptions are required to be attached to the metadata for input normalization.
  • Output tensors must be the 4 outputs of a DetectionPostProcess op, i.e:
    • Location tensor (kTfLiteFloat32):
      • tensor of size [1 x num_results x 4], the inner array representing bounding boxes in the form [top, left, right, bottom].
      • BoundingBoxProperties are required to be attached to the metadata and must specify type=BOUNDARIES and coordinate_type=RATIO.
    • Classes tensor (kTfLiteFloat32):
      • tensor of size [1 x num_results], each value representing the integer index of a class.
      • if label maps are attached to the metadata as TENSOR_VALUE_LABELS associated files, they are used to convert the tensor values into labels.
    • scores tensor (kTfLiteFloat32):
      • tensor of size [1 x num_results], each value representing the score of the detected object.
    • Number of detection tensor (kTfLiteFloat32):
      • integer num_results as a tensor of size [1].

An example of such model can be found on TensorFlow Hub..

Nested Classes

class ObjectDetector.ObjectDetectorOptions Options for setting up an ObjectDetector

Public Methods

static ObjectDetector
createFromBuffer(Context context, ByteBuffer modelBuffer)
Creates an ObjectDetector instance from a model buffer and the default ObjectDetector.ObjectDetectorOptions.
static ObjectDetector
createFromFile(Context context, String modelPath)
Creates an ObjectDetector instance from a model file and the default ObjectDetector.ObjectDetectorOptions.
static ObjectDetector
createFromFile(Context context, File modelFile)
Creates an ObjectDetector instance from a model file and the default ObjectDetector.ObjectDetectorOptions.
static ObjectDetector
ObjectDetectorResult
detect(MPImage image)
Performs object detection on the provided single image with default image processing options, i.e.
ObjectDetectorResult
detect(MPImage image, ImageProcessingOptions imageProcessingOptions)
Performs object detection on the provided single image.
void
detectAsync(MPImage image, ImageProcessingOptions imageProcessingOptions, long timestampMs)
Sends live image data to perform object detection, and the results will be available via the OutputHandler.ResultListener provided in the ObjectDetector.ObjectDetectorOptions.
void
detectAsync(MPImage image, long timestampMs)
Sends live image data to perform object detection with default image processing options, i.e.
ObjectDetectorResult
detectForVideo(MPImage image, ImageProcessingOptions imageProcessingOptions, long timestampMs)
Performs object detection on the provided video frame.
ObjectDetectorResult
detectForVideo(MPImage image, long timestampMs)
Performs object detection on the provided video frame with default image processing options, i.e.

Inherited Methods

Public Methods

public static ObjectDetector createFromBuffer (Context context, ByteBuffer modelBuffer)

Creates an ObjectDetector instance from a model buffer and the default ObjectDetector.ObjectDetectorOptions.

Parameters
context an Android ERROR(/Context).
modelBuffer a direct ByteBuffer or a ERROR(/MappedByteBuffer) of the detection model.
Throws
if there is an error during ObjectDetector creation.

public static ObjectDetector createFromFile (Context context, String modelPath)

Creates an ObjectDetector instance from a model file and the default ObjectDetector.ObjectDetectorOptions.

Parameters
context an Android ERROR(/Context).
modelPath path to the detection model with metadata in the assets.
Throws
if there is an error during ObjectDetector creation.

public static ObjectDetector createFromFile (Context context, File modelFile)

Creates an ObjectDetector instance from a model file and the default ObjectDetector.ObjectDetectorOptions.

Parameters
context an Android ERROR(/Context).
modelFile the detection model File instance.
Throws
IOException if an I/O error occurs when opening the tflite model file.
if there is an error during ObjectDetector creation.

public static ObjectDetector createFromOptions (Context context, ObjectDetector.ObjectDetectorOptions detectorOptions)

Parameters
context an Android ERROR(/Context).
detectorOptions an ObjectDetector.ObjectDetectorOptions instance.
Throws
if there is an error during ObjectDetector creation.

public ObjectDetectorResult detect (MPImage image)

Performs object detection on the provided single image with default image processing options, i.e. without any rotation applied. Only use this method when the ObjectDetect