ImageClassifier

public final class ImageClassifier

Performs classification on images.

The API expects a TFLite model with optional, but strongly recommended, TFLite Model Metadata..

The API supports models with one image input tensor and one or more output tensors. To be more specific, here are the requirements.

  • Input image tensor (kTfLiteUInt8/kTfLiteFloat32)
    • image input of size [batch x height x width x channels].
    • batch inference is not supported (batch is required to be 1).
    • only RGB inputs are supported (channels is required to be 3).
    • if type is kTfLiteFloat32, NormalizationOptions are required to be attached to the metadata for input normalization.
  • At least one output tensor (kTfLiteUInt8/kTfLiteFloat32) with:
    • N classes and either 2 or 4 dimensions, i.e. [1 x N] or [1 x 1 x 1 x N]
    • optional (but recommended) label map(s) as AssociatedFile-s with type TENSOR_AXIS_LABELS, containing one label per line. The first such AssociatedFile (if any) is used to fill the class_name field of the results. The display_name field is filled from the AssociatedFile (if any) whose locale matches the display_names_locale field of the ImageClassifierOptions used at creation time ("en" by default, i.e. English). If none of these are available, only the index field of the results will be filled.
    • optional score calibration can be attached using ScoreCalibrationOptions and an AssociatedFile with type TENSOR_AXIS_SCORE_CALIBRATION. See metadata_schema.fbs for more details.

An example of such model can be found TensorFlow Hub.

Nested Classes

class ImageClassifier.ImageClassifierOptions Options for setting up and ImageClassifier. 

Public Methods

ImageClassifierResult
classify(MPImage image)
Performs classification on the provided single image with default image processing options, i.e.
ImageClassifierResult
classify(MPImage image, ImageProcessingOptions imageProcessingOptions)
Performs classification on the provided single image.
void
classifyAsync(MPImage image, ImageProcessingOptions imageProcessingOptions, long timestampMs)
Sends live image data to perform classification, and the results will be available via the OutputHandler.ResultListener provided in the ImageClassifier.ImageClassifierOptions.
void
classifyAsync(MPImage image, long timestampMs)
Sends live image data to perform classification with default image processing options, i.e.
ImageClassifierResult
classifyForVideo(MPImage image, long timestampMs)
Performs classification on the provided video frame with default image processing options, i.e.
ImageClassifierResult
classifyForVideo(MPImage image, ImageProcessingOptions imageProcessingOptions, long timestampMs)
Performs classification on the provided video frame.
static ImageClassifier
createFromBuffer(Context context, ByteBuffer modelBuffer)
Creates an ImageClassifier instance from a model buffer and default ImageClassifier.ImageClassifierOptions.
static ImageClassifier
createFromFile(Context context, String modelPath)
Creates an ImageClassifier instance from a model file and default ImageClassifier.ImageClassifierOptions.
static ImageClassifier
createFromFile(Context context, File modelFile)
Creates an ImageClassifier instance from a model file and default ImageClassifier.ImageClassifierOptions.
static ImageClassifier
createFromOptions(Context context, ImageClassifier.ImageClassifierOptions options)
Creates an ImageClassifier instance from an ImageClassifier.ImageClassifierOptions instance.

Inherited Methods

Public Methods

public ImageClassifierResult classify (MPImage image)

Performs classification on the provided single image with default image processing options, i.e. using the whole image as region-of-interest and without any rotation applied. Only use this method when the ImageClassifier is created with ERROR(/RunningMode.IMAGE).

ImageClassifier supports the following color space types:

Parameters
image a MediaPipe MPImage object for processing.
Throws
if there is an internal error.

public ImageClassifierResult classify (MPImage image, ImageProcessingOptions imageProcessingOptions)

Performs classification on the provided single image. Only use this method when the ImageClassifier is created with ERROR(/RunningMode.IMAGE).

ImageClassifier supports the following color space types:

Parameters
image a MediaPipe MPImage object for processing.
imageProcessingOptions the ImageProcessingOptions specifying how to process the input image before running inference.
Throws
if there is an internal error.

public void classifyAsync (MPImage image, ImageProcessingOptions imageProcessingOptions, long timestampMs)

Sends live image data to perform classification, and the results will be available via the OutputHandler.ResultListener provided in the