InteractiveSegmenter

public final class InteractiveSegmenter

Performs interactive segmentation on images.

Note that, in addition to the standard segmentation API ERROR(/segment) that takes an input image and returns the outputs, but involves deep copy of the returns, InteractiveSegmenter also supports the callback API, ERROR(/segmentWithResultListener), which allows you to access the outputs through zero copy. Set OutputHandler.ResultListener in InteractiveSegmenter.InteractiveSegmenterOptions properly to use the callback API.

The API expects a TFLite model with,TFLite Model Metadata.. The model expects input with 4 channels, where the first 3 channels represent RGB image, and the last channel represents the user's region of interest.

Nested Classes

class InteractiveSegmenter.InteractiveSegmenterOptions Options for setting up an InteractiveSegmenter
class InteractiveSegmenter.RegionOfInterest The Region-Of-Interest (ROI) to interact with. 

Public Methods

static InteractiveSegmenter
ImageSegmenterResult
segment(MPImage image, InteractiveSegmenter.RegionOfInterest roi)
Performs segmentation on the provided single image with default image processing options, given user's region-of-interest, i.e.
ImageSegmenterResult
segment(MPImage image, InteractiveSegmenter.RegionOfInterest roi, ImageProcessingOptions imageProcessingOptions)
Performs segmentation on the provided single image, given user's region-of-interest.
void
segmentWithResultListener(MPImage image, InteractiveSegmenter.RegionOfInterest roi)
Performs segmentation on the provided single image with default image processing options, given user's region-of-interest, i.e.
void
segmentWithResultListener(MPImage image, InteractiveSegmenter.RegionOfInterest roi, ImageProcessingOptions imageProcessingOptions)
Performs segmentation on the provided single image given user's region-of-interest, and provides zero-copied results via OutputHandler.ResultListener in InteractiveSegmenter.InteractiveSegmenterOptions.

Inherited Methods

Public Methods

public static InteractiveSegmenter createFromOptions (Context context, InteractiveSegmenter.InteractiveSegmenterOptions segmenterOptions)

Parameters
context an Android ERROR(/Context).
segmenterOptions an InteractiveSegmenter.InteractiveSegmenterOptions instance.
Throws
if there is an error during InteractiveSegmenter creation.

public ImageSegmenterResult segment (MPImage image, InteractiveSegmenter.RegionOfInterest roi)

Performs segmentation on the provided single image with default image processing options, given user's region-of-interest, i.e. without any rotation applied. TODO update java doc for input image format.

Users can represent user interaction through InteractiveSegmenter.RegionOfInterest, which gives a hint to perform segmentation focusing on the given region of interest.

InteractiveSegmenter supports the following color space types:

Parameters
image a MediaPipe MPImage object for processing.
roi a InteractiveSegmenter.RegionOfInterest object to represent user interaction.
Throws
if there is an internal error. Or if InteractiveSegmenter is created with a OutputHandler.ResultListener.

public ImageSegmenterResult segment (MPImage image, InteractiveSegmenter.RegionOfInterest roi, ImageProcessingOptions imageProcessingOptions)

Performs segmentation on the provided single image, given user's region-of-interest. TODO update java doc for input image format.

Users can represent user interaction through InteractiveSegmenter.RegionOfInterest, which gives a hint to perform segmentation focusing on the given region of interest.

InteractiveSegmenter supports the following color space types:

Parameters
image a MediaPipe MPImage object for processing.
roi a InteractiveSegmenter.RegionOfInterest object to represent user interaction.
imageProcessingOptions the ImageProcessingOptions specifying how to process the input image before running inference. Note that region-of-interest is not supported by this task: specifying ImageProcessingOptions.regionOfInterest() will result in this method throwing an IllegalArgumentException.
Throws
IllegalArgumentException if the ImageProcessingOptions specify a region-of-interest.
if there is an internal error. Or if InteractiveSegmenter is created with a OutputHandler.ResultListener.

public void segmentWithResultListener (MPImage image, InteractiveSegmenter.RegionOfInterest roi)

Performs segmentation on the provided single image with default image processing options, given user's region-of-interest, i.e. without any rotation applied, and provides zero-copied results via OutputHandler.ResultListener in InteractiveSegmenter.InteractiveSegmenterOptions.

TODO update java doc for input image format.

Users can represent user interaction through InteractiveSegmenter.RegionOfInterest, which gives a hint to perform segmentation focusing on the given region of interest.

InteractiveSegmenter supports the following color space types:

Parameters
image a MediaPipe MPImage object for processing.
roi a InteractiveSegmenter.RegionOfInterest object to represent user interaction.
Throws
IllegalArgumentException if the ImageProcessingOptions specify a region-of-interest.
if there is an internal error. Or if InteractiveSegmenter is not created with OutputHandler.ResultListener set in InteractiveSegmenter.InteractiveSegmenterOptions.

public void segmentWithResultListener (MPImage image, InteractiveSegmenter.RegionOfInterest roi, ImageProcessingOptions imageProcessingOptions)

Performs segmentation on the provided single image given user's region-of-interest, and provides zero-copied results via OutputHandler.ResultListener in InteractiveSegmenter.InteractiveSegmenterOptions.

TODO update java doc for input image format.

Users can represent user interaction through InteractiveSegmenter.RegionOfInterest, which gives a hint to perform segmentation focusing on the given region of interest.

InteractiveSegmenter supports the following color space types:

Parameters
image a MediaPipe MPImage object for processing.
roi a InteractiveSegmenter.RegionOfInterest object to represent user interaction.
imageProcessingOptions the ImageProcessingOptions specifying how to process the input image before running inference. Note that region-of-interest is not supported by this task: specifying ImageProcessingOptions.regionOfInterest() will result in this method throwing an IllegalArgumentException.
Throws
IllegalArgumentException if the ImageProcessingOptions specify a region-of-interest.
if there is an internal error. Or if InteractiveSegmenter is not created with OutputHandler.ResultListener set in InteractiveSegmenter.InteractiveSegmenterOptions.