Introducing LiteRT: Google's high-performance runtime for on-device AI, formerly known as TensorFlow Lite. Learn more

InteractiveSegmenter

public final class InteractiveSegmenter

Performs interactive segmentation on images.

Note that, in addition to the standard segmentation API ERROR(/segment) that takes an input image and returns the outputs, but involves deep copy of the returns, InteractiveSegmenter also supports the callback API, ERROR(/segmentWithResultListener), which allows you to access the outputs through zero copy. Set OutputHandler.ResultListener in InteractiveSegmenter.InteractiveSegmenterOptions properly to use the callback API.

The API expects a TFLite model with,TFLite Model Metadata.. The model expects input with 4 channels, where the first 3 channels represent RGB image, and the last channel represents the user's region of interest.

Input image MPImage
- The image that image segmenter runs on.
Input roi InteractiveSegmenter.RegionOfInterest
- Region of interest based on user interaction.
Output ImageSegmenterResult ImageSegmenterResult
- An ImageSegmenterResult containing segmented masks.

Nested Classes

class	InteractiveSegmenter.InteractiveSegmenterOptions		Options for setting up an `InteractiveSegmenter`.
class	InteractiveSegmenter.RegionOfInterest		The Region-Of-Interest (ROI) to interact with.

Public Methods

static InteractiveSegmenter	createFromOptions(Context context, InteractiveSegmenter.InteractiveSegmenterOptions segmenterOptions) Creates an `InteractiveSegmenter` instance from an `InteractiveSegmenter.InteractiveSegmenterOptions`.
ImageSegmenterResult	segment(MPImage image, InteractiveSegmenter.RegionOfInterest roi) Performs segmentation on the provided single image with default image processing options, given user's region-of-interest, i.e.
ImageSegmenterResult	segment(MPImage image, InteractiveSegmenter.RegionOfInterest roi, ImageProcessingOptions imageProcessingOptions) Performs segmentation on the provided single image, given user's region-of-interest.
void	segmentWithResultListener(MPImage image, InteractiveSegmenter.RegionOfInterest roi) Performs segmentation on the provided single image with default image processing options, given user's region-of-interest, i.e.
void	segmentWithResultListener(MPImage image, InteractiveSegmenter.RegionOfInterest roi, ImageProcessingOptions imageProcessingOptions) Performs segmentation on the provided single image given user's region-of-interest, and provides zero-copied results via `OutputHandler.ResultListener` in `InteractiveSegmenter.InteractiveSegmenterOptions`.

Inherited Methods

From class com.google.mediapipe.tasks.vision.core.BaseVisionTaskApi

void	close() Closes and cleans up the MediaPipe vision task.
static long	generateResultTimestampMs(RunningMode runningMode, Packet packet) Generates the timestamp of a vision task result object from the vision task running mode and the output packet.

From class java.lang.Object

boolean	equals(Object arg0)
final Class<?>	getClass()
int	hashCode()
final void	notify()
final void	notifyAll()
String	toString()
final void	wait(long arg0, int arg1)
final void	wait(long arg0)
final void	wait()

From interface java.lang.AutoCloseable

abstract void

close()

Public Methods

public static InteractiveSegmenter createFromOptions (Context context, InteractiveSegmenter.InteractiveSegmenterOptions segmenterOptions)

Creates an InteractiveSegmenter instance from an InteractiveSegmenter.InteractiveSegmenterOptions.

Parameters

context	an Android `ERROR(/Context)`.
segmenterOptions	an `InteractiveSegmenter.InteractiveSegmenterOptions` instance.

Throws

	if there is an error during `InteractiveSegmenter` creation.

public ImageSegmenterResult segment (MPImage image, InteractiveSegmenter.RegionOfInterest roi)

Performs segmentation on the provided single image with default image processing options, given user's region-of-interest, i.e. without any rotation applied. TODO update java doc for input image format.

Users can represent user interaction through InteractiveSegmenter.RegionOfInterest, which gives a hint to perform segmentation focusing on the given region of interest.

InteractiveSegmenter supports the following color space types:

ERROR(/Bitmap.Config.ARGB_8888)

Parameters

image	a MediaPipe `MPImage` object for processing.
roi	a `InteractiveSegmenter.RegionOfInterest` object to represent user interaction.

Throws

	if there is an internal error. Or if `InteractiveSegmenter` is created with a `OutputHandler.ResultListener`.

public ImageSegmenterResult segment (MPImage image, InteractiveSegmenter.RegionOfInterest roi, ImageProcessingOptions imageProcessingOptions)

Performs segmentation on the provided single image, given user's region-of-interest. TODO update java doc for input image format.

Users can represent user interaction through InteractiveSegmenter.RegionOfInterest, which gives a hint to perform segmentation focusing on the given region of interest.

InteractiveSegmenter supports the following color space types:

ERROR(/Bitmap.Config.ARGB_8888)

Parameters

image	a MediaPipe `MPImage` object for processing.
roi	a `InteractiveSegmenter.RegionOfInterest` object to represent user interaction.
imageProcessingOptions	the `ImageProcessingOptions` specifying how to process the input image before running inference. Note that region-of-interest is not supported by this task: specifying `ImageProcessingOptions.regionOfInterest()` will result in this method throwing an IllegalArgumentException.

Throws

IllegalArgumentException	if the