Performs interactive segmentation on images.
Note that, in addition to the standard segmentation API ERROR(/segment) that takes an input
image and returns the outputs, but involves deep copy of the returns, InteractiveSegmenter also
supports the callback API, ERROR(/segmentWithResultListener), which allows you to access the
outputs through zero copy. Set OutputHandler.ResultListener in InteractiveSegmenter.InteractiveSegmenterOptions
properly to use the callback API.
The API expects a TFLite model with,TFLite Model Metadata.. The model expects input with 4 channels, where the first 3 channels represent RGB image, and the last channel represents the user's region of interest.
- Input image
MPImage- The image that image segmenter runs on.
- Input roi
InteractiveSegmenter.RegionOfInterest- Region of interest based on user interaction.
- Output ImageSegmenterResult
ImageSegmenterResult- An ImageSegmenterResult containing segmented masks.
Nested Classes
| class | InteractiveSegmenter.InteractiveSegmenterOptions | Options for setting up an InteractiveSegmenter. |
|
| class | InteractiveSegmenter.RegionOfInterest | The Region-Of-Interest (ROI) to interact with. | |
Public Methods
| static InteractiveSegmenter |
createFromOptions(Context context, InteractiveSegmenter.InteractiveSegmenterOptions segmenterOptions)
Creates an
InteractiveSegmenter instance from an InteractiveSegmenter.InteractiveSegmenterOptions. |
| ImageSegmenterResult |
segment(MPImage image, InteractiveSegmenter.RegionOfInterest roi)
Performs segmentation on the provided single image with default image processing options, given
user's region-of-interest, i.e.
|
| ImageSegmenterResult |
segment(MPImage image, InteractiveSegmenter.RegionOfInterest roi, ImageProcessingOptions imageProcessingOptions)
Performs segmentation on the provided single image, given user's region-of-interest.
|
| void |
segmentWithResultListener(MPImage image, InteractiveSegmenter.RegionOfInterest roi)
Performs segmentation on the provided single image with default image processing options, given
user's region-of-interest, i.e.
|
| void |
segmentWithResultListener(MPImage image, InteractiveSegmenter.RegionOfInterest roi, ImageProcessingOptions imageProcessingOptions)
Performs segmentation on the provided single image given user's region-of-interest, and
provides zero-copied results via
OutputHandler.ResultListener in InteractiveSegmenter.InteractiveSegmenterOptions. |
Inherited Methods
Public Methods
public static InteractiveSegmenter createFromOptions (Context context, InteractiveSegmenter.InteractiveSegmenterOptions segmenterOptions)
Creates an InteractiveSegmenter instance from an InteractiveSegmenter.InteractiveSegmenterOptions.
Parameters
| context | an Android ERROR(/Context). |
|---|---|
| segmenterOptions | an InteractiveSegmenter.InteractiveSegmenterOptions instance. |
Throws
if there is an error during InteractiveSegmenter creation.
|
public ImageSegmenterResult segment (MPImage image, InteractiveSegmenter.RegionOfInterest roi)
Performs segmentation on the provided single image with default image processing options, given user's region-of-interest, i.e. without any rotation applied. TODO update java doc for input image format.
Users can represent user interaction through InteractiveSegmenter.RegionOfInterest, which gives a hint to
perform segmentation focusing on the given region of interest.
InteractiveSegmenter supports the following color space types:
Parameters
| image | a MediaPipe MPImage object for processing. |
|---|---|
| roi | a InteractiveSegmenter.RegionOfInterest object to represent user interaction. |
Throws
if there is an internal error. Or if InteractiveSegmenter is
created with a OutputHandler.ResultListener.
|
public ImageSegmenterResult segment (MPImage image, InteractiveSegmenter.RegionOfInterest roi, ImageProcessingOptions imageProcessingOptions)
Performs segmentation on the provided single image, given user's region-of-interest. TODO update java doc for input image format.
Users can represent user interaction through InteractiveSegmenter.RegionOfInterest, which gives a hint to
perform segmentation focusing on the given region of interest.
InteractiveSegmenter supports the following color space types:
Parameters
| image | a MediaPipe MPImage object for processing. |
|---|---|
| roi | a InteractiveSegmenter.RegionOfInterest object to represent user interaction. |
| imageProcessingOptions | the ImageProcessingOptions specifying how to process the
input image before running inference. Note that region-of-interest is not supported
by this task: specifying ImageProcessingOptions.regionOfInterest() will result in
this method throwing an IllegalArgumentException. |
Throws
| IllegalArgumentException | if the ImageProcessingOptions specify a
region-of-interest. |
|---|---|
if there is an internal error. Or if InteractiveSegmenter is
created with a OutputHandler.ResultListener.
|
public void segmentWithResultListener (MPImage image, InteractiveSegmenter.RegionOfInterest roi)
Performs segmentation on the provided single image with default image processing options, given
user's region-of-interest, i.e. without any rotation applied, and provides zero-copied results
via OutputHandler.ResultListener in InteractiveSegmenter.InteractiveSegmenterOptions.
TODO update java doc for input image format.
Users can represent user interaction through InteractiveSegmenter.RegionOfInterest, which gives a hint to
perform segmentation focusing on the given region of interest.
InteractiveSegmenter supports the following color space types:
Parameters
| image | a MediaPipe MPImage object for processing. |
|---|---|
| roi | a InteractiveSegmenter.RegionOfInterest object to represent user interaction. |
Throws
| IllegalArgumentException | if the ImageProcessingOptions specify a
region-of-interest. |
|---|---|
if there is an internal error. Or if InteractiveSegmenter is
not created with OutputHandler.ResultListener set in InteractiveSegmenter.InteractiveSegmenterOptions.
|
public void segmentWithResultListener (MPImage image, InteractiveSegmenter.RegionOfInterest roi, ImageProcessingOptions imageProcessingOptions)
Performs segmentation on the provided single image given user's region-of-interest, and
provides zero-copied results via OutputHandler.ResultListener in InteractiveSegmenter.InteractiveSegmenterOptions.
TODO update java doc for input image format.
Users can represent user interaction through InteractiveSegmenter.RegionOfInterest, which gives a hint to
perform segmentation focusing on the given region of interest.
InteractiveSegmenter supports the following color space types:
Parameters
| image | a MediaPipe MPImage object for processing. |
|---|---|
| roi | a InteractiveSegmenter.RegionOfInterest object to represent user interaction. |
| imageProcessingOptions | the ImageProcessingOptions specifying how to process the
input image before running inference. Note that region-of-interest is not supported
by this task: specifying ImageProcessingOptions.regionOfInterest() will result in
this method throwing an IllegalArgumentException. |
Throws
| IllegalArgumentException | if the ImageProcessingOptions specify a
region-of-interest. |
|---|---|
if there is an internal error. Or if InteractiveSegmenter is
not created with OutputHandler.ResultListener set in InteractiveSegmenter.InteractiveSegmenterOptions.
|