Performs interactive segmentation on images.
Note that, in addition to the standard segmentation API ERROR(/segment)
that takes an input
image and returns the outputs, but involves deep copy of the returns, InteractiveSegmenter also
supports the callback API, ERROR(/segmentWithResultListener)
, which allows you to access the
outputs through zero copy. Set OutputHandler.ResultListener
in InteractiveSegmenter.InteractiveSegmenterOptions
properly to use the callback API.
The API expects a TFLite model with,TFLite Model Metadata.. The model expects input with 4 channels, where the first 3 channels represent RGB image, and the last channel represents the user's region of interest.
- Input image
MPImage
- The image that image segmenter runs on.
- Input roi
InteractiveSegmenter.RegionOfInterest
- Region of interest based on user interaction.
- Output ImageSegmenterResult
ImageSegmenterResult
- An ImageSegmenterResult containing segmented masks.
Nested Classes
class | InteractiveSegmenter.InteractiveSegmenterOptions | Options for setting up an InteractiveSegmenter . |
|
class | InteractiveSegmenter.RegionOfInterest | The Region-Of-Interest (ROI) to interact with. |
Public Methods
static InteractiveSegmenter |
createFromOptions(Context context, InteractiveSegmenter.InteractiveSegmenterOptions segmenterOptions)
Creates an
InteractiveSegmenter instance from an InteractiveSegmenter.InteractiveSegmenterOptions . |
ImageSegmenterResult |
segment(MPImage image, InteractiveSegmenter.RegionOfInterest roi)
Performs segmentation on the provided single image with default image processing options, given
user's region-of-interest, i.e.
|
ImageSegmenterResult |
segment(MPImage image, InteractiveSegmenter.RegionOfInterest roi, ImageProcessingOptions imageProcessingOptions)
Performs segmentation on the provided single image, given user's region-of-interest.
|
void |
segmentWithResultListener(MPImage image, InteractiveSegmenter.RegionOfInterest roi)
Performs segmentation on the provided single image with default image processing options, given
user's region-of-interest, i.e.
|
void |
segmentWithResultListener(MPImage image, InteractiveSegmenter.RegionOfInterest roi, ImageProcessingOptions imageProcessingOptions)
Performs segmentation on the provided single image given user's region-of-interest, and
provides zero-copied results via
OutputHandler.ResultListener in InteractiveSegmenter.InteractiveSegmenterOptions . |
Inherited Methods
Public Methods
public static InteractiveSegmenter createFromOptions (Context context, InteractiveSegmenter.InteractiveSegmenterOptions segmenterOptions)
Creates an InteractiveSegmenter
instance from an InteractiveSegmenter.InteractiveSegmenterOptions
.
Parameters
context | an Android ERROR(/Context) . |
---|---|
segmenterOptions | an InteractiveSegmenter.InteractiveSegmenterOptions instance. |
Throws
if there is an error during InteractiveSegmenter creation.
|
public ImageSegmenterResult segment (MPImage image, InteractiveSegmenter.RegionOfInterest roi)
Performs segmentation on the provided single image with default image processing options, given user's region-of-interest, i.e. without any rotation applied. TODO update java doc for input image format.
Users can represent user interaction through InteractiveSegmenter.RegionOfInterest
, which gives a hint to
perform segmentation focusing on the given region of interest.
InteractiveSegmenter
supports the following color space types:
Parameters
image | a MediaPipe MPImage object for processing. |
---|---|
roi | a InteractiveSegmenter.RegionOfInterest object to represent user interaction. |
Throws
if there is an internal error. Or if InteractiveSegmenter is
created with a OutputHandler.ResultListener .
|
public ImageSegmenterResult segment (MPImage image, InteractiveSegmenter.RegionOfInterest roi, ImageProcessingOptions imageProcessingOptions)
Performs segmentation on the provided single image, given user's region-of-interest. TODO update java doc for input image format.
Users can represent user interaction through InteractiveSegmenter.RegionOfInterest
, which gives a hint to
perform segmentation focusing on the given region of interest.
InteractiveSegmenter
supports the following color space types:
Parameters
image | a MediaPipe MPImage object for processing. |
---|---|
roi | a InteractiveSegmenter.RegionOfInterest object to represent user interaction. |
imageProcessingOptions | the ImageProcessingOptions specifying how to process the
input image before running inference. Note that region-of-interest is not supported
by this task: specifying ImageProcessingOptions.regionOfInterest() will result in
this method throwing an IllegalArgumentException. |
Throws
IllegalArgumentException | if the ImageProcessingOptions specify a
region-of-interest. |
---|---|
if there is an internal error. Or if InteractiveSegmenter is
created with a OutputHandler.ResultListener .
|
public void segmentWithResultListener (MPImage image, InteractiveSegmenter.RegionOfInterest roi)
Performs segmentation on the provided single image with default image processing options, given
user's region-of-interest, i.e. without any rotation applied, and provides zero-copied results
via OutputHandler.ResultListener
in InteractiveSegmenter.InteractiveSegmenterOptions
.
TODO update java doc for input image format.
Users can represent user interaction through InteractiveSegmenter.RegionOfInterest
, which gives a hint to
perform segmentation focusing on the given region of interest.
InteractiveSegmenter
supports the following color space types:
Parameters
image | a MediaPipe MPImage object for processing. |
---|---|
roi | a InteractiveSegmenter.RegionOfInterest object to represent user interaction. |
Throws
IllegalArgumentException | if the ImageProcessingOptions specify a
region-of-interest. |
---|---|
if there is an internal error. Or if InteractiveSegmenter is
not created with OutputHandler.ResultListener set in InteractiveSegmenter.InteractiveSegmenterOptions .
|
public void segmentWithResultListener (MPImage image, InteractiveSegmenter.RegionOfInterest roi, ImageProcessingOptions imageProcessingOptions)
Performs segmentation on the provided single image given user's region-of-interest, and
provides zero-copied results via OutputHandler.ResultListener
in InteractiveSegmenter.InteractiveSegmenterOptions
.
TODO update java doc for input image format.
Users can represent user interaction through InteractiveSegmenter.RegionOfInterest
, which gives a hint to
perform segmentation focusing on the given region of interest.
InteractiveSegmenter
supports the following color space types:
Parameters
image | a MediaPipe MPImage object for processing. |
---|---|
roi | a InteractiveSegmenter.RegionOfInterest object to represent user interaction. |
imageProcessingOptions | the ImageProcessingOptions specifying how to process the
input image before running inference. Note that region-of-interest is not supported
by this task: specifying ImageProcessingOptions.regionOfInterest() will result in
this method throwing an IllegalArgumentException. |
Throws
IllegalArgumentException | if the ImageProcessingOptions specify a
region-of-interest. |
---|---|
if there is an internal error. Or if InteractiveSegmenter is
not created with OutputHandler.ResultListener set in InteractiveSegmenter.InteractiveSegmenterOptions .
|