ImageSegmenter class

Performs image segmentation on images.

Signature:

export declare class ImageSegmenter extends VisionTaskRunner 

Extends: VisionTaskRunner

Methods

Method Modifiers Description
createFromModelBuffer(wasmFileset, modelAssetBuffer) static Initializes the Wasm runtime and creates a new image segmenter based on the provided model asset buffer.
createFromModelPath(wasmFileset, modelAssetPath) static Initializes the Wasm runtime and creates a new image segmenter based on the path to the model asset.
createFromOptions(wasmFileset, imageSegmenterOptions) static Initializes the Wasm runtime and creates a new image segmenter from the provided options.
getLabels() Get the category label list of the ImageSegmenter can recognize. For CATEGORY_MASK type, the index in the category mask corresponds to the category in the label list. For CONFIDENCE_MASK type, the output mask list at index corresponds to the category in the label list.If there is no labelmap provided in the model file, empty label array is returned. The labels used by the current model.
segment(image, callback) Performs image segmentation on the provided single image and invokes the callback with the response. The method returns synchronously once the callback returns. Only use this method when the ImageSegmenter is created with running mode image.
segment(image, imageProcessingOptions, callback) Performs image segmentation on the provided single image and invokes the callback with the response. The method returns synchronously once the callback returns. Only use this method when the ImageSegmenter is created with running mode image.
segment(image) Performs image segmentation on the provided single image and returns the segmentation result. This method creates a copy of the resulting masks and should not be used in high-throughput applications. Only use this method when the ImageSegmenter is created with running mode image.
segment(image, imageProcessingOptions) Performs image segmentation on the provided single image and returns the segmentation result. This method creates a copy of the resulting masks and should not be used in high-v applications. Only use this method when the ImageSegmenter is created with running mode image.
segmentForVideo(videoFrame, timestamp, callback) Performs image segmentation on the provided video frame and invokes the callback with the response. The method returns synchronously once the callback returns. Only use this method when the ImageSegmenter is created with running mode video.
segmentForVideo(videoFrame, timestamp, imageProcessingOptions, callback) Performs image segmentation on the provided video frame and invokes the callback with the response. The method returns synchronously once the callback returns. Only use this method when the ImageSegmenter is created with running mode video.
segmentForVideo(videoFrame, timestamp) Performs image segmentation on the provided video frame and returns the segmentation result. This method creates a copy of the resulting masks and should not be used in high-throughput applications. Only use this method when the ImageSegmenter is created with running mode video.
segmentForVideo(videoFrame, timestamp, imageProcessingOptions) Performs image segmentation on the provided video frame and returns the segmentation result. This method creates a copy of the resulting masks and should not be used in high-v applications. Only use this method when the ImageSegmenter is created with running mode video.
setOptions(options) Sets new options for the image segmenter.Calling setOptions() with a subset of options only affects those options. You can reset an option back to its default value by explicitly setting it to undefined.

ImageSegmenter.createFromModelBuffer()

Initializes the Wasm runtime and creates a new image segmenter based on the provided model asset buffer.

Signature:

static createFromModelBuffer(wasmFileset: WasmFileset, modelAssetBuffer: Uint8Array): Promise<ImageSegmenter>;

Parameters

Parameter Type Description
wasmFileset WasmFileset A configuration object that provides the location of the Wasm binary and its loader.
modelAssetBuffer Uint8Array A binary representation of the model.

Returns:

Promise<ImageSegmenter>

ImageSegmenter.createFromModelPath()

Initializes the Wasm runtime and creates a new image segmenter based on the path to the model asset.

Signature:

static createFromModelPath(wasmFileset: WasmFileset, modelAssetPath: string): Promise<ImageSegmenter>;

Parameters

Parameter Type Description
wasmFileset WasmFileset A configuration object that provides the location of the Wasm binary and its loader.
modelAssetPath string The path to the model asset.

Returns:

Promise<ImageSegmenter>

ImageSegmenter.createFromOptions()

Initializes the Wasm runtime and creates a new image segmenter from the provided options.

Signature:

static createFromOptions(wasmFileset: WasmFileset, imageSegmenterOptions: ImageSegmenterOptions): Promise<ImageSegmenter>;

Parameters

Parameter Type Description
wasmFileset WasmFileset A configuration object that provides the location of the Wasm binary and its loader.
imageSegmenterOptions ImageSegmenterOptions The options for the Image Segmenter. Note that either a path to the model asset or a model buffer needs to be provided (via baseOptions).

Returns:

Promise<ImageSegmenter>

ImageSegmenter.getLabels()

Get the category label list of the ImageSegmenter can recognize. For CATEGORY_MASK type, the index in the category mask corresponds to the category in the label list. For CONFIDENCE_MASK type, the output mask list at index corresponds to the category in the label list.

If there is no labelmap provided in the model file, empty label array is returned.

The labels used by the current model.

Signature:

getLabels(): string[];

Returns:

string[]

ImageSegmenter.segment()

Performs image segmentation on the provided single image and invokes the callback with the response. The method returns synchronously once the callback returns. Only use this method when the ImageSegmenter is created with running mode image.

Signature:

segment(image: ImageSource, callback: ImageSegmenterCallback): void;

Parameters

Parameter Type Description
image ImageSource An image to process.
callback ImageSegmenterCallback The callback that is invoked with the segmented masks. The lifetime of the returned data is only guaranteed for the duration of the callback.

Returns:

void

ImageSegmenter.segment()

Performs image segmentation on the provided single image and invokes the callback with the response. The method returns synchronously once the callback returns. Only use this method when the ImageSegmenter is created with running mode image.

Signature:

segment(image: ImageSource, imageProcessingOptions: ImageProcessingOptions, callback: ImageSegmenterCallback): void;

Parameters

Parameter Type Description
image ImageSource An image to process.
imageProcessingOptions ImageProcessingOptions the ImageProcessingOptions specifying how to process the input image before running inference.
callback ImageSegmenterCallback The callback that is invoked with the segmented masks. The lifetime of the returned data is only guaranteed for the duration of the callback.

Returns:

void

ImageSegmenter.segment()

Performs image segmentation on the provided single image and returns the segmentation result. This method creates a copy of the resulting masks and should not be used in high-throughput applications. Only use this method when the ImageSegmenter is created with running mode image.

Signature:

segment(image: ImageSource): ImageSegmenterResult;

Parameters

Parameter Type Description
image ImageSource An image to process. The segmentation result. The data is copied to avoid lifetime issues.

Returns:

ImageSegmenterResult

ImageSegmenter.segment()

Performs image segmentation on the provided single image and returns the segmentation result. This method creates a copy of the resulting masks and should not be used in high-v applications. Only use this method when the ImageSegmenter is created with running mode image.

Signature:

segment(image: ImageSource, imageProcessingOptions: ImageProcessingOptions): ImageSegmenterResult;

Parameters

Parameter Type Description
image ImageSource An image to process.
imageProcessingOptions ImageProcessingOptions the ImageProcessingOptions specifying how to process the input image before running inference. The segmentation result. The data is copied to avoid lifetime issues.

Returns:

ImageSegmenterResult

ImageSegmenter.segmentForVideo()

Performs image segmentation on the provided video frame and invokes the callback with the response. The method returns synchronously once the callback returns. Only use this method when the ImageSegmenter is created with running mode video.

Signature:

segmentForVideo(videoFrame: ImageSource, timestamp: number, callback: ImageSegmenterCallback): void;

Parameters

Parameter Type Description
videoFrame ImageSource A video frame to process.
timestamp number The timestamp of the current frame, in ms.
callback ImageSegmenterCallback The callback that is invoked with the segmented masks. The lifetime of the returned data is only guaranteed for the duration of the callback.

Returns:

void

ImageSegmenter.segmentForVideo()

Performs image segmentation on the provided video frame and invokes the callback with the response. The method returns synchronously once the callback returns. Only use this method when the ImageSegmenter is created with running mode video.

Signature:

segmentForVideo(videoFrame: ImageSource, timestamp: number, imageProcessingOptions: ImageProcessingOptions, callback: ImageSegmenterCallback): void;

Parameters

Parameter Type Description
videoFrame ImageSource A video frame to process.
timestamp number The timestamp of the current frame, in ms.
imageProcessingOptions ImageProcessingOptions the ImageProcessingOptions specifying how to process the input frame before running inference.
callback ImageSegmenterCallback The callback that is invoked with the segmented masks. The lifetime of the returned data is only guaranteed for the duration of the callback.

Returns:

void

ImageSegmenter.segmentForVideo()

Performs image segmentation on the provided video frame and returns the segmentation result. This method creates a copy of the resulting masks and should not be used in high-throughput applications. Only use this method when the ImageSegmenter is created with running mode video.

Signature: