Introducing LiteRT: Google's high-performance runtime for on-device AI, formerly known as TensorFlow Lite. Learn more

HolisticLandmarker class

Performs holistic landmarks detection on images.

Signature:

export declare class HolisticLandmarker extends VisionTaskRunner

Extends: VisionTaskRunner

Properties

Property	Modifiers	Type	Description
FACE_LANDMARKS_CONTOURS	`static`	Connection[]	Landmark connections to draw the face's contour.
FACE_LANDMARKS_FACE_OVAL	`static`	Connection[]	Landmark connections to draw the face's oval.
FACE_LANDMARKS_LEFT_EYE	`static`	Connection[]	Landmark connections to draw the connection between a face's left eye.
FACE_LANDMARKS_LEFT_EYEBROW	`static`	Connection[]	Landmark connections to draw the connection between a face's left eyebrow.
FACE_LANDMARKS_LEFT_IRIS	`static`	Connection[]	Landmark connections to draw the connection between a face's left iris.
FACE_LANDMARKS_LIPS	`static`	Connection[]	Landmark connections to draw the connection between a face's lips.
FACE_LANDMARKS_RIGHT_EYE	`static`	Connection[]	Landmark connections to draw the connection between a face's right eye.
FACE_LANDMARKS_RIGHT_EYEBROW	`static`	Connection[]	Landmark connections to draw the connection between a face's right eyebrow.
FACE_LANDMARKS_RIGHT_IRIS	`static`	Connection[]	Landmark connections to draw the connection between a face's right iris.
FACE_LANDMARKS_TESSELATION	`static`	Connection[]	Landmark connections to draw the face's tesselation.
HAND_CONNECTIONS	`static`	Connection[]	An array containing the pairs of hand landmark indices to be rendered with connections.
POSE_CONNECTIONS	`static`	Connection[]	An array containing the pairs of pose landmark indices to be rendered with connections.

Methods

Method	Modifiers	Description
createFromModelBuffer(wasmFileset, modelAssetBuffer)	`static`	Initializes the Wasm runtime and creates a new `HolisticLandmarker` based on the provided model asset buffer.
createFromModelPath(wasmFileset, modelAssetPath)	`static`	Initializes the Wasm runtime and creates a new `HolisticLandmarker` based on the path to the model asset.
createFromOptions(wasmFileset, holisticLandmarkerOptions)	`static`	Initializes the Wasm runtime and creates a new `HolisticLandmarker` from the provided options.
detect(image, callback)		Performs holistic landmarks detection on the provided single image and invokes the callback with the response. The method returns synchronously once the callback returns. Only use this method when the HolisticLandmarker is created with running mode `image`.
detect(image, imageProcessingOptions, callback)		Performs holistic landmarks detection on the provided single image and invokes the callback with the response. The method returns synchronously once the callback returns. Only use this method when the HolisticLandmarker is created with running mode `image`.
detect(image)		Performs holistic landmarks detection on the provided single image and waits synchronously for the response. This method creates a copy of the resulting masks and should not be used in high-throughput applications. Only use this method when the HolisticLandmarker is created with running mode `image`.
detect(image, imageProcessingOptions)		Performs holistic landmarks detection on the provided single image and waits synchronously for the response. This method creates a copy of the resulting masks and should not be used in high-throughput applications. Only use this method when the HolisticLandmarker is created with running mode `image`.
detectForVideo(videoFrame, timestamp, callback)		Performs holistic landmarks detection on the provided video frame and invokes the callback with the response. The method returns synchronously once the callback returns. Only use this method when the HolisticLandmarker is created with running mode `video`.
detectForVideo(videoFrame, timestamp, imageProcessingOptions, callback)		Performs holistic landmarks detection on the provided video frame and invokes the callback with the response. The method returns synchronously once the callback returns. Only use this method when the holisticLandmarker is created with running mode `video`.
detectForVideo(videoFrame, timestamp)		Performs holistic landmarks detection on the provided video frame and returns the result. This method creates a copy of the resulting masks and should not be used in high-throughput applications. Only use this method when the HolisticLandmarker is created with running mode `video`.
detectForVideo(videoFrame, timestamp, imageProcessingOptions)		Performs holistic landmarks detection on the provided video frame and waits synchronously for the response. Only use this method when the HolisticLandmarker is created with running mode `video`.
setOptions(options)		Sets new options for this `HolisticLandmarker`.Calling `setOptions()` with a subset of options only affects those options. You can reset an option back to its default value by explicitly setting it to `undefined`.

HolisticLandmarker.FACE_LANDMARKS_CONTOURS

Landmark connections to draw the face's contour.

Signature:

static FACE_LANDMARKS_CONTOURS: Connection[];

HolisticLandmarker.FACE_LANDMARKS_FACE_OVAL

Landmark connections to draw the face's oval.

Signature:

static FACE_LANDMARKS_FACE_OVAL: Connection[];

HolisticLandmarker.FACE_LANDMARKS_LEFT_EYE

Landmark connections to draw the connection between a face's left eye.

Signature:

static FACE_LANDMARKS_LEFT_EYE: Connection[];

HolisticLandmarker.FACE_LANDMARKS_LEFT_EYEBROW

Landmark connections to draw the connection between a face's left eyebrow.

Signature:

static FACE_LANDMARKS_LEFT_EYEBROW: Connection[];

HolisticLandmarker.FACE_LANDMARKS_LEFT_IRIS

Landmark connections to draw the connection between a face's left iris.

Signature:

static FACE_LANDMARKS_LEFT_IRIS: Connection[];

HolisticLandmarker.FACE_LANDMARKS_LIPS

Landmark connections to draw the connection between a face's lips.

Signature:

static FACE_LANDMARKS_LIPS: Connection[];

HolisticLandmarker.FACE_LANDMARKS_RIGHT_EYE

Landmark connections to draw the connection between a face's right eye.

Signature:

static FACE_LANDMARKS_RIGHT_EYE: Connection[];

HolisticLandmarker.FACE_LANDMARKS_RIGHT_EYEBROW

Landmark connections to draw the connection between a face's right eyebrow.

Signature:

static FACE_LANDMARKS_RIGHT_EYEBROW: Connection[];

HolisticLandmarker.FACE_LANDMARKS_RIGHT_IRIS

Landmark connections to draw the connection between a face's right iris.

Signature:

static FACE_LANDMARKS_RIGHT_IRIS: Connection[];

HolisticLandmarker.FACE_LANDMARKS_TESSELATION

Landmark connections to draw the face's tesselation.

Signature:

static FACE_LANDMARKS_TESSELATION: Connection[];

HolisticLandmarker.HAND_CONNECTIONS

An array containing the pairs of hand landmark indices to be rendered with connections.

Signature:

static HAND_CONNECTIONS: Connection[];

HolisticLandmarker.POSE_CONNECTIONS

An array containing the pairs of pose landmark indices to be rendered with connections.

Signature:

static POSE_CONNECTIONS: Connection[];

HolisticLandmarker.createFromModelBuffer()

Initializes the Wasm runtime and creates a new HolisticLandmarker based on the provided model asset buffer.

Signature:

static createFromModelBuffer(wasmFileset: WasmFileset, modelAssetBuffer: Uint8Array): Promise<HolisticLandmarker>;

Parameters

Parameter	Type	Description
wasmFileset	WasmFileset	A configuration object that provides the location of the Wasm binary and its loader.
modelAssetBuffer	Uint8Array	A binary representation of the model.

Returns:

Promise<HolisticLandmarker>

HolisticLandmarker.createFromModelPath()

Initializes the Wasm runtime and creates a new HolisticLandmarker based on the path to the model asset.

Signature:

static createFromModelPath(wasmFileset: WasmFileset, modelAssetPath: string): Promise<HolisticLandmarker>;

Parameters

Parameter	Type	Description
wasmFileset	WasmFileset	A configuration object that provides the location of the Wasm binary and its loader.
modelAssetPath	string	The path to the model asset.

Returns:

Promise<HolisticLandmarker>

HolisticLandmarker.createFromOptions()

Initializes the Wasm runtime and creates a new HolisticLandmarker from the provided options.

Signature:

static createFromOptions(wasmFileset: WasmFileset, holisticLandmarkerOptions: HolisticLandmarkerOptions): Promise<HolisticLandmarker>;

Parameters

Parameter	Type	Description
wasmFileset	WasmFileset	A configuration object that provides the location of the Wasm binary and its loader.
holisticLandmarkerOptions	HolisticLandmarkerOptions	The options for the HolisticLandmarker. Note that either a path to the model asset or a model buffer needs to be provided (via `baseOptions`).

Returns:

Promise<HolisticLandmarker>

HolisticLandmarker.detect()

Performs holistic landmarks detection on the provided single image and invokes the callback with the response. The method returns synchronously once the callback returns. Only use this method when the HolisticLandmarker is created with running mode image.

Signature:

detect(image: ImageSource, callback: HolisticLandmarkerCallback): void;

Parameters

Parameter	Type	Description
image	ImageSource	An image to process.
callback	HolisticLandmarkerCallback	The callback that is invoked with the result. The lifetime of the returned masks is only guaranteed for the duration of the callback.

Returns:

void

HolisticLandmarker.detect()

Signature:

detect(image: ImageSource, imageProcessingOptions: ImageProcessingOptions, callback: HolisticLandmarkerCallback): void;

Parameters

Parameter	Type	Description
image	ImageSource	An image to process.
imageProcessingOptions	ImageProcessingOptions	the `ImageProcessingOptions` specifying how to process the input image before running inference.
callback	HolisticLandmarkerCallback	The callback that is invoked with the result. The lifetime of the returned masks is only guaranteed for the duration of the callback.

Returns:

void

HolisticLandmarker.detect()

Performs holistic landmarks detection on the provided single image and waits synchronously for the response. This method creates a copy of the resulting masks and should not be used in high-throughput applications. Only use this method when the HolisticLandmarker is created with running mode image.

Signature:

detect(image: ImageSource): HolisticLandmarkerResult;

Parameters

Parameter	Type	Description
image	ImageSource	An image to process. The landmarker result. Any masks are copied to avoid lifetime limits. The detected pose landmarks.

Returns:

HolisticLandmarkerResult

HolisticLandmarker.detect()

Signature:

detect(image: ImageSource, imageProcessingOptions: ImageProcessingOptions): HolisticLandmarkerResult;

Parameters

Parameter	Type	Description
image	ImageSource	An image to process. The landmarker result. Any masks are copied to avoid lifetime limits. The detected pose landmarks.
imageProcessingOptions	ImageProcessingOptions

Returns:

HolisticLandmarkerResult

HolisticLandmarker.detectForVideo()

Performs holistic landmarks detection on the provided video frame and invokes the callback with the response. The method returns synchronously once the callback returns. Only use this method when the HolisticLandmarker is created with running mode video.

Signature:

detectForVideo(videoFrame: ImageSource, timestamp: number, callback: HolisticLandmarkerCallback): void;

Parameters

Parameter	Type	Description
videoFrame	ImageSource	A video frame to process.
timestamp	number	The timestamp of the current frame, in ms.
callback	HolisticLandmarkerCallback	The callback that is invoked with the result. The lifetime of the returned masks is only guaranteed for the duration of the callback.

Returns:

void

HolisticLandmarker.detectForVideo()

Performs holistic landmarks detection on the provided video frame and invokes the callback with the response. The method returns synchronously once the callback returns. Only use this method when the holisticLandmarker is created with running mode video.

Signature:

detectForVideo(videoFrame: ImageSource, timestamp: number, imageProcessingOptions: ImageProcessingOptions, callback: HolisticLandmarkerCallback): void;

Parameters

Parameter	Type	Description
videoFrame	ImageSource	A video frame to process.
timestamp	number	The timestamp of the current frame, in ms.
imageProcessingOptions	ImageProcessingOptions	the `ImageProcessingOptions` specifying how to process the input image before running inference.
callback	HolisticLandmarkerCallback	The callback that is invoked with the result. The lifetime of the returned masks is only guaranteed for the duration of the callback.

Returns:

void

HolisticLandmarker.detectForVideo()

Performs holistic landmarks detection on the provided video frame and returns the result. This method creates a copy of the resulting masks and should not be used in high-throughput applications. Only use this method when the HolisticLandmarker is created with running mode video.

Signature:

detectForVideo(videoFrame: ImageSource, timestamp: number): HolisticLandmarkerResult;

Parameters

Parameter	Type	Description
videoFrame	ImageSource	A video frame to process.
timestamp	number	The timestamp of the current frame, in ms. The landmarker result. Any masks are copied to extend the lifetime of the returned data.

Returns:

HolisticLandmarkerResult

HolisticLandmarker.detectForVideo()

Performs holistic landmarks detection on the provided video frame and waits synchronously for the response. Only use this method when the HolisticLandmarker is created with running mode video.

Signature:

detectForVideo(videoFrame: ImageSource, timestamp: number, imageProcessingOptions: ImageProcessingOptions): HolisticLandmarkerResult;

Parameters

Parameter	Type	Description
videoFrame	ImageSource	A video frame to process.
timestamp	number	The timestamp of the current frame, in ms.
imageProcessingOptions	ImageProcessingOptions	the `ImageProcessingOptions` specifying how to process the input image before running inference. The detected holistic landmarks.

Returns:

HolisticLandmarkerResult

HolisticLandmarker.setOptions()

Sets new options for this HolisticLandmarker.

Calling setOptions() with a subset of options only affects those options. You can reset an option back to its default value by explicitly setting it to undefined.

Signature:

setOptions(options: HolisticLandmarkerOptions): Promise<void>;

Parameters

Parameter	Type	Description
options	HolisticLandmarkerOptions	The options for the holistic landmarker.

Returns:

Promise<void>