MediaPipeTasksVision Framework Reference

HandLandmarker

class HandLandmarker : NSObject

@brief Performs hand landmarks detection on images.

This API expects a pre-trained hand landmarks model asset bundle.

  • The array of connections between the landmarks in the palm.

    Declaration

    Swift

    class var handPalmConnections: [Connection] { get }

  • The array of connections between the landmarks in the thumb.

    Declaration

    Swift

    class var handThumbConnections: [Connection] { get }

  • The array of connections between the landmarks in the index finger.

    Declaration

    Swift

    class var handIndexFingerConnections: [Connection] { get }

  • The array of connections between the landmarks in the middle finger.

    Declaration

    Swift

    class var handMiddleFingerConnections: [Connection] { get }

  • The array of connections between the landmarks in the ring finger.

    Declaration

    Swift

    class var handRingFingerConnections: [Connection] { get }

  • The array of connections between the landmarks in the pinky.

    Declaration

    Swift

    class var handPinkyConnections: [Connection] { get }

  • The array of connections between all the landmarks in the hand.

    Declaration

    Swift

    class var handConnections: [Connection] { get }

  • Creates a new instance of HandLandmarker from an absolute path to a model asset bundle stored locally on the device and the default HandLandmarkerOptions.

    Declaration

    Swift

    convenience init(modelPath: String) throws

    Parameters

    modelPath

    An absolute path to a model asset bundle stored locally on the device.
    error

    An optional error parameter populated when there is an error in initializing the hand landmarker.

    Return Value

    A new instance of HandLandmarker with the given model path. nil if there is an error in initializing the hand landmarker.

  • Creates a new instance of HandLandmarker from the given HandLandmarkerOptions.

    Declaration

    Swift

    init(options: HandLandmarkerOptions) throws

    Parameters

    options

    The options of type HandLandmarkerOptions to use for configuring the HandLandmarker.
    error

    An optional error parameter populated when there is an error in initializing the hand landmarker.

    Return Value

    A new instance of HandLandmarker with the given options. nil if there is an error in initializing the hand landmarker.

  • Performs hand landmarks detection on the provided MPImage using the whole image as region of interest. Rotation will be applied according to the orientation property of the provided MPImage. Only use this method when the HandLandmarker is created with running mode, .image.

    This method supports performing hand landmarks detection on RGBA images. If your MPImage has a source type of .pixelBuffer or .sampleBuffer, the underlying pixel buffer must use kCVPixelFormatType_32BGRA as its pixel format.

    If your MPImage has a source type of .image ensure that the color space is RGB with an Alpha channel.

    Declaration

    Swift

    func detect(image: MPImage) throws -> HandLandmarkerResult

    Parameters

    image

    The MPImage on which hand landmarks detection is to be performed.
    error

    An optional error parameter populated when there is an error in performing hand landmarks detection on the input image.

    Return Value

    An HandLandmarkerResult object that contains the hand hand landmarks detection results.

  • Performs hand landmarks detection on the provided video frame of type MPImage using the whole image as region of interest. Rotation will be applied according to the orientation property of the provided MPImage. Only use this method when the HandLandmarker is created with running mode, .video.

    It’s required to provide the video frame’s timestamp (in milliseconds). The input timestamps must be monotonically increasing.

    This method supports performing hand landmarks detection on RGBA images. If your MPImage has a source type of .pixelBuffer or .sampleBuffer, the underlying pixel buffer must use kCVPixelFormatType_32BGRA as its pixel format.

    If your MPImage has a source type of .image ensure that the color space is RGB with an Alpha channel.

    Declaration

    Swift

    func detect(videoFrame image: MPImage, timestampInMilliseconds: Int) throws -> HandLandmarkerResult

    Parameters

    image

    The MPImage on which hand landmarks detection is to be performed.
    timestampInMilliseconds

    The video frame’s timestamp (in milliseconds). The input timestamps must be monotonically increasing.
    error

    An optional error parameter populated when there is an error in performing hand landmarks detection on the input video frame.

    Return Value

    An