MediaPipeTasksVision Framework Reference

ImageEmbedder

class ImageEmbedder : NSObject

@brief Performs embedding extraction on images.

The API expects a TFLite model with optional, but strongly recommended, TFLite Model Metadata..

The API supports models with one image input tensor and one or more output tensors. To be more specific, here are the requirements.

Input image tensor (kTfLiteUInt8/kTfLiteFloat32)

  • image input of size [batch x height x width x channels].
  • batch inference is not supported (batch is required to be 1).
  • only RGB inputs are supported (channels is required to be 3).
  • if type is kTfLiteFloat32, NormalizationOptions are required to be attached to the metadata for input normalization.

At least one output tensor (kTfLiteUInt8/kTfLiteFloat32) with shape [1 x N] where N is the number of dimensions in the produced embeddings.

  • Creates a new instance of ImageEmbedder from an absolute path to a TensorFlow Lite model file stored locally on the device and the default ImageEmbedderOptions.

    Declaration

    Swift

    convenience init(modelPath: String) throws

    Parameters

    modelPath

    An absolute path to a TensorFlow Lite model file stored locally on the device.

    Return Value

    A new instance of ImageEmbedder with the given model path. nil if there is an error in initializing the image embedder.

  • Creates a new instance of ImageEmbedder from the given ImageEmbedderOptions.

    Declaration

    Swift

    init(options: ImageEmbedderOptions) throws

    Parameters

    options

    The options of type ImageEmbedderOptions to use for configuring the ImageEmbedder.

    Return Value

    A new instance of ImageEmbedder with the given options. nil if there is an error in initializing the image embedder.

  • Performs embedding extraction on the provided MPImage using the whole image as region of interest. Rotation will be applied according to the orientation property of the provided MPImage. Only use this method when the ImageEmbedder is created with running mode, .image.

    This method supports embedding extraction on RGBA images. If your MPImage has a source type of .pixelBuffer or .sampleBuffer, the underlying pixel buffer must use kCVPixelFormatType_32BGRA as its pixel format.

    If your MPImage has a source type of .image ensure that the color space is RGB with an Alpha channel.

    Declaration

    Swift

    func embed(image: MPImage) throws -> ImageEmbedderResult