ImageEmbedder
class ImageEmbedder : NSObject
@brief Performs embedding extraction on images.
The API expects a TFLite model with optional, but strongly recommended, TFLite Model Metadata..
The API supports models with one image input tensor and one or more output tensors. To be more specific, here are the requirements.
Input image tensor (kTfLiteUInt8/kTfLiteFloat32)
- image input of size
[batch x height x width x channels]
. - batch inference is not supported (
batch
is required to be 1). - only RGB inputs are supported (
channels
is required to be 3). - if type is kTfLiteFloat32, NormalizationOptions are required to be attached to the metadata for input normalization.
At least one output tensor (kTfLiteUInt8/kTfLiteFloat32) with shape [1 x N]
where N
is the number of dimensions in the produced embeddings.
-
Creates a new instance of
ImageEmbedder
from an absolute path to a TensorFlow Lite model file stored locally on the device and the defaultImageEmbedderOptions
.Declaration
Swift
convenience init(modelPath: String) throws
Parameters
modelPath
An absolute path to a TensorFlow Lite model file stored locally on the device.
Return Value
A new instance of
ImageEmbedder
with the given model path.nil
if there is an error in initializing the image embedder. -
Creates a new instance of
ImageEmbedder
from the givenImageEmbedderOptions
.Declaration
Swift
init(options: ImageEmbedderOptions) throws
Parameters
options
The options of type
ImageEmbedderOptions
to use for configuring theImageEmbedder
.Return Value
A new instance of
ImageEmbedder
with the given options.nil
if there is an error in initializing the image embedder. -
Performs embedding extraction on the provided
MPImage
using the whole image as region of interest. Rotation will be applied according to theorientation
property of the providedMPImage
. Only use this method when theImageEmbedder
is created with running mode,.image
.This method supports embedding extraction on RGBA images. If your
MPImage
has a source type of.pixelBuffer
or.sampleBuffer
, the underlying pixel buffer must usekCVPixelFormatType_32BGRA
as its pixel format.If your
MPImage
has a source type of.image
ensure that the color space is RGB with an Alpha channel.Declaration
Swift
func embed(image: MPImage) throws -> ImageEmbedderResult
Parameters
image
The
MPImage
on which embedding extraction is to be performed.Return Value
An
ImageEmbedderResult
object that contains a list of embedding extraction. -
Performs embedding extraction on the provided
MPImage
cropped to the specified region of interest. Rotation will be applied on the cropped image according to theorientation
property of the providedMPImage
. Only use this method when theImageEmbedder
is created with running mode,.image
.This method supports embedding extraction on RGBA images. If your
MPImage
has a source type of.pixelBuffer
or.sampleBuffer
, the underlying pixel buffer must usekCVPixelFormatType_32BGRA
as its pixel format.If your
MPImage
has a source type of.image
ensure that the color space is RGB with an Alpha channel.Declaration
Swift
func embed(image: MPImage, regionOfInterest roi: CGRect) throws -> ImageEmbedderResult
Parameters
image
The
MPImage
on which embedding extraction is to be performed.roi
A
CGRect
specifying the region of interest within the givenMPImage
, on which embedding extraction should be performed.Return Value
An
ImageEmbedderResult
object that contains a list of generated image embeddings. -
Performs embedding extraction on the provided video frame of type
MPImage
using the whole image as region of interest. Rotation will be applied according to theorientation
property of the providedMPImage
. Only use this method when theImageEmbedder
is created with running mode.video
.It’s required to provide the video frame’s timestamp (in milliseconds). The input timestamps must be monotonically increasing.
This method supports embedding extraction on RGBA images. If your
MPImage
has a source type of.pixelBuffer
or.sampleBuffer
, the underlying pixel buffer must usekCVPixelFormatType_32BGRA
as its pixel format.If your
MPImage
has a source type of.image
ensure that the color space is RGB with an Alpha channel.Declaration
Swift
func embed(videoFrame image: MPImage, timestampInMilliseconds: Int) throws -> ImageEmbedderResult
Parameters
image
The
MPImage
on which embedding extraction is to be performed.timestampInMilliseconds
The video frame’s timestamp (in milliseconds). The input timestamps must be monotonically increasing.
Return Value
An
ImageEmbedderResult
object that contains a list of generated image embeddings. -
Performs embedding extraction on the provided video frame of type
MPImage
cropped to the specified region of interest. Rotation will be applied according to theorientation
property of the providedMPImage
. Only use this method when theImageEmbedder
is created with.video
.It’s required to provide the video frame’s timestamp (in milliseconds). The input timestamps must be monotonically increasing.
This method supports embedding extraction on RGBA images. If your
MPImage
has a source type of.pixelBuffer
or.sampleBuffer
, the underlying pixel buffer must usekCVPixelFormatType_32BGRA
as its pixel format.If your
MPImage
has a source type of.image
ensure that the color space is RGB with an Alpha channel.Declaration
Swift
func embed(videoFrame image: MPImage, timestampInMilliseconds: Int, regionOfInterest roi: CGRect) throws -> ImageEmbedderResult
Parameters
image
A live stream image data of type
MPImage
on which embedding extraction is to be performed.timestampInMilliseconds
The video frame’s timestamp (in milliseconds). The input timestamps must be monotonically increasing.
roi
A
CGRect
specifying the region of interest within the video frame of typeMPImage
, on which embedding extraction should be performed.Return Value
An
ImageEmbedderResult
object that contains a list of generated image embeddings. -
Sends live stream image data of type
MPImage
to perform embedding extraction using the whole image a