ImageEmbedder

public final class ImageEmbedder

Performs embedding extraction on images.

The API expects a TFLite model with optional, but strongly recommended, TFLite Model Metadata..

The API supports models with one image input tensor and one or more output tensors. To be more specific, here are the requirements.

  • Input image tensor (kTfLiteUInt8/kTfLiteFloat32)
    • image input of size [batch x height x width x channels].
    • batch inference is not supported (batch is required to be 1).
    • only RGB inputs are supported (channels is required to be 3).
    • if type is kTfLiteFloat32, NormalizationOptions are required to be attached to the metadata for input normalization.
  • At least one output tensor (kTfLiteUInt8/kTfLiteFloat32) with shape [1 x N] where N is the number of dimensions in the produced embeddings.

Nested Classes

class ImageEmbedder.ImageEmbedderOptions Options for setting up and ImageEmbedder. 

Public Methods

static double
cosineSimilarity(Embedding u, Embedding v)
Utility function to compute cosine similarity between two Embedding objects.
static ImageEmbedder
createFromBuffer(Context context, ByteBuffer modelBuffer)
Creates an ImageEmbedder instance from a model buffer and default ImageEmbedder.ImageEmbedderOptions.
static ImageEmbedder
createFromFile(Context context, String modelPath)
Creates an ImageEmbedder instance from a model file and default ImageEmbedder.ImageEmbedderOptions.
static ImageEmbedder
createFromFile(Context context, File modelFile)
Creates an ImageEmbedder instance from a model file and default ImageEmbedder.ImageEmbedderOptions.
static ImageEmbedder
createFromOptions(Context context, ImageEmbedder.ImageEmbedderOptions options)
Creates an ImageEmbedder instance from an ImageEmbedder.ImageEmbedderOptions instance.
ImageEmbedderResult
embed(MPImage image, ImageProcessingOptions imageProcessingOptions)
Performs embedding extraction on the provided single image.
ImageEmbedderResult
embed(MPImage image)
Performs embedding extraction on the provided single image with default image processing options, i.e.
void
embedAsync(MPImage image, ImageProcessingOptions imageProcessingOptions, long timestampMs)
Sends live image data to perform embedding extraction, and the results will be available via the OutputHandler.ResultListener provided in the ImageEmbedder.ImageEmbedderOptions.
void
embedAsync(MPImage image, long timestampMs)
Sends live image data to perform embedding extraction with default image processing options, i.e.
ImageEmbedderResult
embedForVideo(MPImage image, ImageProcessingOptions imageProcessingOptions, long timestampMs)
Performs embedding extraction on the provided video frame.
ImageEmbedderResult
embedForVideo(MPImage image, long timestampMs)
Performs embedding extraction on the provided video frame with default image processing options, i.e.

Inherited Methods

Public Methods

public static double cosineSimilarity (Embedding u, Embedding v)

Utility function to compute cosine similarity between two Embedding objects.

Parameters
u
v
Throws
IllegalArgumentException if the embeddings are of different types (float vs. quantized), have different sizes, or have an L2-norm of 0.

public static ImageEmbedder createFromBuffer (Context context, ByteBuffer modelBuffer)

Creates an ImageEmbedder instance from a model buffer and default ImageEmbedder.ImageEmbedderOptions.

Parameters
context an Android ERROR(/Context).
modelBuffer a direct ByteBuffer or a ERROR(/MappedByteBuffer) of the embedding model.
Throws
if there is an error during ImageEmbedder creation.

public static ImageEmbedder createFromFile (Context context, String modelPath)

Creates an ImageEmbedder instance from a model file and default ImageEmbedder.ImageEmbedderOptions.

Parameters
context an Android ERROR(/Context).
modelPath path to the embedding model in the assets.
Throws
if there is an error during ImageEmbedder creation.

public static