AudioEmbedder

public final class AudioEmbedder

Performs audio embedding extraction on audio clips or audio stream.

This API expects a TFLite model with mandatory TFLite Model Metadata that contains the mandatory AudioProperties of the solo input audio tensor and the optional (but recommended) label items as AssociatedFiles with type TENSOR_AXIS_LABELS per output classification tensor.

Input tensor: (kTfLiteFloat32)

  • input audio buffer of size `[batch * samples]`.
  • batch inference is not supported (`batch` is required to be 1).
  • for multi-channel models, the channels need be interleaved.

At least one output tensor with: (kTfLiteFloat32)

  • `N` components corresponding to the `N` dimensions of the returned feature vector for this output layer.
  • Either 2 or 4 dimensions, i.e. `[1 x N]` or `[1 x 1 x 1 x N]`.

Nested Classes

class AudioEmbedder.AudioEmbedderOptions Options for setting up and AudioEmbedder

Public Methods

static AudioEmbedder
createFromBuffer(Context context, ByteBuffer modelBuffer)
Creates an AudioEmbedder instance from a model buffer and default AudioEmbedder.AudioEmbedderOptions.
static AudioEmbedder
createFromFile(Context context, String modelPath)
Creates an AudioEmbedder instance from a model file and default AudioEmbedder.AudioEmbedderOptions.
static AudioEmbedder
createFromFile(Context context, File modelFile)
Creates an AudioEmbedder instance from a model file and default AudioEmbedder.AudioEmbedderOptions.
static AudioEmbedder
createFromOptions(Context context, AudioEmbedder.AudioEmbedderOptions options)
Creates an AudioEmbedder instance from an AudioEmbedder.AudioEmbedderOptions instance.
AudioEmbedderResult
embed(AudioData audioClip)
void
embedAsync(AudioData audioBlock, long timestampMs)

Inherited Methods

Public Methods

public static AudioEmbedder createFromBuffer (Context context, ByteBuffer modelBuffer)

Creates an AudioEmbedder instance from a model buffer and default AudioEmbedder.AudioEmbedderOptions.

Parameters
context an Android ERROR(/Context).
modelBuffer a direct ByteBuffer or a ERROR(/MappedByteBuffer) of the embedding model.
Throws
if there is an error during AudioEmbedder creation.

public static AudioEmbedder createFromFile (Context context, String modelPath)

Creates an AudioEmbedder instance from a model file and default AudioEmbedder.AudioEmbedderOptions.

Parameters
context an Android ERROR(/Context).
modelPath path to the embedding model in the assets.
Throws
if there is an error during AudioEmbedder creation.

public static AudioEmbedder createFromFile (Context context, File modelFile)

Creates an AudioEmbedder instance from a model file and default AudioEmbedder.AudioEmbedderOptions.

Parameters
context an Android ERROR(/Context).
modelFile the embedding model File instance.
Throws
IOException if an I/O error occurs when opening the tflite model file.
if there is an error during AudioEmbedder creation.

public static AudioEmbedder createFromOptions (Context context, AudioEmbedder.AudioEmbedderOptions options)

Creates an AudioEmbedder instance from an AudioEmbedder.AudioEmbedderOptions instance.

Parameters
context an Android ERROR(/Context).
options an AudioEmbedder.AudioEmbedderOptions instance.
Throws
if there is an error during AudioEmbedder creation.

public AudioEmbedderResult embed (AudioData audioClip)

Parameters
audioClip

public void embedAsync (AudioData audioBlock, long timestampMs)

Parameters
audioBlock
timestampMs