AudioClassifier

public final class AudioClassifier

Performs audio classification on audio clips or audio stream.

This API expects a TFLite model with mandatory TFLite Model Metadata that contains the mandatory AudioProperties of the solo input audio tensor and the optional (but recommended) label items as AssociatedFiles with type TENSOR_AXIS_LABELS per output classification tensor.

Input tensor: (kTfLiteFloat32)

  • input audio buffer of size `[batch * samples]`.
  • batch inference is not supported (`batch` is required to be 1).
  • for multi-channel models, the channels need be interleaved.

At least one output tensor with: (kTfLiteFloat32)

  • `[1 x N]` array with `N` represents the number of categories.
  • optional (but recommended) label items as AssociatedFiles with type TENSOR_AXIS_LABELS, containing one label per line. The first such AssociatedFile (if any) is used to fill the `category_name` field of the results. The `display_name` field is filled from the AssociatedFile (if any) whose locale matches the `display_names_locale` field of the `AudioClassifierOptions` used at creation time ("en" by default, i.e. English). If none of these are available, only the `index` field of the results will be filled.

Nested Classes

class AudioClassifier.AudioClassifierOptions Options for setting up and AudioClassifier. 

Public Methods

AudioClassifierResult
classify(AudioData audioClip)
void
classifyAsync(AudioData audioBlock, long timestampMs)
static AudioClassifier
createFromBuffer(Context context, ByteBuffer modelBuffer)
Creates an AudioClassifier instance from a model buffer and default AudioClassifier.AudioClassifierOptions.
static AudioClassifier
createFromFile(Context context, String modelPath)
Creates an AudioClassifier instance from a model file and default AudioClassifier.AudioClassifierOptions.
static AudioClassifier
createFromFile(Context context, File modelFile)
Creates an AudioClassifier instance from a model file and default AudioClassifier.AudioClassifierOptions.
static AudioClassifier
createFromOptions(Context context, AudioClassifier.AudioClassifierOptions options)
Creates an AudioClassifier instance from an AudioClassifier.AudioClassifierOptions instance.

Inherited Methods

com.google.mediapipe.tasks.audio.core.BaseAudioTaskApi
void
close()
Closes and cleans up the MediaPipe audio task.
AudioRecord
createAudioRecord()
Creates an AudioRecord instance to record audio stream that has mono channel at sample rate at sample rate 16kHz, the sample rate required for models like Yamnet.
AudioRecord
createAudioRecord(int numChannels, int sampleRate, int requiredInputBufferSize)
Creates an AudioRecord instance to record audio stream.
boolean
equals(Object arg0)
final Class<?>
getClass()
int
hashCode()
final void
notify()
final void
notifyAll()
String
toString()
final void
wait(long arg0, int arg1)
final void
wait(long arg0)
final void
wait()
abstract void
close()

Public Methods

public AudioClassifierResult classify (AudioData audioClip)

Parameters
audioClip

public void classifyAsync (AudioData audioBlock, long timestampMs)

Parameters
audioBlock
timestampMs

public static AudioClassifier createFromBuffer (Context context, ByteBuffer modelBuffer)

Creates an AudioClassifier instance from a model buffer and default AudioClassifier.AudioClassifierOptions.

Parameters
context an Android ERROR(/Context).
modelBuffer a direct ByteBuffer or a ERROR(/MappedByteBuffer) of the classification model.
Throws
if there is an error during AudioClassifier creation.

public static AudioClassifier createFromFile (Context context, String modelPath)

Creates an AudioClassifier instance from a model file and default AudioClassifier.AudioClassifierOptions.

Parameters
context an Android ERROR(/Context).
modelPath path to the classification model in the assets.
Throws
if there is an error during AudioClassifier creation.

public static AudioClassifier createFromFile (Context context, File modelFile)

Creates an AudioClassifier instance from a model file and default AudioClassifier.AudioClassifierOptions.

Parameters
context an Android ERROR(/Context).
modelFile the classification model File instance.
Throws
IOException if an I/O error occurs when opening the tflite model file.
if there is an error during AudioClassifier creation.

public static AudioClassifier createFromOptions (Context context, AudioClassifier.AudioClassifierOptions options)

Creates an AudioClassifier instance from an AudioClassifier.AudioClassifierOptions instance.

Parameters
context an Android ERROR(/Context).
options an AudioClassifier.AudioClassifierOptions instance.
Throws
if there is an error during AudioClassifier creation.