ImageClassifier
class ImageClassifier : NSObject
@brief Performs classification on images.
The API expects a TFLite model with optional, but strongly recommended, TFLite Model Metadata..
The API supports models with one image input tensor and one or more output tensors. To be more specific, here are the requirements.
Input tensor (kTfLiteUInt8/kTfLiteFloat32)
- image input of size
[batch x height x width x channels]
. - batch inference is not supported (
batch
is required to be 1). - only RGB inputs are supported (
channels
is required to be 3). - if type is kTfLiteFloat32, NormalizationOptions are required to be attached to the metadata for input normalization.
At least one output tensor with: (kTfLiteUInt8/kTfLiteFloat32)
N
classes and either 2 or 4 dimensions, i.e.[1 x N]
or[1 x 1 x 1 x N]
- optional (but recommended) label map(s) as AssociatedFiles with type TENSOR_AXIS_LABELS,
containing one label per line. The first such AssociatedFile (if any) is used to fill the
class_name
field of the results. Thedisplay_name
field is filled from the AssociatedFile (if any) whose locale matches thedisplay_names_locale
field of theImageClassifierOptions
used at creation time (“en” by default, i.e. English). If none of these are available, only theindex
field of the results will be filled. - optional score calibration can be attached using ScoreCalibrationOptions and an AssociatedFile with type TENSOR_AXIS_SCORE_CALIBRATION. See metadata_schema.fbs [1] for more details.
-
Creates a new instance of
ImageClassifier
from an absolute path to a TensorFlow Lite model file stored locally on the device and the defaultImageClassifierOptions
.Declaration
Swift
convenience init(modelPath: String) throws
Parameters
modelPath
An absolute path to a TensorFlow Lite model file stored locally on the device.
Return Value
A new instance of
ImageClassifier
with the given model path.nil
if there is an error in initializing the image classifier. -
Creates a new instance of
ImageClassifier
from the givenImageClassifierOptions
.Declaration
Swift
init(options: ImageClassifierOptions) throws
Parameters
options
The options of type
ImageClassifierOptions
to use for configuring theImageClassifier
.Return Value
A new instance of
ImageClassifier
with the given options.nil
if there is an error in initializing the image classifier. -
Performs image classification on the provided
MPImage
using the whole image as region of interest. Rotation will be applied according to theorientation
property of the providedMPImage
. Only use this method when theImageClassifier
is created with running mode,.image
.This method supports classification of RGBA images. If your
MPImage
has a source type of.pixelBuffer
or.sampleBuffer
, the underlying pixel buffer must usekCVPixelFormatType_32BGRA
as its pixel format.If your
MPImage
has a source type of.image
ensure that the color space is RGB with an Alpha channel.Declaration
Swift
func classify(image: MPImage) throws -> ImageClassifierResult
Parameters
image
The
MPImage
on which image classification is to be performed.Return Value
An
ImageClassifierResult
object that contains a list of image classifications. -
Performs image classification on the provided
MPImage
cropped to the specified region of interest. Rotation will be applied on the cropped image according to theorientation
property of the providedMPImage
. Only use this method when theImageClassifier
is created with running mode,.image
.This method supports classification of RGBA images. If your
MPImage
has a source type of.pixelBuffer
or.sampleBuffer
, the underlying pixel buffer must usekCVPixelFormatType_32BGRA
as its pixel format.If your
MPImage
has a source type of.image
ensure that the color space is RGB with an Alpha channel.Declaration
Swift
func classify(image: MPImage, regionOfInterest roi: CGRect) throws -> ImageClassifierResult
Parameters
image
The
MPImage
on which image classification is to be performed.roi
A
CGRect
specifying the region of interest within the givenMPImage
, on which image classification should be performed.Return Value
An
ImageClassifierResult
object that contains a list of image classifications. -
Performs image classification on the provided video frame of type
MPImage
using the whole image as region of interest. Rotation will be applied according to theorientation
property of the providedMPImage
. Only use this method when theImageClassifier
is created with running mode.video
.It’s required to provide the video frame’s timestamp (in milliseconds). The input timestamps must be monotonically increasing.
This method supports classification of RGBA images. If your
MPImage
has a source type of.pixelBuffer
or.sampleBuffer
, the underlying pixel buffer must usekCVPixelFormatType_32BGRA
as its pixel format.If your
MPImage
has a source type of.image
ensure that the color space is RGB with an Alpha channel.Declaration
Swift
func classify(videoFrame image: MPImage, timestampInMilliseconds: Int) throws -> ImageClassifierResult
Parameters
image
The
MPImage
on which image classification is to be performed.timestampInMilliseconds
The video frame’s timestamp (in milliseconds). The input timestamps must be monotonically increasing.
Return Value
An
ImageClassifierResult
object that contains a list of image classifications. -
Performs image classification on the provided video frame of type
MPImage
cropped to the specified region of interest. Rotation will be applied according to theorientation
property of the providedMPImage
. Only use this method when theImageClassifier
is created with.video
.It’s required to provide the video frame’s timestamp (in milliseconds). The input timestamps must be monotonically increasing.
This method supports classification of RGBA images. If your
MPImage
has a source type of.pixelBuffer
or.sampleBuffer
, the underlying pixel buffer must usekCVPixelFormatType_32BGRA
as its pixel format.If your
MPImage
has a source type of.image
ensure that the color space is RGB with an Alpha channel.Declaration
Swift
func classify(videoFrame image: MPImage, timestampInMilliseconds: Int, regionOfInterest roi: CGRect) throws -> ImageClassifierResult
Parameters
image
A live stream image data of type
MPImage
on which image classification is to be performed.timestampInMilliseconds
The video frame’s timestamp (in milliseconds). The input timestamps must be monotonically increasing.
roi
A
CGRect
specifying the region of interest within the video frame of typeMPImage
, on which image classification should be performed.Return Value
An
ImageClassifierResult
object that contains a list of image classifications. -
Sends live stream image data of type
MPImage
to perform image classification using the whole image as region of interest. Rotation will be applied according to theorientation
property of the providedMPImage
. Only use this method when theImageClassifier
is created with running mode.liveStream
.The object which needs to be continuously notified of the available results of image classification must confirm to