MediaPipeTasksVision Framework Reference

MPImage

class MPImage : NSObject

An image used in on-device machine learning using MediaPipe Task library.

  • Width of the image in pixels.

    Declaration

    Swift

    var width: CGFloat { get }

  • Height of the image in pixels.

    Declaration

    Swift

    var height: CGFloat { get }

  • The display orientation of the image. If imageSourceType is .image, the default value is image.imageOrientation; otherwise the default value is UIImage.Orientation.up. If the MPImage is being used as input for any MediaPipe vision tasks and is set to any orientation other than UIImage.Orientation.up, inference will be performed on a rotated copy of the image according to the orientation.

    Declaration

    Swift

    var orientation: UIImage.Orientation { get }

  • The type of the image source.

    Declaration

    Swift

    var imageSourceType: MPImageSourceType { get }

  • The source image. nil if imageSourceType is not .image.

    Declaration

    Swift

    var image: UIImage? { get }

  • The source pixel buffer. nil if imageSourceType is not .pixelBuffer.

    Declaration

    Swift

    var pixelBuffer: CVPixelBuffer? { get }

  • The source sample buffer. nil if imageSourceType is not .sampleBuffer.

    Declaration

    Swift

    var sampleBuffer: CMSampleBuffer? { get }

  • Initializes an MPImage object with the given UIImage.

    The orientation of the newly created MPImage will be equal to the imageOrientation of UIImage and when sent to the vision tasks for inference, rotation will be applied accordingly. To create an MPImage with an orientation different from its imageOrientation, please use MPImage(uiImage:orientation:)s.

    Declaration

    Swift

    convenience init(uiImage image: UIImage) throws

    Parameters

    image

    The image to use as the source. Its CGImage property must not be NULL.
    error

    An optional error parameter populated when there is an error in initializing the MPImage.

    Return Value

    A new MPImage instance with the given image as the source. nil if the given image is nil or invalid.

  • Initializes an MPImage object with the given UIImage and orientation.

    The given orientation will be used to calculate the rotation to be applied to the UIImage before inference is performed on it by the vision tasks. The imageOrientation stored in the UIImage is ignored when MPImage objects created by this method are sent to the vision tasks for inference. Use MPImage(uiImage:) to initialize images with the imageOrientation of UIImage.

    If the newly created MPImage is used as input for any MediaPipe vision tasks, inference will be performed on a copy of the image rotated according to the orientation.

    Declaration

    Swift

    init(uiImage image: UIImage, orientation: UIImage.Orientation) throws

    Parameters

    image

    The image to use as the source. Its CGImage property must not be NULL.
    orientation

    The display orientation of the image. This will be stored in the property orientation MPImage and will override the imageOrientation of the passed in UIImage.
    error

    An optional error parameter populated when there is an error in initializing the MPImage.

    Return Value

    A new MPImage instance with the given image as the source. nil if the given image is nil or invalid.

  • Initializes an MPImage object with the given pixel buffer.

    The orientation of the newly created MPImage will be UIImageOrientationUp. Hence, if this image is used as input for any MediaPipe vision tasks, inference will be performed on the it without any rotation. To create an MPImage with a different orientation, please use MPImage(pixelBuffer:orientation:).

    Declaration

    Swift

    convenience init(pixelBuffer: CVPixelBuffer) throws

    Parameters

    pixelBuffer

    The pixel buffer to use as the source. It will be retained by the new MPImage instance for the duration of its lifecycle.
    error

    An optional error parameter populated when there is an error in initializing the MPImage.

    Return Value

    A new MPImage instance with the given pixel buffer as the source. nil if the given pixel buffer is nil or invalid.

  • Initializes an MPImage object with the given pixel buffer and orientation.

    If the newly created MPImage is used as input for any MediaPipe vision tasks, inference will be performed on a copy of the image rotated according to the orientation.

    Declaration

    Swift

    init(pixelBuffer: CVPixelBuffer, orientation: UIImage.Orientation) throws

    Parameters

    pixelBuffer

    The pixel buffer to use as the source. It will be retained by the new MPImage instance for the duration of its lifecycle.
    orientation

    The display orientation of the image.
    error

    An optional error parameter populated when there is an error in initializing the MPImage.

    Return Value

    A new MPImage instance with the given orientation and pixel buffer as the source. nil if the given pixel buffer is nil or invalid.

  • Initializes an MPImage object with the given sample buffer.

    The orientation of the newly created MPImage will be UIImageOrientationUp. Hence, if this image is used as input for any MediaPipe vision tasks, inference will be performed on the it without any rotation. To create an MPImage with a different orientation, please use MPImage(sampleBuffer:orientation:).

    Declaration

    Swift

    convenience init(sampleBuffer: CMSampleBuffer) throws

    Parameters

    sampleBuffer

    The sample buffer to use as the source. It will be retained by the new MPImage instance for the duration of its lifecycle. The sample buffer must be based on a pixel buffer (not compressed data). In practice, it should be the video output of the camera on an iOS device, not other arbitrary types of CMSampleBuffers.

    Return Value

    A new MPImage instance with the given sample buffer as the source. nil if the given sample buffer is nil or invalid.

  • Initializes an MPImage object with the given sample buffer and orientation.

    If the newly created MPImage is used as input for any MediaPipe vision tasks, inference will be performed on a copy of the image rotated according to the orientation.

    Declaration

    Swift

    init(sampleBuffer: CMSampleBuffer, orientation: UIImage.Orientation) throws

    Parameters

    sampleBuffer

    The sample buffer to use as the source. It will be retained by the new MPImage instance for the duration of its lifecycle. The sample buffer must be based on a pixel buffer (not compressed data). In practice, it should be the video output of the camera on an iOS device, not other arbitrary types of CMSampleBuffers.
    orientation

    The display orientation of the image.

    Return Value

    A new MPImage instance with the given orientation and sample buffer as the source. nil if the given sample buffer is nil or invalid.

  • Unavailable.

  • Undocumented