TensorImage

public class TensorImage

TensorImage is the wrapper class for Image object. When using image processing utils in TFLite.support library, it's common to convert image objects in variant types to TensorImage at first.

At present, only RGB images are supported, and the A channel is always ignored.

Details of data storage: a TensorImage object may have 2 potential sources of truth: a Bitmap or a TensorBuffer. TensorImage maintains the state and only converts one to the other when needed. A typical use case of TensorImage is to first load a Bitmap image, then process it using ImageProcessor, and finally get the underlying ByteBuffer of the TensorBuffer and feed it into the TFLite interpreter.

IMPORTANT: to achieve the best performance, TensorImage avoids copying data whenever it's possible. Therefore, it doesn't own its data. Callers should not modify data objects those are passed to load(Bitmap) or load(TensorBuffer, ColorSpaceType).

IMPORTANT: all methods are not proved thread-safe.

See Also

Public Constructors

TensorImage()
Initializes a TensorImage object.
TensorImage(DataType dataType)
Initializes a TensorImage object with the specified data type.

Public Methods

static TensorImage
createFrom(TensorImage src, DataType dataType)
Creates a deep-copy of a given TensorImage with the desired data type.
static TensorImage
fromBitmap(Bitmap bitmap)
Initializes a TensorImage object of DataType.UINT8 with a Bitmap .
Bitmap
getBitmap()
Returns a Bitmap representation of this TensorImage.
ByteBuffer
getBuffer()
Returns a ByteBuffer representation of this TensorImage with the expected data type.
ColorSpaceType
getColorSpaceType()
Gets the color space type of this TensorImage.
DataType
getDataType()
Gets the data type of this TensorImage.
int
getHeight()
Gets the image height.
Image
getMediaImage()
Returns an Image representation of this TensorImage.
TensorBuffer
getTensorBuffer()
Returns a TensorBuffer representation of this TensorImage with the expected data type.
int
getWidth()
Gets the image width.
void
load(TensorBuffer buffer, ColorSpaceType colorSpaceType)
Loads a TensorBuffer containing pixel values with the specific ColorSpaceType.
void
load(Bitmap bitmap)
Loads a Bitmap image object into this TensorImage.
void
load(int[] pixels, int[] shape)
Loads an int array as RGB pixels into this TensorImage, representing the pixels inside.
void
load(float[] pixels, int[] shape)
Loads a float array as RGB pixels into this TensorImage, representing the pixels inside.
void
load(ByteBuffer buffer, ImageProperties imageProperties)
Loads a ByteBuffer containing pixel values with the specific ImageProperties.
void
load(TensorBuffer buffer, ImageProperties imageProperties)
Loads a TensorBuffer containing pixel values with the specific ImageProperties.
void
load(Image image)
Loads an Image object into this TensorImage.

Inherited Methods

Public Constructors

public TensorImage ()

Initializes a TensorImage object.

Note: the data type of this TensorImage is DataType.UINT8. Use TensorImage(DataType) if other data types are preferred.

public TensorImage (DataType dataType)

Initializes a TensorImage object with the specified data type.

When getting a TensorBuffer or a ByteBuffer from this TensorImage, such as using getTensorBuffer() and getBuffer(), the data values will be converted to the specified data type.

Note: the shape of a TensorImage is not fixed. It can be adjusted to the shape of the image being loaded to this TensorImage.

Parameters
dataType the expected data type of the resulting TensorBuffer. The type is always fixed during the lifetime of the TensorImage. To convert the data type, use createFrom(TensorImage, DataType) to create a copy and convert data type at the same time.
Throws
IllegalArgumentException if dataType is neither DataType.UINT8 nor DataType.FLOAT32

Public Methods

public static TensorImage createFrom (TensorImage src, DataType dataType)

Creates a deep-copy of a given TensorImage with the desired data type.

Parameters
src the TensorImage to copy from
dataType the expected data type of newly created TensorImage
Returns
  • a TensorImage whose data is copied from src and data type is dataType

public static TensorImage fromBitmap (Bitmap bitmap)

Initializes a TensorImage object of DataType.UINT8 with a Bitmap .

Parameters
bitmap
See Also

public Bitmap getBitmap ()

Returns a Bitmap representation of this TensorImage.

Numeric casting and clamping will be applied if the stored data is not uint8.

Note that, the reliable way to get pixels from an ALPHA_8 Bitmap is to use copyPixelsToBuffer. Bitmap methods such as, `setPixels()` and `getPixels` do not work.

Important: it's only a reference. DO NOT MODIFY. We don't create a copy here for performance concern, but if modification is necessary, please make a copy.

Returns
Throws
IllegalStateException if the TensorImage never loads data

public ByteBuffer getBuffer ()

Returns a ByteBuffer representation of this TensorImage with the expected data type.

Numeric casting and clamping will be applied if the stored data is different from the data type of the TensorImage.

Important: it's only a reference. DO NOT MODIFY. We don't create a copy here for performance concern, but if modification is necessary, please make a copy.

It's essentially a short cut for getTensorBuffer().getBuffer().

Returns
  • a reference to a ByteBuffer which holds the image data
Throws
IllegalStateException if the TensorImage never loads data

public ColorSpaceType getColorSpaceType ()

Gets the color space type of this TensorImage.

Throws
IllegalStateException if the TensorImage never loads data

public DataType getDataType ()

Gets the data type of this TensorImage.

Returns

public int getHeight ()

Gets the image height.

Throws
IllegalStateException if the TensorImage never loads data
IllegalArgumentException if the underlying data is corrupted

public Image getMediaImage ()

Returns an Image representation of this TensorImage.

This method only works when the TensorImage is backed by an Image, meaning you need to first load an Image through load(Image).

Important: it's only a reference. DO NOT MODIFY. We don't create a copy here for performance concern, but if modification is necessary, please make a copy.

Returns
Throws
IllegalStateException if the TensorImage never loads data

public TensorBuffer getTensorBuffer ()

Returns a TensorBuffer representation of this TensorImage with the expected data type.

Numeric casting and clamping will be applied if the stored data is different from the data type of the TensorImage.

Important: it's only a reference. DO NOT MODIFY. We don't create a copy here for performance concern, but if modification is necessary, please make a copy.

Returns
Throws
IllegalStateException if the TensorImage never loads data

public int getWidth ()

Gets the image width.

Throws
IllegalStateException if the TensorImage never loads data
IllegalArgumentException if the underlying data is corrupted

public void load (TensorBuffer buffer, ColorSpaceType colorSpaceType)

Loads a TensorBuffer containing pixel values with the specific ColorSpaceType.

Only supports ColorSpaceType.RGB and ColorSpaceType.GRAYSCALE. Use load(TensorBuffer, ImageProperties) for other color space types.

Note: if the data type of buffer does not match that of this TensorImage, numeric casting and clamping will be applied when calling getTensorBuffer() and getBuffer().

Parameters
buffer the TensorBuffer to be loaded. Its shape should be either (h, w, 3) or (1, h, w, 3) for RGB images, and either (h, w) or (1, h, w) for GRAYSCALE images
colorSpaceType
Throws
IllegalArgumentException if the shape of buffer does not match the color space type, or if the color space type is not supported

public void load (Bitmap bitmap)

Loads a Bitmap image object into this TensorImage.

Note: if the TensorImage has data type other than DataType.UINT8, numeric casting and clamping will be applied when calling getTensorBuffer() and getBuffer(), where the Bitmap will be converted into a TensorBuffer.

Important: when loading a bitmap, DO NOT MODIFY the bitmap from the caller side anymore. The TensorImage object will rely on the bitmap. It will probably modify the bitmap as well. In this method, we perform a zero-copy approach for that bitmap, by simply holding its reference. Use bitmap.copy(bitmap.getConfig(), true) to create a copy if necessary.

Note: to get the best performance, please load images in the same shape to avoid memory re-allocation.

Parameters
bitmap
Throws
IllegalArgumentException if bitmap is not in ARGB_8888

public void load (int[] pixels, int[] shape)

Loads an int array as RGB pixels into this TensorImage, representing the pixels inside.

Note: numeric casting and clamping will be applied to convert the values into the data type of this TensorImage when calling getTensorBuffer() and getBuffer().

Parameters
pixels the RGB pixels representing the image
shape the shape of the image, should either in form (h, w, 3), or in form (1, h, w, 3)
Throws
IllegalArgumentException if the shape is neither (h, w, 3) nor (1, h, w, 3)

public void load (float[] pixels, int[] shape)

Loads a float array as RGB pixels into this TensorImage, representing the pixels inside.

Note: if the TensorImage has a data type other than DataType.FLOAT32, numeric casting and clamping will be applied when calling getTensorBuffer() and getBuffer().

Parameters
pixels the RGB pixels representing the image
shape the shape of the image, should either in form (h, w, 3), or in form (1, h, w, 3)
Throws
IllegalArgumentException if the shape is neither (h, w, 3) nor (1, h, w, 3)

public void load (ByteBuffer buffer, ImageProperties imageProperties)

Loads a ByteBuffer containing pixel values with the specific ImageProperties.

Note: if the data type of buffer does not match that of this TensorImage, numeric casting and clamping will be applied when calling getTensorBuffer() and getBuffer().

Parameters
buffer
imageProperties
Throws
IllegalArgumentException if buffer size is less than the image size indicated by image height, width, and color space type in ImageProperties

public void load (TensorBuffer buffer, ImageProperties imageProperties)

Loads a TensorBuffer containing pixel values with the specific ImageProperties.

The shape of the TensorBuffer will not be used to determine image height and width. Set image properties through ImageProperties.

Note: if the data type of buffer does not match that of this TensorImage, numeric casting and clamping will be applied when calling getTensorBuffer() and getBuffer().

Parameters
buffer
imageProperties
Throws
IllegalArgumentException if buffer size is less than the image size indicated by image height, width, and color space type in ImageProperties

public void load (Image image)

Loads an Image object into this TensorImage.

The main usage of this method is to load an Image object as model input to the https://www.tensorflow.org/lite/inference_with_metadata/task_library/overview. TensorImage backed by Image is not supported by ImageProcessor.

* @throws IllegalArgumentException if the ImageFormat of image is not YUV_420_888

Parameters
image