TensorImage

    public class TensorImage

    TensorImage is the wrapper class for Image object. When using image processing utils in TFLite.support library, it's common to convert image objects in variant types to TensorImage at first.

    At present, only RGB images are supported, and the A channel is always ignored.

    Details of data storage: a TensorImage object may have 2 potential sources of truth: a Bitmap or a TensorBuffer . TensorImage maintains the state and only converts one to the other when needed. A typical use case of TensorImage is to first load a Bitmap image, then process it using ImageProcessor , and finally get the underlying ByteBuffer of the TensorBuffer and feed it into the TFLite interpreter.

    IMPORTANT: to achieve the best performance, TensorImage avoids copying data whenever it's possible. Therefore, it doesn't own its data. Callers should not modify data objects those are passed to load(Bitmap) or load(TensorBuffer, ColorSpaceType) .

    IMPORTANT: all methods are not proved thread-safe.

    See Also

    Public Constructors

    *
    TensorImage ()
    Initializes a TensorImage object.
    *
    TensorImage ( DataType dataType)
    Initializes a TensorImage object with the specified data type.

    Public Methods

    static TensorImage
    createFrom ( TensorImage src, DataType dataType)
    Creates a deep-copy of a given TensorImage with the desired data type.
    static TensorImage
    fromBitmap (Bitmap bitmap)
    Initializes a TensorImage object of DataType.UINT8 with a Bitmap .
    Bitmap
    getBitmap ()
    Returns a Bitmap representation of this TensorImage .
    ByteBuffer
    getBuffer ()
    Returns a ByteBuffer representation of this TensorImage with the expected data type.
    ColorSpaceType
    getColorSpaceType ()
    Gets the color space type of this TensorImage .
    DataType
    getDataType ()
    Gets the data type of this TensorImage .
    int
    getHeight ()
    Gets the image height.
    Image
    getMediaImage ()
    Returns an Image representation of this TensorImage .
    TensorBuffer
    getTensorBuffer ()
    Returns a TensorBuffer representation of this TensorImage with the expected data type.
    int
    getWidth ()
    Gets the image width.
    void
    load ( TensorBuffer buffer, ColorSpaceType colorSpaceType)
    Loads a TensorBuffer containing pixel values with the specific ColorSpaceType .
    void
    load (Bitmap bitmap)
    Loads a Bitmap image object into this TensorImage .
    void
    load (int[] pixels, int[] shape)
    Loads an int array as RGB pixels into this TensorImage , representing the pixels inside.
    void
    load (float[] pixels, int[] shape)
    Loads a float array as RGB pixels into this TensorImage , representing the pixels inside.
    void
    load ( ByteBuffer buffer, ImageProperties imageProperties)
    Loads a ByteBuffer containing pixel values with the specific ImageProperties .
    void
    load ( TensorBuffer buffer, ImageProperties imageProperties)
    Loads a TensorBuffer containing pixel values with the specific ImageProperties .
    void
    load (Image image)
    Loads an Image object into this TensorImage .

    Inherited Methods

    Public Constructors

    public TensorImage ()

    Initializes a TensorImage object.

    Note: the data type of this TensorImage is DataType.UINT8 . Use TensorImage(DataType) if other data types are preferred.

    public TensorImage ( DataType dataType)

    Initializes a TensorImage object with the specified data type.

    When getting a TensorBuffer or a ByteBuffer from this TensorImage , such as using getTensorBuffer() and getBuffer() , the data values will be converted to the specified data type.

    Note: the shape of a TensorImage is not fixed. It can be adjusted to the shape of the image being loaded to this TensorImage .

    Parameters
    dataType
    the expected data type of the resulting TensorBuffer . The type is always fixed during the lifetime of the TensorImage . To convert the data type, use createFrom(TensorImage, DataType) to create a copy and convert data type at the same time.
    Throws
    IllegalArgumentException
    if dataType is neither DataType.UINT8 nor DataType.FLOAT32

    Public Methods

    public static TensorImage createFrom ( TensorImage src, DataType dataType)

    Creates a deep-copy of a given TensorImage with the desired data type.

    Parameters
    src the TensorImage to copy from
    dataType the expected data type of newly created TensorImage
    Returns
    • a TensorImage whose data is copied from src and data type is dataType

    public static TensorImage fromBitmap (Bitmap bitmap)

    Initializes a TensorImage object of DataType.UINT8 with a Bitmap .

    Parameters
    bitmap
    See Also

    public Bitmap getBitmap ()

    Returns a Bitmap representation of this TensorImage .

    Numeric casting and clamping will be applied if the stored data is not uint8.

    Note that, the reliable way to get pixels from an ALPHA_8 Bitmap is to use copyPixelsToBuffer . Bitmap methods such as, `setPixels()` and `getPixels` do not work.

    Important: it's only a reference. DO NOT MODIFY. We don't create a copy here for performance concern, but if modification is necessary, please make a copy.

    Returns
    Throws
    IllegalStateException
    if the TensorImage never loads data

    public ByteBuffer getBuffer ()

    Returns a ByteBuffer representation of this TensorImage with the expected data type.

    Numeric casting and clamping will be applied if the stored data is different from the data type of the TensorImage .

    Important: it's only a reference. DO NOT MODIFY. We don't create a copy here for performance concern, but if modification is necessary, please make a copy.

    It's essentially a short cut for getTensorBuffer().getBuffer() .

    Returns
    • a reference to a ByteBuffer which holds the image data
    Throws
    IllegalStateException
    if the TensorImage never loads data

    public ColorSpaceType getColorSpaceType ()

    Gets the color space type of this TensorImage .

    Throws
    IllegalStateException
    if the TensorImage never loads data

    public DataType getDataType ()

    Gets the data type of this TensorImage .

    Returns

    public int getHeight ()

    Gets the image height.

    Throws
    IllegalStateException if the TensorImage never loads data
    IllegalArgumentException if the underlying data is corrupted

    public Image getMediaImage ()

    Returns an Image representation of this TensorImage .

    This method only works when the TensorImage is backed by an Image , meaning you need to first load an Image through load(Image) .

    Important: it's only a reference. DO NOT MODIFY. We don't create a copy here for performance concern, but if modification is necessary, please make a copy.

    Returns
    Throws
    IllegalStateException
    if the TensorImage never loads data

    public TensorBuffer getTensorBuffer ()

    Returns a TensorBuffer representation of this TensorImage with the expected data type.

    Numeric casting and clamping will be applied if the stored data is different from the data type of the TensorImage .

    Important: it's only a reference. DO NOT MODIFY. We don't create a copy here for performance concern, but if modification is necessary, please make a copy.

    Returns
    Throws
    IllegalStateException
    if the TensorImage never loads data

    public int getWidth ()

    Gets the image width.

    Throws
    IllegalStateException if the TensorImage never loads data
    IllegalArgumentException if the underlying data is corrupted

    public void load ( TensorBuffer buffer, ColorSpaceType colorSpaceType)

    Loads a TensorBuffer containing pixel values with the specific ColorSpaceType .

    Only supports ColorSpaceType.RGB and ColorSpaceType.GRAYSCALE . Use load(TensorBuffer, ImageProperties) for other color space types.

    Note: if the data type of buffer does not match that of this TensorImage , numeric casting and clamping will be applied when calling getTensorBuffer() and getBuffer() .

    Parameters
    buffer the TensorBuffer to be loaded. Its shape should be either (h, w, 3) or (1, h, w, 3) for RGB images, and either (h, w) or (1, h, w) for GRAYSCALE images
    colorSpaceType
    Throws
    IllegalArgumentException
    if the shape of buffer does not match the color space type, or if the color space type is not supported

    public void load (Bitmap bitmap)

    Loads a Bitmap image object into this TensorImage .

    Note: if the TensorImage has data type other than DataType.UINT8 , numeric casting and clamping will be applied when calling getTensorBuffer() and getBuffer() , where the Bitmap will be converted into a TensorBuffer .

    Important: when loading a bitmap, DO NOT MODIFY the bitmap from the caller side anymore. The TensorImage object will rely on the bitmap. It will probably modify the bitmap as well. In this method, we perform a zero-copy approach for that bitmap, by simply holding its reference. Use bitmap.copy(bitmap.getConfig(), true) to create a copy if necessary.

    Note: to get the best performance, please load images in the same shape to avoid memory re-allocation.

    Parameters
    bitmap
    Throws
    IllegalArgumentException
    if bitmap is not in ARGB_8888

    public void load (int[] pixels, int[] shape)

    Loads an int array as RGB pixels into this TensorImage , representing the pixels inside.

    Note: numeric casting and clamping will be applied to convert the values into the data type of this TensorImage when calling getTensorBuffer() and getBuffer() .

    Parameters
    pixels the RGB pixels representing the image
    shape the shape of the image, should either in form (h, w, 3), or in form (1, h, w, 3)
    Throws
    IllegalArgumentException
    if the shape is neither (h, w, 3) nor (1, h, w, 3)

    public void load (float[] pixels, int[] shape)

    Loads a float array as RGB pixels into this TensorImage , representing the pixels inside.

    Note: if the TensorImage has a data type other than DataType.FLOAT32 , numeric casting and clamping will be applied when calling getTensorBuffer() and getBuffer() .

    Parameters
    pixels the RGB pixels representing the image
    shape the shape of the image, should either in form (h, w, 3), or in form (1, h, w, 3)
    Throws
    IllegalArgumentException
    if the shape is neither (h, w, 3) nor (1, h, w, 3)

    public void load ( ByteBuffer buffer, ImageProperties imageProperties)

    Loads a ByteBuffer containing pixel values with the specific ImageProperties .

    Note: if the data type of buffer does not match that of this TensorImage , numeric casting and clamping will be applied when calling getTensorBuffer() and getBuffer() .

    Parameters
    buffer
    imageProperties
    Throws
    IllegalArgumentException
    if buffer size is less than the image size indicated by image height, width, and color space type in ImageProperties

    public void load ( TensorBuffer buffer, ImageProperties imageProperties)

    Loads a TensorBuffer containing pixel values with the specific ImageProperties .

    The shape of the TensorBuffer will not be used to determine image height and width. Set image properties through ImageProperties .

    Note: if the data type of buffer does not match that of this TensorImage , numeric casting and clamping will be applied when calling getTensorBuffer() and getBuffer() .

    Parameters
    buffer
    imageProperties
    Throws
    IllegalArgumentException
    if buffer size is less than the image size indicated by image height, width, and color space type in ImageProperties

    public void load (Image image)

    Loads an Image object into this TensorImage .

    The main usage of this method is to load an Image object as model input to the https://www.tensorflow.org/lite/inference_with_metadata/task_library/overview . TensorImage backed by Image is not supported by ImageProcessor .

    * @throws IllegalArgumentException if the ImageFormat of image is not YUV_420_888

    Parameters
    image
    Create a Mobile Website
    View Site in Mobile | Classic
    Share by: