litert::TensorBuffer

#include <litert_tensor_buffer.h>

A C++ wrapper for LiteRtTensorBuffer, representing a tensor and its associated backing buffer.

Summary

Inheritance

Inherits from: internal::Handle< LiteRtTensorBuffer, LiteRtDestroyTensorBuffer >

Constructors and Destructors
TensorBuffer()

Public types
LockMode enum

Public functions
BufferType() const
Expected< TensorBufferType >
ClearEvent()
Expected< void >
Duplicate() const
Creates a duplicate of the current TensorBuffer object.
GetAhwb() const
litert::Expected< AHardwareBuffer * >
GetDmaBuf() const
GetEvent() const
GetGlBuffer() const
GetGlTexture() const
GetMetalBuffer() const
Expected< void * >
GetOpenClMemory() const
Expected< cl_mem >
GetVulkanMemory() const
Expected< HwMemoryHandle >
GetWebGpuBuffer() const
Expected< HwMemoryHandle >
HasEvent() const
bool
HasType(const RankedTensorType & type) const
bool
HasType(const LiteRtRankedTensorType & type) const
bool
IsMetalMemory() const
bool
Returns true if the tensor buffer is a Metal memory object.
IsOpenClMemory() const
bool
Returns true if the tensor buffer is an OpenCL memory object.
IsVulkanMemory() const
bool
Returns true if the tensor buffer is a Vulkan memory object.
IsWebGpuMemory() const
bool
Returns true if the tensor buffer is a WebGPU memory object.
Lock(LockMode mode)
Expected< void * >
Offset() const
Expected< size_t >
PackedSize() const
Expected< size_t >
Returns the size of the tensor buffer in packed bytes.
Read(absl::Span< T > data)
Expected< void >
Reads data from the tensor buffer into a user-provided absl::Span.
SetEvent(Event && event)
Expected< void >
Sets the C++ Event object for the tensor buffer.
Size() const
Expected< size_t >
Returns the size of the underlying hardware tensor buffer.
TensorType() const
Expected< RankedTensorType >
Unlock()
Expected< void >
Write(absl::Span< const T > data)
Expected< void >
Writes data from a user-provided absl::Span to the tensor buffer.

Public static functions
CreateFromAhwb(const Environment & env, const RankedTensorType & tensor_type, AHardwareBuffer *ahwb, size_t ahwb_offset)
Creates a TensorBuffer that wraps an Android Hardware Buffer.
CreateFromClBuffer(const Environment & env, const RankedTensorType & tensor_type, TensorBufferType buffer_type, cl_mem cl_memory, size_t size_bytes)
CreateFromGlBuffer(const Environment & env, const RankedTensorType & tensor_type, LiteRtGLenum target, LiteRtGLuint id, size_t size_bytes, size_t offset)
CreateFromGlTexture(const Environment & env, const RankedTensorType & tensor_type, LiteRtGLenum target, LiteRtGLuint id, LiteRtGLenum format, size_t size_bytes, LiteRtGLint layer)
CreateFromHostMemory(const Environment & env, const RankedTensorType & tensor_type, void *host_mem_addr, size_t buffer_size)
Creates a TensorBuffer that wraps the provided host memory.
CreateFromHostMemory(const RankedTensorType & tensor_type, void *host_mem_addr, size_t buffer_size)
CreateManaged(const Environment & env, TensorBufferType buffer_type, const RankedTensorType & tensor_type, size_t buffer_size)
Creates a managed TensorBuffer of a given buffer type and size.
CreateManagedHostMemory(const RankedTensorType & tensor_type, size_t buffer_size)
Creates a managed host memory TensorBuffer using the default environment (if applicable).
ToLiteRtLockMode(LockMode mode)
LiteRtTensorBufferLockMode
WrapCObject(LiteRtTensorBuffer tensor_buffer, OwnHandle owned)

Structs
litert::TensorBuffer::DmaBuf
litert::TensorBuffer::GlBuffer
litert::TensorBuffer::GlTexture

Public types

LockMode

 LockMode

Public functions

BufferType

Expected< TensorBufferType > BufferType() const

ClearEvent

Expected< void > ClearEvent()

Duplicate

Expected< TensorBuffer > Duplicate() const

Creates a duplicate of the current TensorBuffer object.

The returned object is reference-counted, so the underlying LiteRtTensorBuffer handle is not released until the last reference is removed.

GetAhwb

litert::Expected< AHardwareBuffer * > GetAhwb() const

GetDmaBuf

litert::Expected< DmaBuf > GetDmaBuf() const

GetEvent

Expected< Event > GetEvent() const

GetGlBuffer

Expected< GlBuffer > GetGlBuffer() const

GetGlTexture

Expected< GlTexture > GetGlTexture() const

GetMetalBuffer

Expected< void * > GetMetalBuffer() const

GetOpenClMemory

Expected< cl_mem > GetOpenClMemory() const

GetVulkanMemory

Expected< HwMemoryHandle > GetVulkanMemory() const

GetWebGpuBuffer

Expected< HwMemoryHandle > GetWebGpuBuffer() const

HasEvent

bool HasEvent() const

HasType

bool HasType(
  const RankedTensorType & type
) const

HasType

bool HasType(
  const LiteRtRankedTensorType & type
) const

IsMetalMemory

bool IsMetalMemory() const

Returns true if the tensor buffer is a Metal memory object.

IsOpenClMemory

bool IsOpenClMemory() const

Returns true if the tensor buffer is an OpenCL memory object.

IsVulkanMemory

bool IsVulkanMemory() const

Returns true if the tensor buffer is a Vulkan memory object.

IsWebGpuMemory

bool IsWebGpuMemory() const

Returns true if the tensor buffer is a WebGPU memory object.

Lock

Expected< void * > Lock(
  LockMode mode
)

Offset

Expected< size_t > Offset() const

PackedSize

Expected< size_t > PackedSize() const

Returns the size of the tensor buffer in packed bytes.

This size is used for reading/writing data on a locked tensor buffer.

Read

Expected< void > Read(
  absl::Span< T > data
)

Reads data from the tensor buffer into a user-provided absl::Span.

If the provided buffer is smaller than the tensor buffer, data will be read up to the size of the provided buffer. Returns an error if the provided buffer is larger than the tensor buffer.

SetEvent

Expected< void > SetEvent(
  Event && event
)

Sets the C++ Event object for the tensor buffer.

This function takes ownership of the provided Event object.

Size

Expected< size_t > Size() const

Returns the size of the underlying hardware tensor buffer.

This size can differ from PackedSize() if striding and padding exist.

TensorBuffer

 TensorBuffer()=default

TensorType

Expected< RankedTensorType > TensorType() const

Unlock

Expected< void > Unlock()

Write

Expected< void > Write(
  absl::Span< const T > data
)

Writes data from a user-provided absl::Span to the tensor buffer.

Returns an error if the provided buffer is larger than the tensor buffer's size.

Public static functions

CreateFromAhwb

Expected< TensorBuffer > CreateFromAhwb(
  const Environment & env,
  const RankedTensorType & tensor_type,
  AHardwareBuffer *ahwb,
  size_t ahwb_offset
)

Creates a TensorBuffer that wraps an Android Hardware Buffer.

The provided AHardwareBuffer is not owned by the TensorBuffer and must outlive it.

Details
Parameters
ahwb_offset
The offset in bytes from the start of the AHardwareBuffer where the tensor data begins.

CreateFromClBuffer

Expected< TensorBuffer > CreateFromClBuffer(
  const Environment & env,
  const RankedTensorType & tensor_type,
  TensorBufferType buffer_type,
  cl_mem cl_memory,
  size_t size_bytes
)

CreateFromGlBuffer

Expected< TensorBuffer > CreateFromGlBuffer(
  const Environment & env,
  const RankedTensorType & tensor_type,
  LiteRtGLenum target,
  LiteRtGLuint id,
  size_t size_bytes,
  size_t offset
)

CreateFromGlTexture

Expected< TensorBuffer > CreateFromGlTexture(
  const Environment & env,
  const RankedTensorType & tensor_type,
  LiteRtGLenum target,
  LiteRtGLuint id,
  LiteRtGLenum format,
  size_t size_bytes,
  LiteRtGLint layer
)

CreateFromHostMemory

Expected< TensorBuffer > CreateFromHostMemory(
  const Environment & env,
  const RankedTensorType & tensor_type,
  void *host_mem_addr,
  size_t buffer_size
)

Creates a TensorBuffer that wraps the provided host memory.

The provided host memory is not owned by the TensorBuffer object and must outlive it. Callers are responsible for ensuring that the pointer is aligned to at least LITERT_HOST_MEMORY_BUFFER_ALIGNMENT bytes and that any required padding for delegates like XNNPACK is included and initialized.

CreateFromHostMemory

Expected< TensorBuffer > CreateFromHostMemory(
  const RankedTensorType & tensor_type,
  void *host_mem_addr,
  size_t buffer_size
)

CreateManaged

Expected< TensorBuffer > CreateManaged(
  const Environment & env,
  TensorBufferType buffer_type,
  const RankedTensorType & tensor_type,
  size_t buffer_size
)

Creates a managed TensorBuffer of a given buffer type and size.

The returned object is owned by the caller. For host memory, this allocator guarantees LITERT_HOST_MEMORY_BUFFER_ALIGNMENT alignment and reserves any delegate-specific padding (e.g., XNNPACK extra bytes), so callers do not need to over-allocate manually.

CreateManagedHostMemory

Expected< TensorBuffer > CreateManagedHostMemory(
  const RankedTensorType & tensor_type,
  size_t buffer_size
)

Creates a managed host memory TensorBuffer using the default environment (if applicable).

The returned object is owned by the caller.

ToLiteRtLockMode

LiteRtTensorBufferLockMode ToLiteRtLockMode(
  LockMode mode
)

WrapCObject

TensorBuffer WrapCObject(
  LiteRtTensorBuffer tensor_buffer,
  OwnHandle owned
)