litert:: Options

    #include <litert_options.h>

    Manages the configuration options for compiling a LiteRT model.

    Summary

    This class provides methods to set hardware accelerators, add custom operations, bind external tensors, and configure various backend-specific options (e.g., GPU, CPU, Qualcomm, MediaTek, etc.).

    Inheritance

    Inherits from: internal::BaseHandle< LiteRtOptions >

    Constructors and Destructors

    Options ()
    Options (LiteRtOptions compilation_options, OwnHandle owned)
    Constructs an Options object from a C handle.

    Public types

    ScopedWeightSectionMap
    using
    absl::flat_hash_map< std::string, ScopedWeightSection >
    A map from a group name to a weight section.

    Friend classes

    CompiledModel
    friend class
    CompiledModelNext
    friend class
    tools::ApplyPlugin
    friend LiteRtStatus

    Public functions

    AddCustomOpKernel (const std::string & custom_op_name, int custom_op_version, const LiteRtCustomOpKernel & custom_op_kernel, void *custom_op_kernel_user_data)
    Expected < void >
    Adds a custom operator kernel.
    AddCustomOpKernel ( CustomOpKernel & custom_op_kernel)
    Expected < void >
    Adds a custom operator kernel.
    AddExternalTensorBinding (const std::string & signature_name, const std::string & tensor_name, void *data, size_t size_bytes)
    Expected < void >
    Binds an external memory buffer to a specific tensor in the model.
    AddOpaqueOptions ( OpaqueOptions && options)
    Expected < void >
    Deprecated. Use the GetXXXOptions() methods instead.
    GetCompilerOptions ()
    Expected < CompilerOptions & >
    Returns a reference to the compiler options.
    GetCpuOptions ()
    Expected < CpuOptions & >
    Returns a reference to the CPU options.
    GetGoogleTensorOptions ()
    Expected < google_tensor::GoogleTensorOptions & >
    Returns a reference to the Google Tensor options.
    GetGpuOptions ()
    Expected < GpuOptions & >
    Returns a reference to the GPU options.
    GetHardwareAccelerators ()
    Expected < LiteRtHwAcceleratorSet >
    Retrieves the currently set hardware accelerators.
    GetIntelOpenVinoOptions ()
    Expected < intel_openvino::IntelOpenVinoOptions & >
    Returns a reference to the Intel OpenVINO options.
    GetMediatekOptions ()
    Expected < mediatek::MediatekOptions & >
    Returns a reference to the MediaTek options.
    GetOpaqueOptions ()
    Retrieves the opaque options.
    GetQualcommOptions ()
    Expected < qualcomm::QualcommOptions & >
    Returns a reference to the Qualcomm options.
    GetRuntimeOptions ()
    Expected < RuntimeOptions & >
    Returns a reference to the runtime options.
    GetSamsungOptions ()
    Expected < samsung::SamsungOptions & >
    Returns a reference to the Samsung options.
    SetExternalWeightScopedFile (ScopedFile & scoped_file, ScopedWeightSectionMap sections)
    Expected < void >
    Registers a ScopedFile that contains all external buffer groups.
    SetHardwareAccelerators (HwAccelerators accelerators)
    Expected < void >
    Sets the hardware accelerators to be used for the model.
    SetHardwareAccelerators (HwAcceleratorSet accelerators)
    Expected < void >
    Sets the hardware accelerators to be used for the model.

    Public static functions

    Create ()
    Creates a new Options object.

    Public types

    ScopedWeightSectionMap

    absl::flat_hash_map< std::string, ScopedWeightSection > ScopedWeightSectionMap

    A map from a group name to a weight section.

    A weight section contains the offset and length of a contiguous region inside a ScopedFile that backs a single external buffer group. This map provides the mapping between the group name and its section.

    Friend classes

    CompiledModel

    friend class CompiledModel

    CompiledModelNext

    friend class CompiledModelNext

    tools::ApplyPlugin

     friend 
  
 LiteRtStatus 
  
 tools 
 :: 
 ApplyPlugin 
 ( 
 std 
 :: 
 unique_ptr 
<  
 tools 
 :: 
 ApplyPluginRun 
 > 
 run 
 )

    Public functions

    AddCustomOpKernel

      Expected 
 
<  
 void 
 > 
 AddCustomOpKernel 
 ( 
  
 const 
  
 std 
 :: 
 string 
 & 
 custom_op_name 
 , 
  
 int 
  
 custom_op_version 
 , 
  
 const 
  
 LiteRtCustomOpKernel 
 & 
 custom_op_kernel 
 , 
  
 void 
  
 * 
 custom_op_kernel_user_data 
 )

    Adds a custom operator kernel.

    Details
    Parameters
    custom_op_name
    The name of the custom operator.
    custom_op_version
    The version of the custom operator.
    custom_op_kernel
    The custom operator kernel implementation.
    custom_op_kernel_user_data
    User data to be passed to the kernel.
    Returns
    An Expected object that is empty on success, or contains an error.

    AddCustomOpKernel

     Expected 
< void > AddCustomOpKernel( CustomOpKernel 
& custom_op_kernel
)

    Adds a custom operator kernel.

    Details
    Parameters
    custom_op_kernel
    The custom operator kernel to add.
    Returns
    An Expected object that is empty on success, or contains an error.

    AddExternalTensorBinding

      Expected 
 
<  
 void 
 > 
 AddExternalTensorBinding 
 ( 
  
 const 
  
 std 
 :: 
 string 
 & 
 signature_name 
 , 
  
 const 
  
 std 
 :: 
 string 
 & 
 tensor_name 
 , 
  
 void 
  
 * 
 data 
 , 
  
 size_t 
  
 size_bytes 
 )

    Binds an external memory buffer to a specific tensor in the model.

    This function sets the tensor's allocation type to kTfLiteCustom , making it appear as a constant tensor with a pre-allocated buffer.

    Details
    Parameters
    signature_name
    The name of the signature containing the tensor.
    tensor_name
    The name of the tensor to bind.
    data
    A pointer to the external memory buffer.
    size_bytes
    The size of the external memory buffer in bytes.
    Returns
    An Expected object that is empty on success, or contains an error.

    AddOpaqueOptions

     Expected 
< void > AddOpaqueOptions( OpaqueOptions 
&& options
)

    Deprecated. Use the GetXXXOptions() methods instead.

    GetCompilerOptions

     Expected 
< CompilerOptions & > GetCompilerOptions()

    Returns a reference to the compiler options.

    GetCpuOptions

     Expected 
< CpuOptions & > GetCpuOptions()

    Returns a reference to the CPU options.

    Use this to configure CPU-specific settings.

    GetGoogleTensorOptions

     Expected 
< google_tensor::GoogleTensorOptions & > GetGoogleTensorOptions()

    Returns a reference to the Google Tensor options.

    Use this to configure Google Tensor-specific settings.

    GetGpuOptions

     Expected 
< GpuOptions & > GetGpuOptions()

    Returns a reference to the GPU options.

    Use this to configure GPU-specific settings.

    GetHardwareAccelerators

     Expected 
< LiteRtHwAcceleratorSet > GetHardwareAccelerators()

    Retrieves the currently set hardware accelerators.

    Details
    Returns
    An Expected object containing the set of hardware accelerators, or an error.

    GetIntelOpenVinoOptions

     Expected 
< intel_openvino::IntelOpenVinoOptions & > GetIntelOpenVinoOptions()

    Returns a reference to the Intel OpenVINO options.

    Use this to configure Intel OpenVINO-specific settings.

    GetMediatekOptions

     Expected 
< mediatek::MediatekOptions & > GetMediatekOptions()

    Returns a reference to the MediaTek options.

    Use this to configure MediaTek-specific settings.

    GetOpaqueOptions

     Expected 
< OpaqueOptions 
> GetOpaqueOptions()

    Retrieves the opaque options.

    Details
    Returns
    An Expected object containing the opaque options, or an error.

    GetQualcommOptions

     Expected 
< qualcomm::QualcommOptions & > GetQualcommOptions()

    Returns a reference to the Qualcomm options.

    Use this to configure Qualcomm-specific settings.

    GetRuntimeOptions

     Expected 
< RuntimeOptions & > GetRuntimeOptions()

    Returns a reference to the runtime options.

    GetSamsungOptions

     Expected 
< samsung::SamsungOptions & > GetSamsungOptions()

    Returns a reference to the Samsung options.

    Use this to configure Samsung-specific settings.

    Options

    Options()=default

    Options

    Options(
  LiteRtOptions compilation_options,
  OwnHandle owned
)

    Constructs an Options object from a C handle.

    Details
    Parameters
    compilation_options
    The C handle to the LiteRT options.
    owned
    Indicates whether this object should take ownership of the provided handle.

    SetExternalWeightScopedFile

     Expected 
< void > SetExternalWeightScopedFile(
  ScopedFile & scoped_file, ScopedWeightSectionMap 
sections
)

    Registers a ScopedFile that contains all external buffer groups.

    Details
    Parameters
    scoped_file
    The file containing the external weights.
    sections
    A map from group names to their respective sections in the file.
    Returns
    An Expected object that is empty on success, or contains an error.

    SetHardwareAccelerators

     Expected 
< void > SetHardwareAccelerators(
  HwAccelerators accelerators
)

    Sets the hardware accelerators to be used for the model.

    Details
    Parameters
    accelerators
    A bitmask of hardware accelerators.
    Returns
    An Expected object that is empty on success, or contains an error.

    SetHardwareAccelerators

     Expected 
< void > SetHardwareAccelerators(
  HwAcceleratorSet accelerators
)

    Sets the hardware accelerators to be used for the model.

    Details
    Parameters
    accelerators
    A set of hardware accelerators.
    Returns
    An Expected object that is empty on success, or contains an error.

    Public static functions

    Create

     Expected 
< Options 
> Create()

    Creates a new Options object.

    Details
    Returns
    An Expected object containing the new Options instance, or an error if creation fails.
    Create a Mobile Website
    View Site in Mobile | Classic
    Share by: