__rte_experimentalintrte_ml_dev_init(size_tdev_max)
Maximum number of devices if rte_ml_dev_init() is not called. Initialize the device array before probing
devices. If not called, the first device probed would initialize the array to a size of
RTE_MLDEV_DEFAULT_MAX.
Parametersdev_max Maximum number of devices.
Returns
0 on success, -rte_errno otherwise:
• ENOMEM if out of memory
• EINVAL if 0 size
• EBUSY if already initialized
__rte_experimentaluint16_trte_ml_dev_count(void)
Get the total number of ML devices that have been successfully initialised.
Returns
• The total number of usable ML devices.
__rte_experimentalintrte_ml_dev_is_valid_dev(int16_tdev_id)
Check if the device is in ready state.
Parametersdev_id The identifier of the device.
Returns
• 0 if device state is not in ready state.
• 1 if device state is ready state.
__rte_experimentalintrte_ml_dev_socket_id(int16_tdev_id)
Return the NUMA socket to which a device is connected.
Parametersdev_id The identifier of the device.
Returns
• The NUMA socket id to which the device is connected
• 0 If the socket could not be determined.
• -EINVAL: if the dev_id value is not valid.
__rte_experimentalintrte_ml_dev_info_get(int16_tdev_id,structrte_ml_dev_info*dev_info)
Retrieve the information of the device.
Parametersdev_id The identifier of the device.
dev_info A pointer to a structure of type rte_ml_dev_infotobefilledwiththeinfoofthedevice.Returns
• 0: Success, driver updates the information of the ML device
• < 0: Error code returned by the driver info get function.
__rte_experimentalintrte_ml_dev_configure(int16_tdev_id,conststructrte_ml_dev_config*config)
Configure an ML device.
This function must be invoked first before any other function in the API.
ML Device can be re-configured, when in a stopped state. Device cannot be re-configured after
rte_ml_dev_close() is called.
The caller may use rte_ml_dev_info_get() to get the capability of each resources available for this ML
device.
Parametersdev_id The identifier of the device to configure.
config The ML device configuration structure.
Returns
• 0: Success, device configured.
• < 0: Error code returned by the driver configuration function.
__rte_experimentaluint16_trte_ml_dev_queue_pair_count(int16_tdev_id)
Get the number of queue pairs on a specific ML device.
Parametersdev_id The identifier of the device.
Returns
• The number of configured queue pairs.
__rte_experimentalintrte_ml_dev_queue_pair_setup(int16_tdev_id,uint16_tqueue_pair_id,conststructrte_ml_dev_qp_conf*qp_conf,intsocket_id)
Set up a queue pair for a device. This should only be called when the device is stopped.
Parametersdev_id The identifier of the device.
queue_pair_id The index of the queue pairs to set up. The value must be in the range [0,
nb_queue_pairs - 1] previously supplied to rte_ml_dev_configure().
qp_conf The pointer to the configuration data to be used for the queue pair.
socket_id The socket_id argument is the socket identifier in case of NUMA. The value can be
SOCKET_ID_ANY if there is no NUMA constraint for the memory allocated for the queue pair.
Returns
• 0: Success, queue pair correctly set up.
• < 0: Queue pair configuration failed.
__rte_experimentalintrte_ml_dev_start(int16_tdev_id)
Start an ML device.
The device start step consists of setting the configured features and enabling the ML device to accept
inference jobs.
Parametersdev_id The identifier of the device.
Returns
• 0: Success, device started.
• <0: Error code of the driver device start function.
__rte_experimentalintrte_ml_dev_stop(int16_tdev_id)
Stop an ML device. A stopped device cannot accept inference jobs. The device can be restarted with a call
to rte_ml_dev_start().
Parametersdev_id The identifier of the device.
Returns
• 0: Success, device stopped.
• <0: Error code of the driver device stop function.
__rte_experimentalintrte_ml_dev_close(int16_tdev_id)
Close an ML device. The device cannot be restarted!
Parametersdev_id The identifier of the device.
Returns
• 0 on successfully closing device.
• <0 on failure to close device.
__rte_experimentaluint16_trte_ml_enqueue_burst(int16_tdev_id,uint16_tqp_id,structrte_ml_op**ops,uint16_tnb_ops)
Enqueue a burst of ML inferences for processing on an ML device.
The rte_ml_enqueue_burst() function is invoked to place ML inference operations on the queue qp_id of the
device designated by its dev_id.
The nb_ops parameter is the number of inferences to process which are supplied in the ops array of
rte_ml_opstructures.
The rte_ml_enqueue_burst() function returns the number of inferences it actually enqueued for processing.
A return value equal to nb_ops means that all packets have been enqueued.
Parametersdev_id The identifier of the device.
qp_id The index of the queue pair which inferences are to be enqueued for processing. The value must
be in the range [0, nb_queue_pairs - 1] previously supplied to rte_ml_dev_configure.
ops The address of an array of nb_ops pointers to rte_ml_opstructureswhichcontaintheMLinferencestobeprocessed.nb_opsThenumberofoperationstoprocess.Returns
The number of inference operations actually enqueued to the ML device. The return value can be less
than the value of the nb_ops parameter when the ML device queue is full or if invalid parameters are
specified in a rte_ml_op.__rte_experimentaluint16_trte_ml_dequeue_burst(int16_tdev_id,uint16_tqp_id,structrte_ml_op**ops,uint16_tnb_ops)
Dequeue a burst of processed ML inferences operations from a queue on the ML device. The dequeued
operations are stored in rte_ml_opstructureswhosepointersaresuppliedintheopsarray.
The rte_ml_dequeue_burst() function returns the number of inferences actually dequeued, which is the
number of rte_ml_opdatastructureseffectivelysuppliedintotheopsarray.
A return value equal to nb_ops indicates that the queue contained at least nb_ops* operations, and this
is likely to signify that other processed operations remain in the devices output queue. Application
implementing a 'retrieve as many processed operations as possible' policy can check this specific case
and keep invoking the rte_ml_dequeue_burst() function until a value less than nb_ops is returned.
The rte_ml_dequeue_burst() function does not provide any error notification to avoid the corresponding
overhead.
Parametersdev_id The identifier of the device.
qp_id The index of the queue pair from which to retrieve processed packets. The value must be in the
range [0, nb_queue_pairs - 1] previously supplied to rte_ml_dev_configure().
ops The address of an array of pointers to rte_ml_opstructuresthatmustbelargeenoughtostorenb_opspointersinit.nb_opsThemaximumnumberofinferencestodequeue.Returns
The number of operations actually dequeued, which is the number of pointers to rte_ml_opstructureseffectivelysuppliedtotheopsarray.__rte_experimentalintrte_ml_op_error_get(int16_tdev_id,structrte_ml_op*op,structrte_ml_op_error*error)
Get PMD specific error information for an ML op.
When an ML operation completed with RTE_ML_OP_STATUS_ERROR as status, This API allows to get PMD specific
error details.
Parametersdev_id Device identifier
op Handle of ML operation
error Address of structure rte_ml_op_error to be filled
Returns
• Returns 0 on success
• Returns negative value on failure
__rte_experimentalintrte_ml_dev_stats_get(int16_tdev_id,structrte_ml_dev_stats*stats)
Retrieve the general I/O statistics of a device.
Parametersdev_id The identifier of the device.
stats Pointer to structure to where statistics will be copied. On error, this location may or may not
have been modified.
Returns
• 0 on success
• -EINVAL: If invalid parameter pointer is provided.
__rte_experimentalvoidrte_ml_dev_stats_reset(int16_tdev_id)
Reset the statistics of a device.
Parametersdev_id The identifier of the device.
__rte_experimentalintrte_ml_dev_xstats_names_get(int16_tdev_id,enumrte_ml_dev_xstats_modemode,int32_tmodel_id,structrte_ml_dev_xstats_map*xstats_map,uint32_tsize)
Retrieve names of extended statistics of an ML device.
Parametersdev_id The identifier of the device.
mode Mode of statistics to retrieve. Choices include the device statistics and model statistics.
model_id Used to specify the model number in model mode, and is ignored in device mode.
xstats_map Block of memory to insert names and ids into. Must be at least size in capacity. If set to
NULL, function returns required capacity. The id values returned can be passed to
rte_ml_dev_xstats_get to select statistics.
size Capacity of xstats_names (number of xstats_map).
Returns
• Positive value lower or equal to size: success. The return value is the number of entries filled in
the stats table.
• Positive value higher than size: error, the given statistics table is too small. The return value
corresponds to the size that should be given to succeed. The entries in the table are not valid and
shall not be used by the caller.
• Negative value on error: -ENODEV for invalid dev_id. -EINVAL for invalid mode, model parameters.
-ENOTSUP if the device doesn't support this function.
__rte_experimentalintrte_ml_dev_xstats_by_name_get(int16_tdev_id,constchar*name,uint16_t*stat_id,uint64_t*value)
Retrieve the value of a single stat by requesting it by name.
Parametersdev_id The identifier of the device.
name Name of stat name to retrieve.
stat_id If non-NULL, the numerical id of the stat will be returned, so that further requests for the
stat can be got using rte_ml_dev_xstats_get, which will be faster as it doesn't need to scan a list
of names for the stat. If the stat cannot be found, the id returned will be (unsigned)-1.
value Value of the stat to be returned.
Returns
• Zero: No error.
• Negative value: -EINVAL if stat not found, -ENOTSUP if not supported.
__rte_experimentalintrte_ml_dev_xstats_get(int16_tdev_id,enumrte_ml_dev_xstats_modemode,int32_tmodel_id,constuint16_tstat_ids[],uint64_tvalues[],uint16_tnb_ids)
Retrieve extended statistics of an ML device.
Parametersdev_id The identifier of the device.
mode Mode of statistics to retrieve. Choices include the device statistics and model statistics.
model_id Used to specify the model id in model mode, and is ignored in device mode.
stat_ids ID numbers of the stats to get. The ids can be got from the stat position in the stat list
from rte_ml_dev_xstats_names_get(), or by using rte_ml_dev_xstats_by_name_get().
values Values for each stats request by ID.
nb_ids Number of stats requested.
Returns
• Positive value: number of stat entries filled into the values array
• Negative value on error: -ENODEV for invalid dev_id. -EINVAL for invalid mode, model id or stat id
parameters. -ENOTSUP if the device doesn't support this function.
__rte_experimentalintrte_ml_dev_xstats_reset(int16_tdev_id,enumrte_ml_dev_xstats_modemode,int32_tmodel_id,constuint16_tstat_ids[],uint16_tnb_ids)
Reset the values of the xstats of the selected component in the device.
Parametersdev_id The identifier of the device.
mode Mode of the statistics to reset. Choose from device or model.
model_id Model stats to reset. 0 and positive values select models, while -1 indicates all models.
stat_ids Selects specific statistics to be reset. When NULL, all statistics selected by mode will be
reset. If non-NULL, must point to array of at least nb_ids size.
nb_ids The number of ids available from the ids array. Ignored when ids is NULL.
Returns
• Zero: successfully reset the statistics.
• Negative value: -EINVAL invalid parameters, -ENOTSUP if not supported.
__rte_experimentalintrte_ml_dev_dump(int16_tdev_id,FILE*fd)
Dump internal information about dev_id to the FILE* provided in fd.
Parametersdev_id The identifier of the device.
fd A pointer to a file for output.
Returns
• 0: on success.
• <0: on failure.
__rte_experimentalintrte_ml_dev_selftest(int16_tdev_id)
Trigger the ML device self test.
Parametersdev_id The identifier of the device.
Returns
• 0: Selftest successful.
• -ENOTSUP: if the device doesn't support selftest.
• other values < 0 on failure.
__rte_experimentalintrte_ml_model_load(int16_tdev_id,structrte_ml_model_params*params,uint16_t*model_id)
Load an ML model to the device.
Load an ML model to the device with parameters requested in the structure rte_ml_model_params.
Parametersdev_id The identifier of the device.
params Parameters for the model to be loaded.
model_id Identifier of the model loaded.
Returns
• 0: Success, Model loaded.
• < 0: Failure, Error code of the model load driver function.
__rte_experimentalintrte_ml_model_unload(int16_tdev_id,uint16_tmodel_id)
Unload an ML model from the device.
Parametersdev_id The identifier of the device.
model_id Identifier of the model to be unloaded.
Returns
• 0: Success, Model unloaded.
• < 0: Failure, Error code of the model unload driver function.
__rte_experimentalintrte_ml_model_start(int16_tdev_id,uint16_tmodel_id)
Start an ML model for the given device ID.
Start an ML model to accept inference requests.
Parametersdev_id The identifier of the device.
model_id Identifier of the model to be started.
Returns
• 0: Success, Model loaded.
• < 0: Failure, Error code of the model start driver function.
__rte_experimentalintrte_ml_model_stop(int16_tdev_id,uint16_tmodel_id)
Stop an ML model for the given device ID.
Model stop would disable the ML model to be used for inference jobs. All inference jobs must have been
completed before model stop is attempted.
Parametersdev_id The identifier of the device.
model_id Identifier of the model to be stopped.
Returns
• 0: Success, Model unloaded.
• < 0: Failure, Error code of the model stop driver function.
__rte_experimentalintrte_ml_model_info_get(int16_tdev_id,uint16_tmodel_id,structrte_ml_model_info*model_info)
Get ML model information.
Parametersdev_id The identifier of the device.
model_id Identifier for the model created
model_info Pointer to a model info structure
Returns
• Returns 0 on success
• Returns negative value on failure
__rte_experimentalintrte_ml_model_params_update(int16_tdev_id,uint16_tmodel_id,void*buffer)
Update the model parameters without unloading model.
Update model parameters such as weights and bias without unloading the model. rte_ml_model_stop() must be
called before invoking this API.
Parametersdev_id The identifier of the device.
model_id Identifier for the model created
buffer Pointer to the model weights and bias buffer. Size of the buffer is equal to wb_size returned
in rte_ml_model_info.Returns
• Returns 0 on success
• Returns negative value on failure
__rte_experimentalintrte_ml_io_float32_to_int8(constvoid*fp32,void*i8,uint64_tnb_elements,floatscale,int8_tzero_point)
Convert a buffer containing numbers in single precision floating format (float32) to signed 8-bit integer
format (INT8).
Parametersfp32 Input buffer containing float32 numbers. Size of buffer is equal to (nb_elements * 4) bytes.
i8 Output buffer to store INT8 numbers. Size of buffer is equal to (nb_elements * 1) bytes.
nb_elements Number of elements in the buffer.
scale Scale factor for conversion.
zero_point Zero point for conversion.
Returns
• 0, Success.
• < 0, Error code on failure.
__rte_experimentalintrte_ml_io_int8_to_float32(constvoid*i8,void*fp32,uint64_tnb_elements,floatscale,int8_tzero_point)
Convert a buffer containing numbers in signed 8-bit integer format (INT8) to single precision floating
format (float32).
Parametersi8 Input buffer containing INT8 numbers. Size of buffer is equal to (nb_elements * 1) bytes.
fp32 Output buffer to store float32 numbers. Size of buffer is equal to (nb_elements * 4) bytes.
nb_elements Number of elements in the buffer.
scale Scale factor for conversion.
zero_point Zero point for conversion.
Returns
• 0, Success.
• < 0, Error code on failure.
__rte_experimentalintrte_ml_io_float32_to_uint8(constvoid*fp32,void*ui8,uint64_tnb_elements,floatscale,uint8_tzero_point)
Convert a buffer containing numbers in single precision floating format (float32) to unsigned 8-bit
integer format (UINT8).
Parametersfp32 Input buffer containing float32 numbers. Size of buffer is equal to (nb_elements * 4) bytes.
ui8 Output buffer to store UINT8 numbers. Size of buffer is equal to (nb_elements * 1) bytes.
nb_elements Number of elements in the buffer.
scale Scale factor for conversion.
zero_point Zero point for conversion.
Returns
• 0, Success.
• < 0, Error code on failure.
__rte_experimentalintrte_ml_io_uint8_to_float32(constvoid*ui8,void*fp32,uint64_tnb_elements,floatscale,uint8_tzero_point)
Convert a buffer containing numbers in unsigned 8-bit integer format (UINT8) to single precision floating
format (float32).
Parametersui8 Input buffer containing UINT8 numbers. Size of buffer is equal to (nb_elements * 1) bytes.
fp32 Output buffer to store float32 numbers. Size of buffer is equal to (nb_elements * 4) bytes.
nb_elements Number of elements in the buffer.
scale Scale factor for conversion.
zero_point Zero point for conversion.
Returns
• 0, Success.
• < 0, Error code on failure.
__rte_experimentalintrte_ml_io_float32_to_int16(constvoid*fp32,void*i16,uint64_tnb_elements,floatscale,int16_tzero_point)
Convert a buffer containing numbers in single precision floating format (float32) to signed 16-bit
integer format (INT16).
Parametersfp32 Input buffer containing float32 numbers. Size of buffer is equal to (nb_elements * 4) bytes.
i16 Output buffer to store INT16 numbers. Size of buffer is equal to (nb_elements * 2) bytes.
nb_elements Number of elements in the buffer.
scale Scale factor for conversion.
zero_point Zero point for conversion.
Returns
• 0, Success.
• < 0, Error code on failure.
__rte_experimentalintrte_ml_io_int16_to_float32(constvoid*i16,void*fp32,uint64_tnb_elements,floatscale,int16_tzero_point)
Convert a buffer containing numbers in signed 16-bit integer format (INT16) to single precision floating
format (float32).
Parametersi16 Input buffer containing INT16 numbers. Size of buffer is equal to (nb_elements * 2) bytes.
fp32 Output buffer to store float32 numbers. Size of buffer is equal to (nb_elements * 4) bytes.
nb_elements Number of elements in the buffer.
scale Scale factor for conversion.
zero_point Zero point for conversion.
Returns
• 0, Success.
• < 0, Error code on failure.
__rte_experimentalintrte_ml_io_float32_to_uint16(constvoid*fp32,void*ui16,uint64_tnb_elements,floatscale,uint16_tzero_point)
Convert a buffer containing numbers in single precision floating format (float32) to unsigned 16-bit
integer format (UINT16).
Parametersfp32 Input buffer containing float32 numbers. Size of buffer is equal to (nb_elements * 4) bytes.
ui16 Output buffer to store UINT16 numbers. Size of buffer is equal to (nb_elements * 2) bytes.
nb_elements Number of elements in the buffer.
scale Scale factor for conversion.
zero_point Zero point for conversion.
Returns
• 0, Success.
• < 0, Error code on failure.
__rte_experimentalintrte_ml_io_uint16_to_float32(constvoid*ui16,void*fp32,uint64_tnb_elements,floatscale,uint16_tzero_point)
Convert a buffer containing numbers in unsigned 16-bit integer format (UINT16) to single precision
floating format (float32).
Parametersui16 Input buffer containing UINT16 numbers. Size of buffer is equal to (nb_elements * 2) bytes.
fp32 Output buffer to store float32 numbers. Size of buffer is equal to (nb_elements * 4) bytes.
nb_elements Number of elements in the buffer.
scale Scale factor for conversion.
zero_point Zero point for conversion.
Returns
• 0, Success.
• < 0, Error code on failure.
__rte_experimentalintrte_ml_io_float32_to_int32(constvoid*fp32,void*i32,uint64_tnb_elements,floatscale,int32_tzero_point)
Convert a buffer containing numbers in single precision floating format (float32) to signed 32-bit
integer format (INT32).
Parametersfp32 Input buffer containing float32 numbers. Size of buffer is equal to (nb_elements * 4) bytes.
i32 Output buffer to store INT32 numbers. Size of buffer is equal to (nb_elements * 4) bytes.
nb_elements Number of elements in the buffer.
scale Scale factor for conversion.
zero_point Zero point for conversion.
Returns
• 0, Success.
• < 0, Error code on failure.
__rte_experimentalintrte_ml_io_int32_to_float32(constvoid*i32,void*fp32,uint64_tnb_elements,floatscale,int32_tzero_point)
Convert a buffer containing numbers in signed 32-bit integer format (INT32) to single precision floating
format (float32).
Parametersi32 Input buffer containing INT32 numbers. Size of buffer is equal to (nb_elements * 4) bytes.
fp32 Output buffer to store float32 numbers. Size of buffer is equal to (nb_elements * 4) bytes.
nb_elements Number of elements in the buffer.
scale Scale factor for conversion.
zero_point Zero point for conversion.
Returns
• 0, Success.
• < 0, Error code on failure.
__rte_experimentalintrte_ml_io_float32_to_uint32(constvoid*fp32,void*ui32,uint64_tnb_elements,floatscale,uint32_tzero_point)
Convert a buffer containing numbers in single precision floating format (float32) to unsigned 32-bit
integer format (UINT32).
Parametersfp32 Input buffer containing float32 numbers. Size of buffer is equal to (nb_elements * 4) bytes.
ui32 Output buffer to store UINT32 numbers. Size of buffer is equal to (nb_elements * 4) bytes.
nb_elements Number of elements in the buffer.
scale Scale factor for conversion.
zero_point Zero point for conversion.
Returns
• 0, Success.
• < 0, Error code on failure.
__rte_experimentalintrte_ml_io_uint32_to_float32(constvoid*ui32,void*fp32,uint64_tnb_elements,floatscale,uint32_tzero_point)
Convert a buffer containing numbers in unsigned 32-bit integer format (UINT32) to single precision
floating format (float32).
Parametersui32 Input buffer containing UINT32 numbers. Size of buffer is equal to (nb_elements * 4) bytes.
fp32 Output buffer to store float32 numbers. Size of buffer is equal to (nb_elements * 4) bytes.
nb_elements Number of elements in the buffer.
scale Scale factor for conversion.
zero_point Zero point for conversion.
Returns
• 0, Success.
• < 0, Error code on failure.
__rte_experimentalintrte_ml_io_float32_to_int64(constvoid*fp32,void*i64,uint64_tnb_elements,floatscale,int64_tzero_point)
Convert a buffer containing numbers in single precision floating format (float32) to signed 64-bit
integer format (INT64).
Parametersfp32 Input buffer containing float32 numbers. Size of buffer is equal to (nb_elements * 4) bytes.
i64 Output buffer to store INT64 numbers. Size of buffer is equal to (nb_elements * 4) bytes.
nb_elements Number of elements in the buffer.
scale Scale factor for conversion.
zero_point Zero point for conversion.
Returns
• 0, Success.
• < 0, Error code on failure.
__rte_experimentalintrte_ml_io_int64_to_float32(constvoid*i64,void*fp32,uint64_tnb_elements,floatscale,int64_tzero_point)
Convert a buffer containing numbers in signed 64-bit integer format (INT64) to single precision floating
format (float32).
Parametersi64 Input buffer containing INT64 numbers. Size of buffer is equal to (nb_elements * 4) bytes.
fp32 Output buffer to store float32 numbers. Size of buffer is equal to (nb_elements * 4) bytes.
nb_elements Number of elements in the buffer.
scale Scale factor for conversion.
zero_point Zero point for conversion.
Returns
• 0, Success.
• < 0, Error code on failure.
__rte_experimentalintrte_ml_io_float32_to_uint64(constvoid*fp32,void*ui64,uint64_tnb_elements,floatscale,uint64_tzero_point)
Convert a buffer containing numbers in single precision floating format (float32) to unsigned 64-bit
integer format (UINT64).
Parametersfp32 Input buffer containing float32 numbers. Size of buffer is equal to (nb_elements * 4) bytes.
ui64 Output buffer to store UINT64 numbers. Size of buffer is equal to (nb_elements * 4) bytes.
nb_elements Number of elements in the buffer.
scale Scale factor for conversion.
zero_point Zero point for conversion.
Returns
• 0, Success.
• < 0, Error code on failure.
__rte_experimentalintrte_ml_io_uint64_to_float32(constvoid*ui64,void*fp32,uint64_tnb_elements,floatscale,uint64_tzero_point)
Convert a buffer containing numbers in unsigned 64-bit integer format (UINT64) to single precision
floating format (float32).
Parametersui64 Input buffer containing UINT64 numbers. Size of buffer is equal to (nb_elements * 4) bytes.
fp32 Output buffer to store float32 numbers. Size of buffer is equal to (nb_elements * 4) bytes.
nb_elements Number of elements in the buffer.
scale Scale factor for conversion.
zero_point Zero point for conversion.
Returns
• 0, Success.
• < 0, Error code on failure.
__rte_experimentalintrte_ml_io_float32_to_float16(constvoid*fp32,void*fp16,uint64_tnb_elements)
Convert a buffer containing numbers in single precision floating format (float32) to half precision
floating point format (FP16).
Parametersfp32 Input buffer containing float32 numbers. Size of buffer is equal to (nb_elements *4) bytes.
fp16 Output buffer to store float16 numbers. Size of buffer is equal to (nb_elements * 2) bytes.
nb_elements Number of elements in the buffer.
Returns
• 0, Success.
• < 0, Error code on failure.
__rte_experimentalintrte_ml_io_float16_to_float32(constvoid*fp16,void*fp32,uint64_tnb_elements)
Convert a buffer containing numbers in half precision floating format (FP16) to single precision floating
point format (float32).
Parametersfp16 Input buffer containing float16 numbers. Size of buffer is equal to (nb_elements * 2) bytes.
fp32 Output buffer to store float32 numbers. Size of buffer is equal to (nb_elements * 4) bytes.
nb_elements Number of elements in the buffer.
Returns
• 0, Success.
• < 0, Error code on failure.
__rte_experimentalintrte_ml_io_float32_to_bfloat16(constvoid*fp32,void*bf16,uint64_tnb_elements)
Convert a buffer containing numbers in single precision floating format (float32) to brain floating point
format (bfloat16).
Parametersfp32 Input buffer containing float32 numbers. Size of buffer is equal to (nb_elements *4) bytes.
bf16 Output buffer to store bfloat16 numbers. Size of buffer is equal to (nb_elements * 2) bytes.
nb_elements Number of elements in the buffer.
Returns
• 0, Success.
• < 0, Error code on failure.
__rte_experimentalintrte_ml_io_bfloat16_to_float32(constvoid*bf16,void*fp32,uint64_tnb_elements)
Convert a buffer containing numbers in brain floating point format (bfloat16) to single precision
floating point format (float32).
Parametersbf16 Input buffer containing bfloat16 numbers. Size of buffer is equal to (nb_elements * 2) bytes.
fp32 Output buffer to store float32 numbers. Size of buffer is equal to (nb_elements * 4) bytes.
nb_elements Number of elements in the buffer.
Returns
• 0, Success.
• < 0, Error code on failure.
__rte_experimentalintrte_ml_io_quantize(int16_tdev_id,uint16_tmodel_id,structrte_ml_buff_seg**dbuffer,structrte_ml_buff_seg**qbuffer)
Quantize input data.
Quantization converts data from a higher precision types to a lower precision types to improve the
throughput and efficiency of the model execution with minimal loss of accuracy. Types of dequantized data
and quantized data are specified by the model.
Parametersdev_id The identifier of the device.
model_id Identifier for the model
dbuffer Address of dequantized input data
qbuffer Address of quantized input data
Returns
• Returns 0 on success
• Returns negative value on failure
__rte_experimentalintrte_ml_io_dequantize(int16_tdev_id,uint16_tmodel_id,structrte_ml_buff_seg**qbuffer,structrte_ml_buff_seg**dbuffer)
Dequantize output data.
Dequantization converts data from a lower precision type to a higher precision type. Types of quantized
data and dequantized are specified by the model.
Parametersdev_id The identifier of the device.
model_id Identifier for the model
qbuffer Address of quantized output data
dbuffer Address of dequantized output data
Returns
• Returns 0 on success
• Returns negative value on failure
__rte_experimentalstructrte_mempool*rte_ml_op_pool_create(constchar*name,unsignedintnb_elts,unsignedintcache_size,uint16_tuser_size,intsocket_id)
Create an ML operation pool
Parametersname ML operations pool name
nb_elts Number of elements in pool
cache_size Number of elements to cache on lcore, see rte_mempool_create for further details about
cache size
user_size Size of private data to allocate for user with each operation
socket_id Socket to identifier allocate memory on
Returns
• On success pointer to mempool
• On failure NULL
__rte_experimentalvoidrte_ml_op_pool_free(structrte_mempool*mempool)
Free an ML operation pool
Parametersmempool A pointer to the mempool structure. If NULL then, the function does nothing.
Install Now
PNG Icons
