CUfileError_tcuFileBatchIOCancel(CUfileBatchHandle_tbatch_idp)voidcuFileBatchIODestroy(CUfileBatchHandle_tbatch_idp)CUfileError_tcuFileBatchIOGetStatus(CUfileBatchHandle_tbatch_idp,unsignedmin_nr,unsigned*nr,CUfileIOEvents_t*iocbp,structtimespec*timeout)CUfileError_tcuFileBatchIOSetUp(CUfileBatchHandle_t*batch_idp,unsignednr)CUfileError_tcuFileBatchIOSubmit(CUfileBatchHandle_tbatch_idp,unsignednr,CUfileIOParams_t*iocbp,unsignedintflags)CUfileError_tcuFileBufDeregister(constvoid*bufPtr_base)
deregister an already registered device or host memory from cuFile
Parameters:bufPtr_base buffer pointer to deregister
Returns:
CU_FILE_SUCCESS on success
CU_FILE_INVALID_VALUE on invalid memory pointer or unregistered memory pointer
Seealso:cuFileBufRegistercuFileReadcuFileWriteCUfileError_tcuFileBufRegister(constvoid*bufPtr_base,size_tlength,intflags)
register an existing cudaMalloced memory with cuFile to pin for GPUDirect Storage access or register host
allocated memory with cuFile.
Parameters:bufPtr_base buffer pointer allocated
length size of memory region from the above specified bufPtr
flags CU_FILE_RDMA_REGISTER
Returns:
CU_FILE_SUCCESS on success
CU_FILE_NVFS_DRIVER_ERROR
CU_FILE_INVALID_VALUE
CU_FILE_CUDA_ERROR for unsupported memory type
CU_FILE_MEMORY_ALREADY_REGISTERED on error
CU_FILE_GPU_MEMORY_PINNING_FAILED if not enough pinned memory is available
Note:
This memory will be use to perform GPU direct DMA from the supported storage.
Warning:
This API is intended for usecases where the memory is used as streaming buffer that is reused across
multiple cuFile IO operations before calling cuFileBufDeregisterSeealso:cuFileBufDeregistercuFileReadcuFileWriteCUfileError_tcuFileDriverClose(void)CUfileError_tcuFileDriverClose_v2(void)
reset the cuFile library and release the nvidia-fs driver
Returns:
CU_FILE_SUCCESS on success
CU_FILE_DRIVER_CLOSING if there are any active IO operations using cuFileRead or cuFileWriteSeealso:cuFileDriverOpenCUfileError_tcuFileDriverGetProperties(CUfileDrvProps_t*props)
Gets the Driver session properties.
Returns:
CU_FILE_SUCCESS on success
Seealso:cuFileDriverSetPollModecuFileDriverSetMaxDirectIOSizecuFileDriverSetMaxCacheSizecuFileDriverSetMaxPinnedMemSizeCUfileError_tcuFileDriverOpen(void)
Initialize the cuFile library and open the nvidia-fs driver.
Returns:
CU_FILE_SUCCESS on success
CU_FILE_DRIVER_NOT_INITIALIZED
CU_FILE_DRIVER_VERSION_MISMATCH on driver version mismatch error
Seealso:cuFileDriverCloseCUfileError_tcuFileDriverSetMaxCacheSize(size_tmax_cache_size)
Control parameter to set maximum GPU memory reserved per device by the library for internal buffering.
Parameters:max_cache_size The maximum GPU buffer space per device used for internal use in KB
Returns:
CU_FILE_SUCCESS on success
CU_FILE_DRIVER_NOT_INITIALIZED if the driver is not initialized
CU_FILE_DRIVER_VERSION_MISMATCH, CU_FILE_DRIVER_UNSUPPORTED_LIMIT on error
Warning:
This is an advanced command and should be tuned based on supported GPU memory
Seealso:cuFileDriverGetPropertiesCUfileError_tcuFileDriverSetMaxDirectIOSize(size_tmax_direct_io_size)
Control parameter to set max IO size(KB) used by the library to talk to nvidia-fs driver.
Parameters:max_direct_io_size maximum allowed direct io size in KB
Returns:
CU_FILE_SUCCESS on success
CU_FILE_DRIVER_NOT_INITIALIZED if the driver is not initialized
CU_FILE_DRIVER_VERSION_MISMATCH, CU_FILE_DRIVER_UNSUPPORTED_LIMIT on error
Warning:
This is an advanced command and should be tuned based on available system memory
Seealso:cuFileDriverGetPropertiesCUfileError_tcuFileDriverSetMaxPinnedMemSize(size_tmax_pinned_size)
Sets maximum buffer space that is pinned in KB for use by cuFileBufRegister.
Parameters:max_pinned_size maximum buffer space that is pinned in KB
Returns:
CU_FILE_SUCCESS on success
CU_FILE_DRIVER_NOT_INITIALIZED if the driver is not initialized
CU_FILE_DRIVER_VERSION_MISMATCH, CU_FILE_DRIVER_UNSUPPORTED_LIMIT on error
Warning:
This is an advanced command and should be tuned based on supported GPU memory
Seealso:cuFileDriverGetPropertiesCUfileError_tcuFileDriverSetPollMode(boolpoll,size_tpoll_threshold_size)
Sets whether the Read/Write APIs use polling to do IO operations.
Parameters:poll boolean to indicate whether to use poll mode or not
poll_threshold_size max IO size to use for POLLING mode in KB
Returns:
CU_FILE_SUCCESS on success
CU_FILE_DRIVER_NOT_INITIALIZED if the driver is not initialized
CU_FILE_DRIVER_VERSION_MISMATCH, CU_FILE_DRIVER_UNSUPPORTED_LIMIT on error
Warning:
This is an advanced command and should be tuned based on available system memory
Seealso:cuFileDriverGetPropertiesCUfileError_tcuFileGetVersion(int*version)Returns:
cufile library version.
version is returned as (1000 major + 10 minor). example, CUFILE 1.7.0 would be represented by 1070.
Note:
This is useful for applications that need to inquire the library.
Returns:
CU_FILE_SUCCESS on success
CU_FILE_INVALID_VALUE if the input parameter is null.
CU_FILE_DRIVER_VERSION_READ_ERROR if the version is not available.
voidcuFileHandleDeregister(CUfileHandle_tfh)
releases a registered filehandle from cuFile
Parameters:fhCUfileHandle_t file handle
Returns:
void
Seealso:cuFileHandleRegisterCUfileError_tcuFileHandleRegister(CUfileHandle_t*fh,CUfileDescr_t*descr)
cuFileHandleRegister is required, and performs extra checking that is memoized to provide increased
performance on later cuFile operations.
Parameters:fhCUfileHandle_t opaque file handle for IO operations
descrCUfileDescr_t file descriptor (OS agnostic)
Returns:
CU_FILE_SUCCESS on successful completion. fh will be updated for use in cuFileRead, cuFileWrite,
cuFileHandleDeregister
CU_FILE_DRIVER_NOT_INITIALIZED on failure to load driver
CU_FILE_IO_NOT_SUPPORTED - if filesystem is not supported
CU_FILE_INVALID_VALUE if null or bad api arguments
CU_FILE_INVALID_FILE_OPEN_FLAG if file is opened with unsupported modes like no O_DIRECT
CU_FILE_INVALID_FILE_TYPE if filepath is not valid or is not a regular file
CU_FILE_HANDLE_ALREADY_REGISTERED if file handle/descriptor is already registered
Description cuFileHandleRegister registers the open file descriptor for use with cuFile IO operations.
This API will ensure that the file’s descriptor is checked for GPUDirect Storage support and returns a
valid file handle on CU_FILE_SUCCESS.
Note:
the file needs to be opened in O_DIRECT mode to support GPUDirect Storage.
Seealso:cuFileReadcuFileWritecuFileHandleDeregisterssize_tcuFileRead(CUfileHandle_tfh,void*bufPtr_base,size_tsize,off_tfile_offset,off_tbufPtr_offset)
read data from a registered file handle to a specified device or host memory
Parameters:fhCUfileHandle_t opaque file handle
bufPtr_base base address of buffer in device or host memory
size size bytes to read
file_offset file-offset from beginning of the file
bufPtr_offset offset relative to the bufPtr_base pointer to read into.
Returns:
size of bytes successfully read
-1 on error, in which case errno is set to indicate filesystem errors.
all other errors will return a negative integer value of CUfileOpError enum value.
Note:
If the bufPtr is not registered with cuFileBufRegister, the data will be buffered through
preallocated pinned buffers if needed.
This is useful for applications that need to perform IO to unaligned file offsets and/or size. This
is also recommended for cases where the BAR1 memory size is smaller than the size of the allocated
memory.
Seealso:cuFileBufRegistercuFileHandleRegistercuFileWriteCUfileError_tcuFileReadAsync(CUfileHandle_tfh,void*bufPtr_base,size_t*size_p,off_t*file_offset_p,off_t*bufPtr_offset_p,ssize_t*bytes_read_p,CUstreamstream)Parameters:fh The cuFile handle for the file.
bufPtr_base base address of buffer in device or host memory
size_p pointer to size bytes to read
Note:
*size_p if the size is not known at the time of submission, then must provide the max possible size
for I/O request.
Parameters:file_offset_p pointer to file-offset from beginning of the file
bufPtr_offset_p pointer to offset relative to the bufPtr_base pointer to read into.
bytes_read_p pointer to the number of bytes that were successfully read.
CUstream stream cuda stream for the operation.
Returns:
size of bytes successfully read in *bytes_read_p
-1 on error, in which case errno is set to indicate filesystem errors.
all other errors will return a negative integer value of CUfileOpError enum value.
Note:
If the bufPtr_base is not registered with cuFileBufRegister, the data will be buffered through
preallocated pinned buffers.
This is useful for applications that need to perform IO to unaligned file offsets and/or size. This
is also recommended for cases where the BAR1 memory size is smaller than the size of the allocated
memory.
If the stream is registered with cuFileStreamRegister, the IO setup and teardown overhead will be
reduced.
on cuda stream errors, the user must call cuFileStreamDeregister to release any outstanding cuFile
resources for the stream.
Seealso:cuFileBufRegistercuFileHandleRegistercuFileReadcuFileStreamRegistercuFileStreamDeregisterCUfileError_tcuFileStreamDeregister(CUstreamstream)Parameters:CUstream cuda stream for the operation.
Note:
deallocates resources used by previous cuFile asynchronous operations for the cuda stream
highly recommend to call after cuda stream errors to release any outstanding cuFile resources for
this stream
must be called before cuStreamDestroy call for the specified stream.
This is useful for applications that need to perform IO to unaligned file offsets and/or size. This
is also recommended for cases where the BAR1 memory size is smaller than the size of the allocated
memory.
Returns:
CU_FILE_SUCCESS on success
CU_FILE_DRIVER_NOT_INITIALIZED if the driver is not initialized
CU_FILE_INVALID_VALUE if the stream is invalid
Seealso:cuFileReadAsynccuFileWriteAsynccuFileStreamRegisterCUfileError_tcuFileStreamRegister(CUstreamstream,unsignedflags)Parameters:CUstream cuda stream for the operation.
flags for the stream to improve the stream execution of IO based on input parameters.
Note:
supported FLAGS are
CU_FILE_STREAM_FIXED_BUF_OFFSET - buffer pointer offset is set at submission time
CU_FILE_STREAM_FIXED_FILE_OFFSET - file offset is set at submission time
CU_FILE_STREAM_FIXED_FILE_SIZE - file size is set at submission time
CU_FILE_STREAM_PAGE_ALIGNED_INPUTS - size, offset and buffer offset are 4k aligned
allocates resources needed to support cuFile operations asynchronously for the cuda stream
This is useful for applications that need to perform IO to unaligned file offsets and/or size. This
is also recommended for cases where the BAR1 memory size is smaller than the size of the allocated
memory.
Returns:
CU_FILE_SUCCESS on success
CU_FILE_DRIVER_NOT_INITIALIZED if the driver is not initialized
CU_FILE_INVALID_VALUE if the stream is invalid
Seealso:cuFileReadAsynccuFileWriteAsynccuFileStreamDeregisterlongcuFileUseCount(void)
returns use count of cufile drivers at that moment by the process.
ssize_tcuFileWrite(CUfileHandle_tfh,constvoid*bufPtr_base,size_tsize,off_tfile_offset,off_tbufPtr_offset)
write data from a specified device or host memory to a registered file handle
Parameters:fhCUfileHandle_t opaque file handle
bufPtr_base base address of buffer in device or host memory
size size bytes to write
file_offset file-offset from beginning of the file
bufPtr_offset offset relative to the bufPtr_base pointer to write from.
Returns:
size of bytes successfully written
-1 on error, in which case errno is set to indicate filesystem errors.
all other errors will return a negative integer value of CUfileOpError enum value.
Note:
If the bufPtr is not registered with cuFileBufRegister, the data will be buffered through
preallocated pinned buffers if needed.
This is useful for applications that need to perform IO to unaligned file offsets and/or size. This
is also recommended for cases where the BAR1 memory size is smaller than the size of the allocated
memory.
Seealso:cuFileBufRegistercuFileHandleRegistercuFileReadCUfileError_tcuFileWriteAsync(CUfileHandle_tfh,void*bufPtr_base,size_t*size_p,off_t*file_offset_p,off_t*bufPtr_offset_p,ssize_t*bytes_written_p,CUstreamstream)Parameters:fh The cuFile handle for the file.
bufPtr_base base address of buffer in device or host memory
size_p pointer to size bytes to write.
Note:
*size_p if the size is not known at the time of submission, then must provide the max possible size
for I/O request.
Parameters:file_offset_p pointer to file-offset from beginning of the file
bufPtr_offset_p pointer to offset relative to the bufPtr_base pointer to write from.
bytes_written_p pointer to the number of bytes that were successfully written.
CUstream cuda stream for the operation.
Returns:
size of bytes successfully written in *bytes_written_p
-1 on error, in which case errno is set to indicate filesystem errors.
all other errors will return a negative integer value of CUfileOpError enum value.
Note:
If the bufPtr_base is not registered with cuFileBufRegister, the data will be buffered through
preallocated pinned buffers.
This is useful for applications that need to perform IO to unaligned file offsets and/or size. This
is also recommended for cases where the BAR1 memory size is smaller than the size of the allocated
memory.
If the stream is registered with cuFileStreamRegister prior to this call, the IO setup and teardown
overhead will be reduced.
on cuda stream errors, the user must call cuFileStreamDeregister to release any outstanding cuFile
resources for the stream.
Seealso:cuFileBufRegistercuFileHandleRegistercuFileWritecuFileStreamRegistercuFileStreamDeregister
Install Now
PNG Icons
