nvme_queue_rq函数用于提交一个req到nvme submission queue中。
1,nvme_setup_cmd用于构造nvme cmd:
nvme_setup_rw函数用于设置nvme读写命令的基础字段:
2,nvme_map_data函数为req的bio中所有vector建立dma散列表映射
老的磁盘控制器仅仅支持“简单”的DMA方式:磁盘必须与RAM中连续的内存单元传送数据。但是新的磁盘控制器支持分散聚集(scatter-gather)DMA方式,磁盘可以与一些非连续的内存区域相互传送数据。为了支持分散聚集DMA方式,块设备驱动程序必须能够处理称为段的数据存储单元,一次分散聚集DMA可以传送几个段(nr_phys_segments),一个段就是一个内存页或者内存页的一部分,它们包含磁盘上物理相邻的数据块。如果不同的段在RAM中相应的页框正好是连续的并且在磁盘上相应的数据也是相邻的,那么通用块层就可以进行合并,这种合并方式产生的更大的内存区域就称为物理段。当然nvme设置是支持这种DMA传送方式的。
https://lwn.net/Articles/256368/ 中关于散列表的描述:
Scatter/gather I/O allows the system to perform DMA I/O operations on buffers which are scattered throughout physical memory. Consider, for example, the case of a large (multi-page) buffer created in user space. The application sees a continuous range of virtual addresses, but the physical pages behind those addresses will almost certainly not be adjacent to each other. If that buffer is to be written to a device in a single I/O operation, one of two things must be done: (1) the data must be copied into a physically-contiguous buffer, or (2) the device must be able to work with a list of physical addresses and lengths, grabbing the right amount of data from each segment. Scatter/gather I/O, by eliminating the need to copy data into contiguous buffers, can greatly increase the efficiency of I/O operations while simultaneously getting around the problem that the creation of large, physically-contiguous buffers can be problematic in the first place.
块设备的request请求中所能携带的最大的段数,由下面的函数设置:
blk_rq_map_sq函数是核心的dma映射函数:
散列表sg的结构:
__blk_bvec_map_sq函数用于映射单个bio sector:
__blk_bios_map_sq函数为req中每个bio所有的vector创建dma映射:
3,nvme_pci_setup_prps函数设置nvme cmd中prp字段
在设置prp之前,会判断是用sgl还是prp:
设置prp字段:
4,提交nvme cmd
写sq tail doorbell寄存器