Direct IO uses the kiobuf mechanism [see the Linux
Device Drivers book] to manipulate memory
allocated within the user space so that a lower level (adapter) driver
can DMA directly to or from that user space memory. Since the user
can give a different data buffer to each SCSI command passed through
the sg interface then the kiobuf mechanism needs to setup its structures
(and undo that setup) for each SCSI command.
[1]
Direct IO is available as an option in sg 3.1.18 (before that the sg driver
needed to be recompiled with an altered define). Direct IO support is
designed in such a way that if it is requested and cannot be performed
then the command will still be performed using indirect IO. If direct IO
is requested and has been performed then the SG_INFO_DIRECT_IO bit will
be set in the 'info' member of the sg_io_hdr_t control structure after
the request has been completed. Direct IO is not supported on ISA SCSI
adapters since they only can address a 24 bit address space.
One limit on direct IO is that sg_io_hdr_t::iovec_count==0. So the user
cannot (currently) use application level scatter gather and direct IO on
the same request.
For direct IO to be worthwhile, a reasonable amount of data should be
requested for data transfer. For transfers less than 8 KByte it is
probably not worth the trouble. On the other hand "locking down" a
multiple 512 KB blocks of data for direct IO could adversely impact
overall system performance. Remember that for the duration of a direct
IO request, the data transfer buffer is mapped to a fixed memory location
and locked in such a way that it won't be swapped out. This can "cramp
the style" of the kernel if it is overdone.
Prior to sg 3.1.18 the direct IO code was commented out with the
"SG_ALLOW_DIO" define. In sg 3.1.18 (available for lk 2.4.2 and later)
the direct IO code is active but is defaulted off by a run time value.
This value can be accessed via the "proc" file system at
/proc/scsi/sg/allow_dio . Direct IO is enabled when
a user with root permissions writes "1" to that file:
echo 1 > /proc/scsi/sg/allow_dio . If SG_FLAG_DIRECT_IO
is set in sg_io_hdr::flags but /proc/scsi/sg/allow_dio
holds "0" then indirect IO will be performed (and this is indicated by
((sg_io_hdr::info & SG_INFO_DIRECT_IO_MASK) == SG_INFO_INDIRECT_IO)
after the request is completed).