[SSD] boost I/O performance with blk-mq

Head_on_a_Stick · 2017-02-25 12:45:10

Theory

The Linux kernel's traditional handling of I/O requests (scheduling) presents a bottleneck for modern solid state storage devices because the design is predicted on a single request queue which becomes quickly saturated by modern drives.

As an alternative to the usual I/O schedulers (noop, deadline, cfq & bfq), the Linux Multi-Queue Block I/O Queueing Mechanism can be employed instead:

https://www.thomas-krenn.com/en/wiki/Li … 8blk-mq%29

This splits any I/O requests into submission queues (allocated across all CPUs) and hardware dispatch queues that buffer requests directly from the driver; the parallelisation greatly increases overall I/O capability.

Better explanation here:

https://lwn.net/Articles/552904/

Method

Edit the file at /etc/default/grub (as root!) and change the GRUB_CMDLINE_LINUX line — add the two parameters between the quotation marks, like this:

GRUB_CMDLINE_LINUX="scsi_mod.use_blk_mq=y dm_mod.use_blk_mq=y"

Then save the file and update the GRUB configuration with:

sudo grub-mkconfig -o /boot/grub/grub.cfg

Then reboot and check that no schedulers are being used for the drive:

Helium: ~ $ cat /sys/class/block/sda/queue/scheduler 
none

This method should only be used on solid state drives, spinning rust devices will not benefit.

damo · 2017-02-25 13:13:20

^ O/T - why do you usually recommend using 'grub-mkconfig ....' instead of just 'update-grub'?

Head_on_a_Stick · 2017-02-25 13:17:44

damo wrote:

why do you usually recommend using 'grub-mkconfig ....' instead of just 'update-grub'?

Because I can never remember if the command is `update-grub` or `grub-update`

Anyway, they do exactly the same thing — check the output of:

cat $(which update-grub)

damo · 2017-02-25 13:19:31

Yes, 'update-grub' is a stub for the grub-mkconfig command. Like an alias - much easier for me to remember!

Head_on_a_Stick · 2017-08-13 14:38:33

Since the release of kernel version 4.12, the block I/O mechanism has had some new options added:

empty@Xanadu:~ $ cat /sys/class/block/sda/queue/scheduler                                                                                                                    
mq-deadline [kyber] bfq none
empty@Xanadu:~ $

More details here:

https://lwn.net/Articles/720675/

Phoronix has some benchmarks:

http://www.phoronix.com/scan.php?page=a … 2-io&num=1

I have decided to try the Kyber scheduler which is a very simple implementation (~1,000 lines of code) that attempts to do as little as possible, just like me

Unfortunately, the usual elevator=$scheduler kernel command-line parameter doesn't work with blk-mq (yet) so I have used systemd-tmpfiles to apply the modification:

# /etc/tmpfiles.d/10_ioscheduler.conf                                                                                                                     
w /sys/block/sda/queue/scheduler - - - - kyber

beaker · 2017-08-14 10:08:19

Do you notice a difference between that and one of the i/o schedulers?

Head_on_a_Stick · 2017-08-15 06:01:31

^ Not at all, no

If you check the Phoronix benchmarks the differences are pretty minor and vary according to the task in hand so there is no clear-cut "winner", as such.

I have made my choice based on what I consider to be the best paradigm — it's all about the principle, goshdarnit! 8)

beaker · 2017-08-15 08:35:45

Fair enough.

#1 2017-02-25 12:45:10