Whilst investigating ways to improve the cold boot time of embedded Linux I came across a little-known control parameter of the USB stack known as ‘delay_use‘. It’s a parameter that describes the amount of time given to Mass Storage Devices to allow them to ‘settle down’ before being used. This article examines ‘delay_use’ and identifies how it may be used to reduce boot time and improve responsiveness.
The sole reason for the existence of ‘delay_use’ is that some USB Mass Storage Devices are unable to correctly respond to the kernel immediately after being powered up (e.g. here, here and here). I’m not sure the reasons why but would suspect it’s either because they internally need to load firmware which takes time, or require access to slow hardware (spinning disks) before getting the information they need to respond. In any case, if these devices are not given enough time then they don’t respond correctly and the kernel is unable to detect and use them.
Therefore if the kernel wishes to support a broad range of devices with every chance of success then the kernel should set this settling period high. However this delay affects all mass storage devices and the kernel will wait for the whole settle period regardless to when the device becomes ready – this means that if this value is set high then upon inserting a mass storage device the kernel will take a long time before making it available for us – thus giving the appearance of an unresponsive system. Furthermore, if the mass storage device is a boot device or used during boot then a large settle period may affect boot time.
We can see the effect of the settling period by changing it on a running system. The following command will change the settle period to 0 seconds.
echo 0 > /sys/module/usb_storage/parameters/delay_use
Upon inserting my USB Mass Storage Device (my phone) I see the following output:
[28672.215734] usb 3-1: new high-speed USB device number 4 using xhci_hcd [28672.247360] usb 3-1: ep 0x84 - rounding interval to 32768 microframes, ep desc says 0 microframes [28672.248612] scsi9 : usb-storage 3-1:1.0 [28672.252679] scsi 9:0:0:0: Direct-Access HTC Android Phone 0100 PQ: 0 ANSI: 2 [28672.254530] sd 9:0:0:0: Attached scsi generic sg3 type 0 [28672.261867] sd 9:0:0:0: [sdc] Attached SCSI removable disk
If you look at the printk times you will see that the /dev/sdc device is available within 47 ms of the insert being detected. Now let’s try increasing the settle period:
echo 30 > /sys/module/usb_storage/parameters/delay_use
[28921.169073] usb 3-1: new high-speed USB device number 6 using xhci_hcd [28921.200909] usb 3-1: ep 0x84 - rounding interval to 32768 microframes, ep desc says 0 microframes [28921.203437] scsi11 : usb-storage 3-1:1.0 [28951.288361] scsi 11:0:0:0: Direct-Access HTC Android Phone 0100 PQ: 0 ANSI: 2 [28951.288936] sd 11:0:0:0: Attached scsi generic sg3 type 0 [28951.294657] sd 11:0:0:0: [sdc] Attached SCSI removable disk
This time it took 30.125 seconds!
This give the kernel a dilemma – a large settle period results in slow initial enumeration which can impact on boot time and perceived responsiveness, but a short settle period may result in more devices failing to be detected. What should the kernel use for its default value?
There isn’t a correct value and over the course of the kernel’s development the default value has changed – when ‘delay_use’ was introduced in the 2.6.10 kernel it was set to 5 seconds, but this was later reduced by Linus to 1 second for the 2.6.34 kernel (thought not without side effects).
For embedded devices/products, where the complete range of USB devices to be supported is known or where the USB devices are hard wired then it is possible, through testing/trial-and-error to determine an optimal delay_use value – this may very well end up being 0.
In order to specify a settling period to be used during boot the following kernel argument can be used:
usb-storage.delay_use=0
Additional documentation can be found in Documentation/kernel-parameters.txt
In addition to tweaking the settling period, additional boot time optimisations in this area may include starting USB enumeration as early as possible during boot and starting it in it’s own thread and if some devices do require a high value for delay_use then attempting to apply the delay to only those devices.