[arm-allstar] faster SD card write
Doug Crompton
doug at crompton.com
Sun Nov 22 01:26:14 EST 2015
OK so now we know that DD parameters have little effect on write speed in most cases. What really does have effect is the type of card and the speed of the writer. Most cheap writers are USB2 but to really achieve speed you need USB3 and a FAST card. Of course you pay for it and unless you are really impatient and write a lot of images it would not be worth it. If you are writing an occasional image to SD just start the burn and go get a cup of coffee!
Currently the fastest write speed for a micro SD card seems to be Sandisk at better than 50MB/sec
http://www.bhphotovideo.com/explora/photography/buying-guide/fastest-memory-cards-money-can-buy
http://highonandroid.com/android-accessories/fastest-micro-sd-card-shootout-samsung-vs-sony-vs-sandisk/
BTW they are getting cheaper - the Sandisk Extreme pro 8Gb is $17 at amazon, about twice as much as the standard ultra card.
73 Doug
WA3DSP
http://www.crompton.com/hamradio
> Date: Sat, 21 Nov 2015 21:13:10 -0500
> From: kb4fxc at inttek.net
> To: arm-allstar at hamvoip.org
> Subject: Re: [arm-allstar] faster SD card write
>
>
> Hi Guys,
>
> I found this thread interesting, too. Over the last few years, I've
> written firmware images to hundreds or thousands of flash cards. And,
> years ago, I just increased the buffer sizes in dd (probably following
> advise from some website). The increased size made sense to me, it was
> much faster and I didn't study it any further. Typically, the
> "recommended" command I mention to people is very similar to:
>
> dd if=<input file> of=/dev/sdX bs=1M
>
> or
>
> zcat <gzipped input image file> | dd of=/dev/sdX bs=1M
>
> ...where /dev/sdX is the linux SD card block interface.
>
> So, since there is some interest and I'm wondering about the optimal
> settings, too, I did some googling and testing. I found this webpage that
> has some scripts: http://blog.tdg5.com/tuning-dd-block-size/ ...Please
> find the test script I used attached to bottom of this message (below).
> For this test, I transferred a 64MB file of zeros from /dev/zero.
>
>
> I tried 3 test setups, all Linux:
>
> First, my primary setup is a roaring 4-core high-speed 3.6GHz 64-bit linux
> PC running Debian Wheezy, with a cheapo 8GB Sandisk SD card attached via a
> good performing external USB 2.0 hub. Here are the results:
>
> david-vb:/home/mcgough/ham/RasPi2# ./dd_obs_test.sh /dev/sdf
>
> block size : transfer rate
> 512 : 2.2 MB/s
> 1024 : 2.2 MB/s
> 2048 : 2.2 MB/s
> 4096 : 5.8 MB/s
> 8192 : 5.8 MB/s
> 16384 : 5.8 MB/s
> 32768 : 5.8 MB/s
> 65536 : 5.8 MB/s
> 131072 : 5.8 MB/s
> 262144 : 5.8 MB/s
> 524288 : 5.8 MB/s
> 1048576 : 5.4 MB/s
> 2097152 : 5.8 MB/s
> 4194304 : 5.4 MB/s
> 8388608 : 5.8 MB/s
> 16777216 : 5.4 MB/s
> 33554432 : 5.4 MB/s
> 67108864 : 5.4 MB/s
>
> Second, the same setup as above, except the SD card reader is plugged
> directly into a USB 2.0 port on the PC, rather than the external
> hub...Virtually the same speed:
>
> david-vb:/home/mcgough/ham/RasPi2# ./dd_obs_test.sh /dev/sdf
> block size : transfer rate
> 512 : 2.4 MB/s
> 1024 : 2.2 MB/s
> 2048 : 2.2 MB/s
> 4096 : 5.8 MB/s
> 8192 : 5.8 MB/s
> 16384 : 5.8 MB/s
> 32768 : 5.8 MB/s
> 65536 : 5.8 MB/s
> 131072 : 5.8 MB/s
> 262144 : 5.8 MB/s
> 524288 : 5.8 MB/s
> 1048576 : 5.8 MB/s
> 2097152 : 5.4 MB/s
> 4194304 : 5.8 MB/s
> 8388608 : 5.8 MB/s
> 16777216 : 5.4 MB/s
> 33554432 : 5.3 MB/s
> 67108864 : 5.4 MB/s
>
> Finally, the same test, using the same SD card reader, but now plugged
> into a RPi2 USB port. Note that the RPi2 is running our latest & greatest
> beta firmware with a 4.1.13 kernel. Also, the SD card block device is
> now /dev/sda (it was /dev/sdf on the PC):
>
>
> [root at RPi2-dev2 ~]# ./dd_obs_test.sh /dev/sda
> block size : transfer rate
> 512 : 2.3 MB/s
> 1024 : 2.3 MB/s
> 2048 : 2.4 MB/s
> 4096 : 5.6 MB/s
> 8192 : 5.6 MB/s
> 16384 : 5.6 MB/s
> 32768 : 5.7 MB/s
> 65536 : 5.6 MB/s
> 131072 : 5.6 MB/s
> 262144 : 5.6 MB/s
> 524288 : 5.6 MB/s
> 1048576 : 5.6 MB/s
> 2097152 : 5.6 MB/s
> 4194304 : 5.6 MB/s
> 8388608 : 5.6 MB/s
> 16777216 : 5.6 MB/s
> 33554432 : 5.6 MB/s
> 67108864 : 5.5 MB/s
>
> Again, using the RPi2 also yielded the results I expected.
>
> So, on the positive side, using a 1 megabyte buffer (like I typically
> tell people) is perfectly acceptable....But, a smaller buffer is fine
> too--ANY buffer size at least 4K bytes seems to yield optimal thru-put.
>
> NOTE that another useful test would be to monitor CPU use while performing
> each test and report this as well...I expect the larger buffer sizes will
> win here, slightly, with quickly diminishing returns once past perhaps
> 128K byte sizes....
>
> As with everything, your mileage may vary. I recommend trying this script
> on -YOUR- system, too. And, please report the results!
>
> Here is the script:
>
> #------------------------------------------------------------------------------------
> #!/bin/bash
>
> # Since we're dealing with dd, abort if any errors occur
> set -e
>
> TEST_FILE=${1:-dd_obs_testfile}
> [ -e "$TEST_FILE" ]; TEST_FILE_EXISTS=$?
> ###TEST_FILE_SIZE=134217728
> TEST_FILE_SIZE=67108864
>
> # Header
> PRINTF_FORMAT="%8s : %s\n"
> printf "$PRINTF_FORMAT" 'block size' 'transfer rate'
>
> # Block sizes of 512b 1K 2K 4K 8K 16K 32K 64K 128K 256K 512K 1M 2M 4M 8M 16M 32M 64M
> for BLOCK_SIZE in 512 1024 2048 4096 8192 16384 32768 65536 131072 262144
> 524288 1048576 2097152 4194304 8388608 16777216 33554432 67108864
> do
> # Calculate number of segments required to copy
> COUNT=$(($TEST_FILE_SIZE / $BLOCK_SIZE))
>
> if [ $COUNT -le 0 ]; then
> echo "Block size of $BLOCK_SIZE estimated to require $COUNT blocks, aborting further tests."
> break
> fi
>
> # Create a test file with the specified block size
> DD_RESULT=$(dd if=/dev/zero of=$TEST_FILE bs=$BLOCK_SIZE count=$COUNT 2>&1 1>/dev/null)
>
> # Extract the transfer rate from dd's STDERR output
> TRANSFER_RATE=$(echo $DD_RESULT | \grep --only-matching -E '[0-9.]+ ([MGk]?B|bytes)/s(ec)?')
>
> # Clean up the test file if we created one
> [ $TEST_FILE_EXISTS -ne 0 ] && rm $TEST_FILE
>
> # Output the result
> printf "$PRINTF_FORMAT" "$BLOCK_SIZE" "$TRANSFER_RATE"
> done
> #------------------------------------------------------------------------------------
>
>
> Have fun!
>
> 73, David KB4FXC
>
> _______________________________________________
>
> arm-allstar mailing list
> arm-allstar at hamvoip.org
> http://lists.hamvoip.org/cgi-bin/mailman/listinfo/arm-allstar
>
> Visit the BBB and RPi2 web page - http://hamvoip.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.hamvoip.org/pipermail/arm-allstar/attachments/20151122/94132be8/attachment.html>
More information about the arm-allstar
mailing list