[arm-allstar] SD card recommendation

Tue Jun 16 02:19:25 EST 2015

Thanks Doug..

As some on the list might remember I was having a problem with long
write times to SD-cards.. in a couple of responses the discussion even
became heated by some assuming the speed of the card to be the only
factor, which I was sure of was not correct..  ending the discussion it
with I will research it myself..

Doug posting his card recommendations reminded me that I forgot to
report my findings back to the group.  Perhaps saving someone in the
future some time, and promote additional understanding I will report it
here.   Some of my discoveries, actually required looking at some BSD
kernel code to get this sorted out..  So let me see if I can explain
what was happening..  BTW my problem is now resolved..

My problem was what I considered extreme times in writing a image to a
SD card.. I am talking 55-60 minutes to write an image (MAC Intel, I7
2.88 with 16GB of ram, latest OSX)..  I did not expect slowness from
this system..  I now have those times down to around 8 minutes... It
took a few days to understand what was happening.. 

I started out using this command   dd bs=1m in=<image file>
of=/dev/disk3/   disk3 was the mount point when I plugged in the SD
card..   and before you ask.. I did unmounted the card before DDing the
card   This took between 55 and 60 minutes to complete..

I might also mention at this point that you do need a bs= as part of
your command as the default ibs (read bytes) and obs (write bytes) of DD
by defaulted is 512 bytes.. bs= forces both to be what ever byte size
you wish.. why because when DD was written almost all block devices were
blocked in 512 increments.. a lot of I/O if you don't block up in your
command..

I was very surprised when I found out that using a format like
/dev/disk3 actually is using a buffered I/O... in today's vs
yesterworld. I know I know this was not the case.. as I wrote assembly
and C code to do I/O years ago..  why is this so today.. because MAC as
well as some of the BSD's (FreeBSD) are using buffered I/O on raw block
devices.. I haven't investigated Linux because I spend about 100% of my
time between MAC and BSD OS's.. I would suspect they are doing the same
thing because much of the Linux stuff actually comes from the BSD
developers..   Why are O/S's are now doing this, when in older days they
didn't you might ask, its to protect data in case of a subsystem crash
or memory read error or other happenings. note this is the same type of
I/O that is now done on any disk drive that might be hooked up and
written to or read from.  To the kernel it does not know what type of
block device it is writing to so it does not know the difference between
a buss/USB (or other) attached device, nor does design require it to
know.  it only knows it is writing to a file i.e. the /dev/driver. Keep
this in mind when you read further..

So when using the output command of=/dev/disk3 multiple things were
happening.. first the kernel was reading the file and saving it in 1m
chunks, because I told it to in my dd statement <FILE read  --> Memory>
Then the USB device driver was re-buffering and re-blocking to 128K..
<Memory to Memory --> device > Then the actual SD card i.e. device is
re-blocking it to 4K.  This is also not happening all at once.. as being
a multi-user i.e. actually a multi-process O/S, the DD process as well
as the USB device stuff was also being swapped in and out as the active
process. I made not attempt to understand swaped out process times on
the system because this was insignificant on a fast system.  

Where you might ask is the 128K coming from..  It turns out that 128K is
the maximum block size for a USB device..  It also turns out that most
SD cards are written to and read from in 4K blocks, internal in the
card, so if you hit the card with an 7k block it takes longer to do an
actual write than writing with 2-4k blocks.. So keep data on 4K
boundaries.. is the lesson here..  just look at them as 4K devices.. and
block accordingly..  After exchange some email's with people who do
actual testing of SD cards it turns out they only test with 4k blocks of
data.. because they know that anything else is slower. All of the
going's on is what was causing my long write times..

It also turns out you can un-buffer the I/O by prefixing your actual
device with an r  (OSX) so changing my dd command to

dd bs=1m in=<image file> of=/dev/rdisk3/

dropped my write times from averaging 55 minutes to around 25 minutes.. 
At this point I might mention that my MAC also has a solid state
drive..  it turns out these are also 4k native devices.. and I/O is
being buffered there on reads.. that even added to my times as in
testing showed I could write significantly faster with a much slower BSD
system using spinning disks..

Moving forward as my interested peaked.. I wrote some scrips to check
write times with different blocking factors.. It turns out as I
suspected keeping bs on 4k boundaries did improve speed, but the fastest
speed was with 128K.. why because by going over 128K the USB driver
would then do an additional re-buffer and reblock the data before
passing to the device..  so 1m was badness.. 

Modifying my command to

dd bs=128k in=<image> of=/dev/rdisk3 

gave me times averaging around 8 minutes..  and I was actually doing
much faster I/O than when blocking at 1M as some on the list insisted it be.

While doing this research and testing I also discovered the following.. 
it seems that SD card manufactures are manufacturing there devices for
best performance when using consecutive writes and reads.. i.e. a large
photo for example.  I have here two examples of class 10 and class 4
cards from two different manufactures.. under actual testing the class
10 cards did in fact read and write faster as you would expect.. 
however and I say that with a big HOWEVER  actual real world testing
showed RANDOM read/writes of files to be slower on the faster cards.. 
the random R/W was a factor of around 10 slower on the faster devices..
than on the slower rated devices..  read what I said.. "In my case" 
meaning you can't compare the difference between cards without actually
testing the cards you have,   as I found a Kensington card (4Meg class
4)  to actually be faster with random I/O than  a SanDisk (16Meg class
10) card..  so while it took longer to write the image, the random I/O
was faster on the slower card.

So while I think a faster cards do carry a higher degree of testing
during manufacturing and are generally a much better fit to what we
actually are doing with these devices..  and I do actually think they
are a better more reliable card..  I am not so sure it will provide the
best performance in an actual booted system.. especially since I suspect
that Linux like the BSD and derivatives are doing buffered I/O to those
devices.. i.e. on our Pi2 systems.  I also suspect there might be some
gains in a custom kernel keeping block devices to 4K on Pi platforms..  
That is another day's research.. as is measuring actual I/O performance
on the pi2 when it is running, when and only then will have all the
answers I want..  dam I can be inquisitive at times.. but that is part
of this hobby..

Anyway wanted to report to the group.. keep your nodes up.. hope you
enjoy my findings.. enjoy...

Regards.. from Node 41780.. Fred.. WD8KNI..

On 6/16/15 12:31 AM, Doug Crompton wrote:
> This is the SD card I would recommend for the RPi2 or BBB for any new
> purchases.
>
> http://www.amazon.com/SanDisk-Extreme-Memory-Read-SDSDXN-016G-G46-Version/dp/B00MBFPT3U/ref=pd_sim_147_3?&ie=UTF8&refRID=1HPZNK58SXM37MNTC5WF
>
> *73 Doug
> WA3DSP
> http://www.crompton.com/hamradio*
>
> -- 
> This message has been scanned for viruses and
> dangerous content by *MailScanner* <http://www.mailscanner.info/>, and is
> believed to be clean.
>
>
> _______________________________________________
>
> arm-allstar mailing list
> arm-allstar at hamvoip.org
> http://lists.hamvoip.org/cgi-bin/mailman/listinfo/arm-allstar
>
> Visit the BBB web page - http://www.crompton.com/hamradio/BeagleBoneBlackAllstar/

-- 
Fred Moore
email: fred at fmeco.com
       fred at safes.com
phone:  321-217-8699

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.hamvoip.org/pipermail/arm-allstar/attachments/20150616/c5bb2cf3/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 535 bytes
Desc: OpenPGP digital signature
URL: <http://lists.hamvoip.org/pipermail/arm-allstar/attachments/20150616/c5bb2cf3/attachment-0001.pgp>