<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Thanks Doug.. <br>
<br>
As some on the list might remember I was having a problem with long
write times to SD-cards.. in a couple of responses the discussion
even became heated by some assuming the speed of the card to be the
only factor, which I was sure of was not correct.. ending the
discussion it with I will research it myself.. <br>
<br>
Doug posting his card recommendations reminded me that I forgot to
report my findings back to the group. Perhaps saving someone in the
future some time, and promote additional understanding I will report
it here. Some of my discoveries, actually required looking at some
BSD kernel code to get this sorted out.. So let me see if I can
explain what was happening.. BTW my problem is now resolved.. <br>
<br>
My problem was what I considered extreme times in writing a image to
a SD card.. I am talking 55-60 minutes to write an image (MAC Intel,
I7 2.88 with 16GB of ram, latest OSX).. I did not expect slowness
from this system.. I now have those times down to around 8
minutes... It took a few days to understand what was happening.. <br>
<br>
I started out using this command dd bs=1m in=<image file>
of=/dev/disk3/ disk3 was the mount point when I plugged in the SD
card.. and before you ask.. I did unmounted the card before DDing
the card This took between 55 and 60 minutes to complete.. <br>
<br>
I might also mention at this point that you do need a bs= as part of
your command as the default ibs (read bytes) and obs (write bytes)
of DD by defaulted is 512 bytes.. bs= forces both to be what ever
byte size you wish.. why because when DD was written almost all
block devices were blocked in 512 increments.. a lot of I/O if you
don't block up in your command.. <br>
<br>
I was very surprised when I found out that using a format like
/dev/disk3 actually is using a buffered I/O... in today's vs
yesterworld. I know I know this was not the case.. as I wrote
assembly and C code to do I/O years ago.. why is this so today..
because MAC as well as some of the BSD's (FreeBSD) are using
buffered I/O on raw block devices.. I haven't investigated Linux
because I spend about 100% of my time between MAC and BSD OS's.. I
would suspect they are doing the same thing because much of the
Linux stuff actually comes from the BSD developers.. Why are O/S's
are now doing this, when in older days they didn't you might ask,
its to protect data in case of a subsystem crash or memory read
error or other happenings. note this is the same type of I/O that is
now done on any disk drive that might be hooked up and written to or
read from. To the kernel it does not know what type of block device
it is writing to so it does not know the difference between a
buss/USB (or other) attached device, nor does design require it to
know. it only knows it is writing to a file i.e. the /dev/driver.
Keep this in mind when you read further.. <br>
<br>
So when using the output command of=/dev/disk3 multiple things were
happening.. first the kernel was reading the file and saving it in
1m chunks, because I told it to in my dd statement <FILE read
--> Memory> Then the USB device driver was re-buffering and
re-blocking to 128K.. <Memory to Memory --> device > Then
the actual SD card i.e. device is re-blocking it to 4K. This is
also not happening all at once.. as being a multi-user i.e. actually
a multi-process O/S, the DD process as well as the USB device stuff
was also being swapped in and out as the active process. I made not
attempt to understand swaped out process times on the system because
this was insignificant on a fast system. <br>
<br>
Where you might ask is the 128K coming from.. It turns out that
128K is the maximum block size for a USB device.. It also turns out
that most SD cards are written to and read from in 4K blocks,
internal in the card, so if you hit the card with an 7k block it
takes longer to do an actual write than writing with 2-4k blocks..
So keep data on 4K boundaries.. is the lesson here.. just look at
them as 4K devices.. and block accordingly.. After exchange some
email's with people who do actual testing of SD cards it turns out
they only test with 4k blocks of data.. because they know that
anything else is slower. All of the going's on is what was causing
my long write times.. <br>
<br>
It also turns out you can un-buffer the I/O by prefixing your actual
device with an r (OSX) so changing my dd command to <br>
<br>
dd bs=1m in=<image file> of=/dev/rdisk3/<br>
<br>
dropped my write times from averaging 55 minutes to around 25
minutes.. At this point I might mention that my MAC also has a
solid state drive.. it turns out these are also 4k native devices..
and I/O is being buffered there on reads.. that even added to my
times as in testing showed I could write significantly faster with a
much slower BSD system using spinning disks.. <br>
<br>
Moving forward as my interested peaked.. I wrote some scrips to
check write times with different blocking factors.. It turns out as
I suspected keeping bs on 4k boundaries did improve speed, but the
fastest speed was with 128K.. why because by going over 128K the USB
driver would then do an additional re-buffer and reblock the data
before passing to the device.. so 1m was badness.. <br>
<br>
Modifying my command to <br>
<br>
dd bs=128k in=<image> of=/dev/rdisk3 <br>
<br>
gave me times averaging around 8 minutes.. and I was actually doing
much faster I/O than when blocking at 1M as some on the list
insisted it be.<br>
<br>
While doing this research and testing I also discovered the
following.. it seems that SD card manufactures are manufacturing
there devices for best performance when using consecutive writes and
reads.. i.e. a large photo for example. I have here two examples of
class 10 and class 4 cards from two different manufactures.. under
actual testing the class 10 cards did in fact read and write faster
as you would expect.. however and I say that with a big HOWEVER
actual real world testing showed RANDOM read/writes of files to be
slower on the faster cards.. the random R/W was a factor of around
10 slower on the faster devices.. than on the slower rated
devices.. read what I said.. "In my case" meaning you can't
compare the difference between cards without actually testing the
cards you have, as I found a Kensington card (4Meg class 4) to
actually be faster with random I/O than a SanDisk (16Meg class 10)
card.. so while it took longer to write the image, the random I/O
was faster on the slower card.<br>
<br>
So while I think a faster cards do carry a higher degree of testing
during manufacturing and are generally a much better fit to what we
actually are doing with these devices.. and I do actually think
they are a better more reliable card.. I am not so sure it will
provide the best performance in an actual booted system.. especially
since I suspect that Linux like the BSD and derivatives are doing
buffered I/O to those devices.. i.e. on our Pi2 systems. I also
suspect there might be some gains in a custom kernel keeping block
devices to 4K on Pi platforms.. That is another day's research..
as is measuring actual I/O performance on the pi2 when it is
running, when and only then will have all the answers I want.. dam
I can be inquisitive at times.. but that is part of this hobby.. <br>
<br>
Anyway wanted to report to the group.. keep your nodes up.. hope you
enjoy my findings.. enjoy... <br>
<br>
Regards.. from Node 41780.. Fred.. WD8KNI.. <br>
<br>
<br>
<br>
<br>
<br>
<div class="moz-cite-prefix">On 6/16/15 12:31 AM, Doug Crompton
wrote:<br>
</div>
<blockquote cite="mid:BLU171-W12515D18374489D4F9D9000BAA70@phx.gbl"
type="cite">
<style><!--
.hmmessage P
{
margin:0px;
padding:0px
}
body.hmmessage
{
font-size: 12pt;
font-family:Calibri
}
--></style>
<div dir="ltr"><font style="" color="#000000"
face="Tahoma,sans-serif">This is the SD card I would recommend
for the RPi2 or BBB for any new purchases.<br>
<br>
<a moz-do-not-send="true"
href="http://www.amazon.com/SanDisk-Extreme-Memory-Read-SDSDXN-016G-G46-Version/dp/B00MBFPT3U/ref=pd_sim_147_3?&ie=UTF8&refRID=1HPZNK58SXM37MNTC5WF"
target="_blank">http://www.amazon.com/SanDisk-Extreme-Memory-Read-SDSDXN-016G-G46-Version/dp/B00MBFPT3U/ref=pd_sim_147_3?&ie=UTF8&refRID=1HPZNK58SXM37MNTC5WF</a><br>
<br id="FontBreak">
</font><b><font style="font-size:16pt;" size="4">73 Doug</font><font
style="font-size:16pt;" size="4"><br>
</font><font style="font-size:16pt;" size="4">WA3DSP</font><font
style="font-size:16pt;" size="4"><br>
</font><font style="font-size:16pt;" size="4"><a class="moz-txt-link-freetext" href="http://www.crompton.com/hamradio">http://www.crompton.com/hamradio</a></font></b><font
style="font-size:16pt;" size="4"><br>
</font> </div>
<br>
-- <br>
This message has been scanned for viruses and
<br>
dangerous content by
<a moz-do-not-send="true" href="http://www.mailscanner.info/"><b>MailScanner</b></a>,
and is
<br>
believed to be clean.
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
arm-allstar mailing list
<a class="moz-txt-link-abbreviated" href="mailto:arm-allstar@hamvoip.org">arm-allstar@hamvoip.org</a>
<a class="moz-txt-link-freetext" href="http://lists.hamvoip.org/cgi-bin/mailman/listinfo/arm-allstar">http://lists.hamvoip.org/cgi-bin/mailman/listinfo/arm-allstar</a>
Visit the BBB web page - <a class="moz-txt-link-freetext" href="http://www.crompton.com/hamradio/BeagleBoneBlackAllstar/">http://www.crompton.com/hamradio/BeagleBoneBlackAllstar/</a></pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Fred Moore
email: <a class="moz-txt-link-abbreviated" href="mailto:fred@fmeco.com">fred@fmeco.com</a>
<a class="moz-txt-link-abbreviated" href="mailto:fred@safes.com">fred@safes.com</a>
phone: 321-217-8699
</pre>
</body>
</html>