On 14/04/2012 04:31, Stan Hoeppner wrote:
On 4/13/2012 10:31 AM, Ed W wrote:
On 13/04/2012 13:33, Stan Hoeppner wrote:
In closing, I'll simply say this: If hardware, whether a mobo-down SATA chip, or a $100K SGI SAN RAID controller, allowed silent data corruption or transmission to occur, there would be no storage industry, and we'll all still be using pen and paper. The questions you're asking were solved by hardware and software engineers decades ago. You're fretting and asking about things that were solved decades ago. So why are so many people getting excited about it now? "So many"? I know of one person "getting excited" about it.
You love being vague don't you? Go on, I'll bite again, do you mean yourself?
:-)
Data densities and overall storage sizes and complexity at the top end of the spectrum are increasing at a faster rate than the consistency/validation mechanisms. That's the entire point of the various academic studies on the issue.
Again, you love being vague. By your dismissive "academic studies" phrase, do you mean studies done on a major industrial player, ie NetApp in this case? Or do you mean that it's rubbish because they asked someone with some background in statistics to do the work, rather than asking someone sitting nearby in the office to do it?
I don't think the researcher broke into NetApp to do this research, so we have to conclude that the industrial partner was onboard. NetApp seem to do a bunch of engineering of their own (got enough patents..) that I think we can safely assume they very much do their own research on this and it's not just "academic"... I doubt they publish all their own internal research, be thankful you got to see some of the results this way...
Note that the one study required a sample set of 1.5 million disk drives. If the phenomenon were a regular occurrence as you would have everyone here believe, they could have used a much smaller sample set.
Sigh... You could criticise the study if it had a small number of drives as being under-representive and now you criticise a large study for having too many observations...
You cannot have "too many" observations when measuring a small and unpredictable phenomena...
Where does it say that they could NOT have reproduced this study with just 10 drives? If you have 1.5 million available, why not use all the results??
Ed, this is an academic exercise. Academia leads industry. Almost always has. Academia blows the whistle and waves hands, prompting industry to take action.
Sigh... We are back to the start of the email thread again... Gosh you seem to love arguing and muddying the water for zero reason but to have the last word?
It's *trivial* to do a google search and hit *lots* of reports of corruptions in various parts of the system, from corrupting drivers, to hardware which writes incorrectly, to operating system flaws. I just found a bunch more in the Redhat database today while looking for something else. You yourself are very vocal on avoiding certain brands of HD controller which have been rumoured to cause corrupted data... (and thankyou for revealing that kind of thing - it's very helpful)
Don't veer off at a tangent now: The *original* email this has spawned is about a VERY specific point. RAID1 appears to offer less protection against a class of error conditions than does RAID6. Nothing more, nothing less. Don't veer off and talk about the minutiae of testing studies at universities, this is a straightforward claim that you have been jumping around and avoiding answering with claims of needing to educate me on SCSI protocols and other fatuous responses. Nor deviate and discuss that RAID6 is inappropriate for many situations - we all get that...
There is nothing normal users need to do to address this problem.
...except sit tight and hope they don't loose anything important!
:-)
Having the prestigious degree that you do, you should already understand the relationship between academic research and industry, and the considerable lead times involved.
I'm guessing you haven't attended higher education then? You are confusing graduate and post-graduate systems...
Byee
Ed W