Re: RAC in NAS

  • From: "Mark Brinsmead" <pythianbrinsmead@xxxxxxxxx>
  • To: "Jared Still" <jkstill@xxxxxxxxx>
  • Date: Mon, 31 Jul 2006 10:07:56 -0600

Comments inline below...

On 7/31/06, Jared Still <jkstill@xxxxxxxxx> wrote:



Small wonder.  The technical architects are not usually experts
in storaage.  That is, down to the detail level.


It's pretty tough to be an expert "down to the detail level". I know that *I* cannot claim to be. For this you would probably need proprietary information about about the internal implementation of a given storage solution. Something you *quite* unlikely to get from sales reps.

At the same time, I am often astounded by the apparent ignorance of
even the storage basics.  Like placing 60TB of storage for 24x7 systems
into a single (midrange) disk array with a *maximum* throughput of 400MB/s.
Without any *thought* of how the data will be backed up, let alone
recovered.

(400MB/s *sounds* like a lot of bandwidth, but we need to leave some for
the *application* to use...  And in the instance I am thinking of, this was
only onle application of several dozen sharing a "corporate" backup
infrastructure with a maximum throughput of only about 80MB/s...)

And then there are the guys who will cheerfully place a 400GB database
on a (single) 750GB ATA disk drive.  No problem -- after all, there's plenty
of room for growth, isn't there?

"Detail level" expertise is certainly helpful, but really, all you need to
avoid
some of the more common blunders I have seen is an understanding of
basic disk drive geometry  (there's this "spinning thingy", and then there's
this "seeking thingy", and that's about it...) and some grade-5 (well, these
days, maybe it's grade-8) math skills.

I mean, staying away from the "rediculous" situations is not rocket
science...

I'll admit though, that avoiding "surprises" (e.g., Asynch I/O not working
with
NFS) may require a little more expertise.  I *could* see myself being caught
by surprise on that one (on a really bad day), but I would like to think
that
my usual level of "homework" would have caught that...

(In the case where I first encountered the AIO-with-NFS "mistake", it took
only a few hours of research to learn all that I needed, but this was a
"hindsight" scenario, where the symptoms were already known.  I cannot
honestly say that I would definitely have *predicted* this particular
problem.
Sadly, the guy who sold my client the NFS disk array probably *could*
have.)

The problem is lack of education. The reason for that is that an
education storage management is very difficult to come by unless
you deal with it every day.


Or even then... ;-) I know more than a few IT departments with effectively zero-dollar budgets for education. And those who do have (limited) budgets are unlikely to "waste" them on something like storage management. After all, the salesman who sold the storage arrays *did* say they were *easy* to manage. (And there all *all* those COBOL programmers who need to come up to speed with the latest O-O COBOL...)

Try looking for a book that gives a concise explanation of storage
management.
There isn't one.  I've looked on amazon, googled and asked the people that
would know if such a thing exists.  It doesn't.


I've had the same problem. It's almost like somebody doesn't *want* people to know these things. Hey. Wait a minute ... ;-)

Unless the TA has such an expert in house, or knows where to find one,
there's no one else to turn to than the vendors.


Really? They could maybe start with their DBAs... True, I *have* met a few DBAs who don't know how disk drives work, but I like to think that such DBAs are becoming fewer in number (or at least do not survive for long). Okay, maybe I live in a fantasy world, but at least it's a nice one. ;-) In any event, I would think that most DBAs have the necessary skills to tell a good storage solution from a collossally bad one. And sometimes even to identify those which are "supported" or "not supported" by Oracle.

That said, however, many of the more "curious" technical architecture
decisions I have observed have something else in common (aside from
over-reliance on "vendor input").  They have involved selections of
server platforms or storage arays without obtaining input from the DBAs.
Hmm...


Of course, when doing my own homework, I *do* look pretty closely at
> vendor-supplied information.
> I usually skip the brochures, though, and go straight to the spec sheets
> and (when available) reference
> manuals.  When I feel like having fun, I do this *before* the sales rep
> is invited to visit...  ;-)
>


Those spec sheets are useless unless you know enough about storage to interpret them, or have someone available to do so for you.


True. And you can tell a lot of lies (and hide a large number of embarassing truths) in a page of numbers.

That's why I usually go for the reference manuals.  That's where you can
learn
whether the cool whiz-bang (and expensive) feature the storage sales rep is
pushing will actually benefit you -- or even *work* -- in your particular
situation.

I know more than one organisation  that has coughed up hundreds of thousands
of dollars for "cool" database or storage features, only to discover after
the fact
that the features cannot be used.  (Oddly, in most casese, they were still
paying
for *support* on those unused features five years later...)

This is often dependent on the size of the company. Small to medium
size businesses do not often have the expertise necessary to understand
all the implications of different storage technologies when considered
with a specific use such as Oracle Databases.  They often also do not
have sufficient resources to fully explore this in a lab environment.

Back to the vendors, once again.


"Back to the vendors?" I hope not! Yes, vendors are often an important source of information, but they should (very) rarely become a source of *advice*! In my humble opinion, anyway.

Spending just a few thousand dollars for a consultant can be a fabulous
investment in situations like this.  For SMBs, this is especially crucial,
since they usually have no budget for re-work, and the entire corporation
may depend on the success of a single project.

--
Jared Still
Certifiable Oracle DBA and Part Time Perl Evangelist




-- Cheers, -- Mark Brinsmead Staff DBA, The Pythian Group http://www.pythian.com/blogs

Other related posts: