Re: Miserable Disks

  • From: Wolfgang Breitling <breitliw@xxxxxxxxxxxxx>
  • To: charlottejanehammond@xxxxxxxxx
  • Date: Wed, 24 May 2006 05:38:31 -0600

Charlotte,

we had abysmal IO performance from a EMC Clariion (CX700) hooked up to AIX 5.2/5.3 at one time. It was with the "smaller", faster disks and the cache was not disabled, well at least not intentional. After some investigation it turned out that one of the 5 disks the Clariion uses to manage itself had gone bad and been replaced with the hot spare. The problem was that rebuilding the RAID (probably 5) somehow got stuck, so the SAN was perpetually in the rebuilding mode - which I believe essentially disabled the cache, but that wasn't the only damage to performance.
We didn't even notice immediately because Oracle's buffer cache kept the response from the databases reasonable.
When the storage SAs, under guidance by Dell technicians who sold us the Clariion, tried to get the process unstuck it crashed the entire SAN, both storage processors simultaneously.
It succeeded, however, in getting the process unstuck. ;-)
Not sure yours is the same problem. Sure hope not. Something like that should be a very isolated incident.

At 01:52 AM 5/24/2006, Charlotte Hammond wrote:

Hi All,

Some more details:

The filesystem is ext3 and it's on an EMC Clariion
storage array (low-end SATA unit - don't know the
exact model number), with fiber-channel connection.

The cache has been disabled due to problems with the
UPS - I was assuming that this was part of the problem
but couldn't account for all of it.

There are indeed several bad things happening:

* SATA disks
* RAID-5
* No cache

But even together I didn't expect things to be *this*
bad.  Am I looking for a non-existent fourth problem?
:-)  My current focus is on the 512-byte writes seen
on iostat - any ideas why I could be seeing this?


Regards

Wolfgang Breitling
Centrex Consulting Corporation
www.centrexcc.com 


--
//www.freelists.org/webpage/oracle-l


Other related posts:

  1. Home
  2. oracle-l
  3. Archive
  4. 05-2006