Re: storage service times
- From: Dan Norris <dannorris@xxxxxxxxxxxxx>
- To: mzito@xxxxxxxxxxx, Oracle L <oracle-l@xxxxxxxxxxxxx>
- Date: Thu, 3 Jan 2008 12:54:12 -0800 (PST)
Thanks for the responses--I guess my standards are a bit too high :).
As one off-list responder also noted, the buffer waits are pretty high (it
seemed so to me as well), so I'm digging in to that and quite a few other
things as well. There's an 11 Gb buffer cache in this config, so many things
are on steroids. That's the largest buffer cache I've seen in production. I
guess memory really is getting cheaper!
Thanks again for the "gut checks".
Dan
----- Original Message ----
From: Matthew Zito <mzito@xxxxxxxxxxx>
To: dannorris@xxxxxxxxxxxxx; Oracle L <oracle-l@xxxxxxxxxxxxx>
Sent: Thursday, January 3, 2008 1:30:28 PM
Subject: RE: storage service times
<!--
_filtered {font-family:Tahoma;panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;margin-bottom:.0001pt;font-size:12.0pt;font-family:"Times
New Roman";}
a:link, span.MsoHyperlink
{color:blue;text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{color:purple;text-decoration:underline;}
span.EmailStyle17
{font-family:Arial;color:navy;}
_filtered {margin:1.0in 1.25in 1.0in 1.25in;}
div.Section1
{}
-->
The rule-of-thumb we used to use at EMC
was that DBAs started to notice and complain about storage performance at
around a 10-15ms service time. On lightly loaded arrays, it wasn’t
uncommon to see 1-3ms service times, and on arrays that were doing some work the
numbers you are seeing were right in line.
Now, of course, “performance”
is dependent on “results” – so if the database isn’t
providing the performance you need, then maybe the array isn’t keeping
up. But as an abstract value, 4-7ms service times don’t offend me.
Matt
From:
oracle-l-bounce@xxxxxxxxxxxxx [mailto:oracle-l-bounce@xxxxxxxxxxxxx] On Behalf
Of Dan Norris
Sent: Thursday, January 03, 2008
12:51 PM
To: Oracle L
Subject: storage service times
I'm looking over a system that has what I believe to be a
much-larger-than-average service time for read and write I/O.
DB 9.2.0.8
Solaris SPARC
VxVM/VxFS
Hitachi
(branded Sun 9990) storage array via 2xFC HBAs
8x 12 Gb storage array LUNs striped 256k stripe width using VxVM on the host =
about 90Gb volumes
Given that (somewhat incomplete) footprint, what are your first
"knee-jerk" reactions to these times for a 1-hour statspack report
interval:
Av Av
Av
Av Buffer Av Buf
Reads
Reads/s Rd (ms)
Blks/Rd Writes
Writes/s Waits Wt(ms)
-------------- ------- ------ ------- ------------ -------- ---------- ------
DATA1
397,958
111 8.3
1.0
79,676 22
4,114 4.8
IDX1
410,619
114 9.2
1.0
8,695 2
161,789 7.1
IDX2
159,094
44 8.3
1.0 137,040
38 31 5.2
These are rolled up to the tablespace level and I'm particularly interested in
comparing with other people's Av
Rd (ms) (the 3rd col) and Av Wt(ms) (the last col).
I'll go first...my gut feeling, based on past experiences elsewhere, is that 6+
ms seems about 50-100% higher than I expected to see.
I know I'm getting service times from Oracle without offering any info on what
the OS or Vol Mgr say about the service times viewed from those points of view,
but the service time in the DB is the only metric that matters at this point.
I'll dig in to the other layers as necessary when I start debugging. I'm
looking for your opinion to see if this is a problem or if I just set my
expectations too high. :-)
Thanks,
Dan
Other related posts:
