Re: Storage System Bakeoff test plans
- From: "Greg Rahn" <greg@xxxxxxxxxxxxxxxxxx>
- To: awells@xxxxxxxxxxxx
- Date: Wed, 9 Apr 2008 10:46:14 -0700
Oracle Orion is a IO simulation tool that uses the same code path that
the Oracle database uses for its IO calls. It can be quite useful,
but you have to understand how to use it. It's very important to
understand the IO pattern of the application/database you want to
simulate. Specifically for a warehouse this means understanding your
query workload. Are the queries using partition scans or index
access? Are you using parallel query or not? Are you using
nologging/direct path operations or not? How much of the workload
fits into each of those categories? What is the IOPS and I/O
bandwidth requirement? etc, etc, etc.
I'm a frequent user of Orion to get a "max observable" I/O bandwidth
number for the kits I use in data warehouse benchmarks (I use dd as
well). When I work with a hardware vendor we generally target a given
GB/s I/O throughput number. Some math is worked on paper, and we
validate our math with the Orion and dd tests. This gives us a max
observable I/O bandwidth number for the hardware using a low level I/O
tool. If the observed number is significantly different from the
calculated number, investigation is done to understand why. For these
tests I use a 1MB IO size which is the largest IO size that the Oracle
database currently requests. This effort allows me to understand how
much of the potential resource is being consumed by the database when
the real workload is running. Obviously it is not possible to drive
more I/O with a database than low level tool.
I could see you spending a month doing tests and going through the
results. Consider the number of combinations of vendors, LUN
configurations, RAID configurations, I/O multipathing, I/O load
balancing, simulations, and the fact that you will want to probably
run the tests more than once to make sure there are no anomalies.
It's educational to run some micro benchmarks but I would also run
tests that reflect the important parts of your business processing.
If "all the users in the world" don't use your database, then why test
it? Try and keep your tests clean and incremental. No sense of
running the "everything" test when you haven't tested each individual
part. If the results don't make sense, understand why, maybe it's not
the storage that is the bottleneck. I often tell people benchmarking
is like high school chemistry: First you work out the experiment on
paper, then you go into the lab and see if you get the expected
results. If the actual results don't reflect the expected results,
understand where it went wrong.
Best of luck.
On Wed, Apr 9, 2008 at 5:53 AM, April Wells <awells@xxxxxxxxxxxx> wrote:
> We are looking at doing a 'bakeoff' between three different storage vendors
> and I 'get' to do the test plan (as well as implement most of the plan).
>
> Has anyone used the Oracle Orion tool? The claim is that is all that we
> should have to use to load test the data warehouse on these three platforms.
> I'm not sure I buy it, but I would welcome the input of anyone who has
> actually used the tool. How is it at OLAP profiling?
>
> I am of the opinion that we ought to run actually "stuff" through the
> warehouse to test the performance (you know, things like the nightly ETL job
> at the same time we run the query from… well… you know) but it looks like I
> may be out voted. Opinions?
>
> We are planning on running one day of normal type processing, one day of
> 'the heaviest season of the year' and one day of 'all the users in the world
> are hitting it'… but all programmed with Orion… Suggestions?
--
Regards,
Greg Rahn
http://structureddata.org
--
//www.freelists.org/webpage/oracle-l
Other related posts:
