Re: RMAN Performance Maladies
- From: Naqi Mirza <naqimirza@xxxxxxxxx>
- To: MFontana@xxxxxxxxx
- Date: Mon, 12 Feb 2007 12:03:47 -0800 (PST)
My apolgizes if this has ended up in flooding the list, for some reason I am
having problems in sending this, so here is my third attempt:
.
Hi,
I know one site that has their 4TB database backup in almost 18 hours, they are
on 9.2, using veritas netbackup 5. What are you using by the way, I seemed to
have missed that in the post? You also haven't mentioned whether your backups
are to disk or tape? If tape then which tape technology. If its LTO and i
remember correctly, LTO3 is the latest and each drive has a maximum rate of
80mb/s, So you may want to be the math for that and work out what you can
achieve - at least with all things being equal. Which is really a question in
itself - are all drives streaming with data and if so to what extent?. See from
what I have come to understand about rman - you will only be able to write data
to tape at least as fast as you can read it, if you're not reading fast enough
you won't be able to write much either.
Now, there are two dynamic performance views you can query to obtain this
information. Depending o n how the backup is being done-v$backup_io and
v$backup_aysnc_io (hope thats correct, do check the oracle reference manual).
Have a look at the column - effective_bytes_per_second with respect to the
column type (values here of INPUT - is what you are reading at, and OUTPUT is
what you are writing at - either to disk or tape). See if you are reading at
decent speed , then check to see if you are writing at least at this rate - you
can also check with your backup software to see the throughput that is being
achieved.
The problem I witnessed at a certain site - never got resolved, the site was
also using RAID 5 - I suspected this as the culprit - however incompetence on
my part, I could never really prove it. A few more things which you can check -
since i did the same. Is see whether a a file copy of a particular datafile
results in the same rate being achieved by your backup software. The issue we
had was that netbackup would show the total throughput of the backup as 50mb/s
no matter what was done - channel increase, maxopenfiles, filesperset, etc. The
sysadmin at the site claimed that rman was the problem and that o/s backups
would typically give them 100mb/s. I removed rman from the equation, and had
netbackup backup a datafile in backup mode - guess what it still showed a
throughput of 50mb/s - so it wasn't rman and this is what I believe, alas, to
this day.
Do check iostat , during your backup runs - does it reveal any io contention
that could be hindering the backup. Hows your cpu usage during backups - rman
is 'apparently' heavy on the cpu.
One more thing to add to the equation here, I know another site, which i used
as a comparison to the 18 hour site. Comparing the effective_bytes_per_Second
there showed a tremendous difference, they were reading alot more data and
therefore writing more too, thus , there 1.5TB database would complete its
backup in about 2-3hours (cannot recall now exactly which).
Well, I have digressed a bit here, hoping perhaps someone might be able to give
me a clue or two too. Hope what I have written gives you a start - checkout the
otn.oracle.com site - there are a few papers on there that help you
troubleshoot rman performance problems.
Naqi
----- Original Message ----
From: Michael Fontana <MFontana@xxxxxxxxx>
To: oracle-l@xxxxxxxxxxxxx
Sent: Monday, 12 February, 2007 9:15:22 PM
Subject: RMAN Performance Maladies
We have just implemented RMAN against our largest Siebel implementation
(don't laugh - but it's less then 1/2 terrabyte).
We did so because now that we have upgraded to Oracle 10gR2 (10.2.0.2)
we can take advantage of flash recovery features and full backup
compression similar to which we achived with our old backup product (BMC
Sqlbacktrack). We customarily take full backups nightly, as per our
SLA, and at this point, incrementals are not an option. However, the
actual runtime to complete a full database backup with RMAN compared to
SqlBackTrack has gone from 30 minutes to over 4 1/2 hours!
I have seen anecdotal reports of long runtimes for full RMAN backups on
this list, but I've seen an overwhelming number of guffaws from the more
elite and experienced poster indicating that this must be user error or
some other database performance problem that any competent DBA should
easily be able to solve without posting to this list. However, even
after opening an SR with Oracle support, who seems to believe our
runtime is in the expected range, we still believe something can be done
to overcome the huge IO spike and slow performance we are witnessing
during the backup, and we are not that experienced with RMAN from a
performance and tuning perspective, and would appreciated anyone's
insights or experiences.
My question is - has anyone actually run RMAN against a large (> 200g)
database right out of the box without tuning?
Has anyone spent a great deal of time tuning?
We are running 4 channels, have an 8 cpu Sunsparc solaris with 16g of
memory, and have experimented with recommended parameter settings for
maxsetsize and maxfilesperset without success. Any information or
assistance or comment would be appreciated, thanks in advance!
--
//www.freelists.org/webpage/oracle-l
___________________________________________________________
Now you can scan emails quickly with a reading pane. Get the new Yahoo! Mail.
http://uk.docs.yahoo.com/nowyoucan.html
Other related posts:
