I am taking the stand that the Oracle article could only be referring to direct
attached tape drives as otherwise their statement is patently false.
I am going to have to respectively disagree with you on shoe shining and what
happens in a restore, where the tape would simply spin forward to the next
chunk relative to your back up and continue on if it was a restore on
Netbackup based system.
When the rate of data delivery to the tape head drops below the minimum
streaming rate of the tape drive (for most newer drives 27MB/sec), the tape
will reposition itself by stopping, reversing, the moving forward again for
the contiguous write of data – the “shoe-shining” effect.
Although Shoe Shining applies to writing of tape, multiplexing depending upon
your backup software applies to Restores also and in most cases for
optimization it tries to do as much as possible with one pass of the tape, but
it normally takes a second pass.
Specific to Netbackup:
https://www.veritas.com/support/en_US/article.TECH17269 How MPX_RESTORE_DELAY ;
can be used to improve restore performance on a NetBackup Oracle Recovery
Manager backup that was multiplexed
https://www.veritas.com/support/en_US/article.000076381 Effects of multiplexing ;
and multistreaming on backup and restore
https://www.veritas.com/support/en_US/article.000076379 Example of restore from ;
multiplexed database backup (Oracle)
Matthew Parker
Chief Technologist
Dimensional DBA
425-891-7934 (cell)
D&B 047931344
CAGE 7J5S7
Dimensional.dba@xxxxxxxxxxx
<http://www.linkedin.com/pub/matthew-parker/6/51b/944/> View Matthew Parker's
profile on LinkedIn
www.dimensionaldba.com <http://www.dimensionaldba.com/>
From: Seth Miller [mailto:sethmiller.sm@xxxxxxxxx] ;
Sent: Monday, August 1, 2016 1:41 PM
To: Dimensional DBA
Cc: Chris Taylor; Mladen Gogala; ORACLE-L
Subject: Re: Looking for Suggestions - 5 TB DB WHSE Backup options
Matthew,
The Oracle note isn't referring to backing up the data and has nothing to do
with direct attached tape. It is referring to the restore. Shoe-shining doesn't
just happen on writes, it happens during reads as well if the data being
restored is not written contiguously.
Seth Miller
On Mon, Aug 1, 2016 at 3:04 PM, Dimensional DBA <dimensional.dba@xxxxxxxxxxx>
wrote:
The Oracle advice was only accurate if you used direct attached tape drives to
your servers (circa 1990’s).
As to Seth’s comment on “Shoe Shining”, that only applies to slow writes to the
tape device where the head is having to backtrack to the last write point over
and over again because the stream of data can’t keep up with the streaming
speed of the tape drives. If the streams are maintaining the pipe to the tape
drive itself, then you will not get “shoe shining”.
The math shows 1Gb link, but are your links larger than that from DB server to
Netbackup media server? If it is then yes multiplexing at the Netbackup side
for tape drives can help boost performance. This is where I originally stated
there is architecture and Netbackup tuning that has to be done.
What version of tape drive do you have?
How many tape drives are dedicated to the Netbackup pool for your backups?
How many tape drivers are attached to each media server?
Intel or AMD architecture on the Netbackup media servers?
How many Netbackup media servers do you have to participate in the backup?
How much free memory is available on the Netbackup media server?
What is the chunk size setting of the Netbackup controlled tape drives.
How are you sending the data via RMAN? Are you simply letting Oracle chunk the
data or are you breaking the database backup apart into manageable chunks?
No Mladen, there is still a large portion of the world that uses tape directly
for backups except for slow streaming or glitchy data flow backups (like from
many windows servers). Yes there are customers who use disk too.
As to your question Chris
“Let's say we have 6 tapes and he's configured it to do 2 threads per tape.
Does that mean I can open 12 channels in RMAN? I mean, I would assume so but I
have no idea LOL“
Your math is correct but that assumes no one else is performing backups to
those tape drives. This is where you have to work with your Netbackup admin as
to what overall resources are available to you versus the rest of the backup
system to get the optimal flow.
At Amazon we had Netbackup tape multiplexing set to 8 and I have many clients
that use multiplexing at 4 to 6. IT really depends on what performance you are
getting for each incoming stream and how much shared memory is dedicated to
Netbackup on the media servers and number of tape drives per media server to
help maintaining streaming performance to the drive.
Matthew Parker
Chief Technologist
Dimensional DBA
425-891-7934 (cell)
D&B 047931344
CAGE 7J5S7
Dimensional.dba@xxxxxxxxxxx
<http://www.linkedin.com/pub/matthew-parker/6/51b/944/> View Matthew Parker's
profile on LinkedIn
www.dimensionaldba.com <http://www.dimensionaldba.com/>
From: oracle-l-bounce@xxxxxxxxxxxxx [mailto:oracle-l-bounce@xxxxxxxxxxxxx] On ;
Behalf Of Chris Taylor
Sent: Monday, August 1, 2016 11:31 AM
To: gogala.mladen@xxxxxxxxx
Cc: ORACLE-L
Subject: Re: Looking for Suggestions - 5 TB DB WHSE Backup options
Yeah Seth and I have been discussing this offline.
If a "tape device" has multiple tape writers, then you can allocate multiple
channels equal to the tape writers available in the "tape device". So this
advice in Oracle Doc does seem to be dated in regard to Backup systems like
Commvault, Netbackup etc.
Chris
On Mon, Aug 1, 2016 at 1:28 PM, Mladen Gogala <gogala.mladen@xxxxxxxxx> wrote:
Hi
One little clarification: that only applies if the tape device is really tape,
which is increasingly rare. All backup suites that I have ever worked with
(Commvault, NetBackup, EMC NetWorker) allocate SBT_TAPE device for writing to
disk storage, too. If the device type SBT channel is allocated for writing to
disk, it is reasonable to use multiple channels.
Regards
On 08/01/2016 12:36 PM, Seth Miller wrote:
Duplexing wouldn't make much sense since that would mean copying the same file
to multiple destinations. He is probably referring to multiplexing or
parallelism. There is a difference between RMAN multiplexing and parallelism
and the term "stream" usually relates to the latter. RMAN multiplexing allows
multiple files to be read at the same time within a single channel and defaults
to 8. RMAN parallelism refers to multiple channels writing to the same
destination and is often referred to as multiplexing on the media management
side.
Oracle suggests that no more than 1 channel be used per tape device to prevent
inefficient writes and shoe-shining.
"Do not utilize media management multiplexing (multiple channels per tape
drive). RMAN backup pieces will not be efficiently restored due to the
interleaving of pieces on the same tape volume, which may necessitate the
forward and backward movement of the tape."
Seth Miller
On Mon, Aug 1, 2016 at 11:20 AM, Chris Taylor
<christopherdtaylor1994@xxxxxxxxx> wrote:
*Basically* - He keeps talking about Streams/Duplexing on the tape side - where
2 threads can write to the same tape apparently? I *think* that part is done
on the Backup device outside of RMAN but I'm not sure how many channels I
should allocate within RMAN to take advantage.
For example I'm wondering (and I'll discuss with him) about the following
scenario:
Let's say we have 6 tapes and he's configured it to do 2 threads per tape.
Does that mean I can open 12 channels in RMAN? I mean, I would assume so but I
have no idea LOL
Chris
On Mon, Aug 1, 2016 at 10:44 AM, Seth Miller <sethmiller.sm@xxxxxxxxx> wrote:
Chris,
It sounds like you are currently parallelizing the backup to disk with 4
channels and your NBU guy is suggesting parallelizing your backup directly to
tape using multiple channels and you're asking how that is going to be
different than what you are doing now. Is this correct?
Seth Miller
On Mon, Aug 1, 2016 at 9:31 AM, Chris Taylor <christopherdtaylor1994@xxxxxxxxx>
wrote:
So we ran the first FULL on Friday and it took 9 hours - so your math is
excellent.
We did run into an ancillary problem however and that was that Netbackup was
configured to use a storage pool which filled up on the Netbackup server and
caused subsequent unrelated RMAN backups to fail for other databases. So,
we're working through that. Our Netbackup guy wants us to write directly to
tape and use multiple streams and is going to help us configure the RMAN script
to take advantage of this. I'm unclear on the relationship between RMAN
threads and Netbackup streams so I'm a little in the dark on what might need to
be changed at this point. Currently, I'm using 4 RMAN threads/channels to
write the backup.
Thanks,
Chris
On Tue, Jul 12, 2016 at 5:32 PM, Dimensional DBA <dimensional.dba@xxxxxxxxxxx>
wrote:
It really depends on your netbackup infrastructure and your network
infrastructure in between your database server and your backup infrastructure
or your remote copy on disk and your backup infrastructure.
If you have a 1GB backup network from your db server to your Netbackup
Infrastrucutre with proper tuning you can normally achieve 104MB/sec assuming
proper tuning and architecture or about 432GB/hr or in your case a 9hr full
backup.
If you have a 10Gb backup network from your db server to your Netbackup
Infrastructure with proper tuning you can normally achieve 780MB/sec assuming
proper tuning and architecture or about 2.88TB/hr or in your case a 2hr full
backup.
This is without any special equipment or SW licenses.
You also could
1. If you have the Oracle ASO for advanced compression then you can also
turn on compression in your RMAN backups and decrease the amount of data that
has to be transferred across the link.
2. look at performing mixed incrementals and partial fulls to spread a
full backup over multiple days.
3. Use the Incremental forever, although I wouldn’t recommend this.
Matthew Parker
Chief Technologist
Dimensional DBA
425-891-7934 (cell)
D&B 047931344
CAGE 7J5S7
Dimensional.dba@xxxxxxxxxxx
<http://www.linkedin.com/pub/matthew-parker/6/51b/944/> View Matthew Parker's
profile on LinkedIn
www.dimensionaldba.com <http://www.dimensionaldba.com/>
From: oracle-l-bounce@xxxxxxxxxxxxx [mailto:oracle-l-bounce@xxxxxxxxxxxxx] On ;
Behalf Of Ilmar Kerm
Sent: Tuesday, July 12, 2016 3:20 PM
To: christopherdtaylor1994@xxxxxxxxx
Cc: ORACLE-L
Subject: Re: Looking for Suggestions - 5 TB DB WHSE Backup options
Hi
We do "incremental forever" backups using incrementally updated image copies
that are located on a separate disk storage than the primary database. Before
updating the image copy, we snapshot the backup filesystem to provide backup
history. Also store a second copy of the archivelogs in that same filesystem
where the image copy is located - LOG_ARCHIVE_DEST_x parameter, so no separate
archivelog backup is needed.
Tape is ruled out in this case and you also need either a storage system that
can do NFS, snapshots, compression (optional) and thin clones (Oracle ZFSSA,
Netapp, ...) or a filesystem that can do these things (ACFS, ZFS, ..?). But all
the steps needed (except storage snapshots) can be done using RMAN and no 3rd
party libraries are needed - just need to write a script to orchestrate the
steps.
Ilmar
On Tue, Jul 12, 2016 at 7:42 PM, Chris Taylor
<christopherdtaylor1994@xxxxxxxxx> wrote:
Ok, guys & gals, I'm looking for suggestions for the following challenge. I'm
very familiar with RMAN fulls w/ incrementals writing either to disk or
directly to tape. What I'm NOT familiar with is other options that I may not
know of and that's where I need your help.
Objective:
Nightly backups of 5 TB Data Warehouse that is currently being snapshotted
weekly at the SAN Layer instead of tape or disk based backups.
Hardware/OS:
IBM XIV Storage (not sure of model #)
RedHat Linux OS (5.x)
Oracle 11.2.0.4
Netbackup is tape solution
Method Options:
1. RMAN Fulls on Weekend, (either to disk or direct to tape) with nightly
incrementals. I'm leaning toward disk based backups which are then written to
tape and using parallel threads for the disk based backup to prevent
overwhelming our tape library
2. Other options?
Chris
--
Ilmar Kerm
--
Mladen Gogala
Oracle DBA
Tel: (347) 321-1217 <tel:%28347%29%20321-1217>
