Matthew,
The Oracle note isn't referring to backing up the data and has nothing to
do with direct attached tape. It is referring to the restore. Shoe-shining
doesn't just happen on writes, it happens during reads as well if the data
being restored is not written contiguously.
Seth Miller
On Mon, Aug 1, 2016 at 3:04 PM, Dimensional DBA <dimensional.dba@xxxxxxxxxxx
wrote:
The Oracle advice was only accurate if you used direct attached tape
drives to your servers (circa 1990’s).
As to Seth’s comment on “Shoe Shining”, that only applies to slow writes
to the tape device where the head is having to backtrack to the last write
point over and over again because the stream of data can’t keep up with the
streaming speed of the tape drives. If the streams are maintaining the pipe
to the tape drive itself, then you will not get “shoe shining”.
The math shows 1Gb link, but are your links larger than that from DB
server to Netbackup media server? If it is then yes multiplexing at the
Netbackup side for tape drives can help boost performance. This is where I
originally stated there is architecture and Netbackup tuning that has to be
done.
What version of tape drive do you have?
How many tape drives are dedicated to the Netbackup pool for your backups?
How many tape drivers are attached to each media server?
Intel or AMD architecture on the Netbackup media servers?
How many Netbackup media servers do you have to participate in the backup?
How much free memory is available on the Netbackup media server?
What is the chunk size setting of the Netbackup controlled tape drives.
How are you sending the data via RMAN? Are you simply letting Oracle chunk
the data or are you breaking the database backup apart into manageable
chunks?
No Mladen, there is still a large portion of the world that uses tape
directly for backups except for slow streaming or glitchy data flow backups
(like from many windows servers). Yes there are customers who use disk too.
As to your question Chris
“Let's say we have 6 tapes and he's configured it to do 2 threads per
tape. Does that mean I can open 12 channels in RMAN? I mean, I would
assume so but I have no idea LOL“
Your math is correct but that assumes no one else is performing backups to
those tape drives. This is where you have to work with your Netbackup admin
as to what overall resources are available to you versus the rest of the
backup system to get the optimal flow.
At Amazon we had Netbackup tape multiplexing set to 8 and I have many
clients that use multiplexing at 4 to 6. IT really depends on what
performance you are getting for each incoming stream and how much shared
memory is dedicated to Netbackup on the media servers and number of tape
drives per media server to help maintaining streaming performance to the
drive.
*Matthew Parker*
*Chief Technologist*
*Dimensional DBA*
*425-891-7934 (cell)*
*D&B *047931344
*CAGE *7J5S7
*Dimensional.dba@xxxxxxxxxxx <Dimensional.dba@xxxxxxxxxxx>*
*View Matthew Parker's profile on LinkedIn*
<http://www.linkedin.com/pub/matthew-parker/6/51b/944/>
www.dimensionaldba.com
*From:* oracle-l-bounce@xxxxxxxxxxxxx [mailto:
oracle-l-bounce@xxxxxxxxxxxxx] *On Behalf Of *Chris Taylor
*Sent:* Monday, August 1, 2016 11:31 AM
*To:* gogala.mladen@xxxxxxxxx
*Cc:* ORACLE-L
*Subject:* Re: Looking for Suggestions - 5 TB DB WHSE Backup options
Yeah Seth and I have been discussing this offline.
If a "tape device" has multiple tape writers, then you can allocate
multiple channels equal to the tape writers available in the "tape
device". So this advice in Oracle Doc does seem to be dated in regard to
Backup systems like Commvault, Netbackup etc.
Chris
On Mon, Aug 1, 2016 at 1:28 PM, Mladen Gogala <gogala.mladen@xxxxxxxxx>
wrote:
Hi
One little clarification: that only applies if the tape device is really
tape, which is increasingly rare. All backup suites that I have ever worked
with (Commvault, NetBackup, EMC NetWorker) allocate SBT_TAPE device for
writing to disk storage, too. If the device type SBT channel is allocated
for writing to disk, it is reasonable to use multiple channels.
Regards
On 08/01/2016 12:36 PM, Seth Miller wrote:
Duplexing wouldn't make much sense since that would mean copying the same
file to multiple destinations. He is probably referring to multiplexing or
parallelism. There is a difference between RMAN multiplexing and
parallelism and the term "stream" usually relates to the latter. RMAN
multiplexing allows multiple files to be read at the same time within a
single channel and defaults to 8. RMAN parallelism refers to multiple
channels writing to the same destination and is often referred to as
multiplexing on the media management side.
Oracle suggests that no more than 1 channel be used per tape device to
prevent inefficient writes and shoe-shining.
*"Do not utilize media management multiplexing (multiple channels per tape
drive). RMAN backup pieces will not be efficiently restored due to the
interleaving of pieces on the same tape volume, which may necessitate the
forward and backward movement of the tape."*
Seth Miller
On Mon, Aug 1, 2016 at 11:20 AM, Chris Taylor <
christopherdtaylor1994@xxxxxxxxx> wrote:
*Basically* - He keeps talking about Streams/Duplexing on the tape side -
where 2 threads can write to the same tape apparently? I *think* that part
is done on the Backup device outside of RMAN but I'm not sure how many
channels I should allocate within RMAN to take advantage.
For example I'm wondering (and I'll discuss with him) about the following
scenario:
Let's say we have 6 tapes and he's configured it to do 2 threads per
tape. Does that mean I can open 12 channels in RMAN? I mean, I would
assume so but I have no idea LOL
Chris
On Mon, Aug 1, 2016 at 10:44 AM, Seth Miller <sethmiller.sm@xxxxxxxxx>
wrote:
Chris,
It sounds like you are currently parallelizing the backup to disk with 4
channels and your NBU guy is suggesting parallelizing your backup directly
to tape using multiple channels and you're asking how that is going to be
different than what you are doing now. Is this correct?
Seth Miller
On Mon, Aug 1, 2016 at 9:31 AM, Chris Taylor <
christopherdtaylor1994@xxxxxxxxx> wrote:
So we ran the first FULL on Friday and it took 9 hours - so your math is
excellent.
We did run into an ancillary problem however and that was that Netbackup
was configured to use a storage pool which filled up on the Netbackup
server and caused subsequent unrelated RMAN backups to fail for other
databases. So, we're working through that. Our Netbackup guy wants us to
write directly to tape and use multiple streams and is going to help us
configure the RMAN script to take advantage of this. I'm unclear on the
relationship between RMAN threads and Netbackup streams so I'm a little in
the dark on what might need to be changed at this point. Currently, I'm
using 4 RMAN threads/channels to write the backup.
Thanks,
Chris
On Tue, Jul 12, 2016 at 5:32 PM, Dimensional DBA <
dimensional.dba@xxxxxxxxxxx> wrote:
It really depends on your netbackup infrastructure and your network
infrastructure in between your database server and your backup
infrastructure or your remote copy on disk and your backup infrastructure.
If you have a 1GB backup network from your db server to your Netbackup
Infrastrucutre with proper tuning you can normally achieve 104MB/sec
assuming proper tuning and architecture or about 432GB/hr or in your case a
9hr full backup.
If you have a 10Gb backup network from your db server to your Netbackup
Infrastructure with proper tuning you can normally achieve 780MB/sec
assuming proper tuning and architecture or about 2.88TB/hr or in your case
a 2hr full backup.
This is without any special equipment or SW licenses.
You also could
1. If you have the Oracle ASO for advanced compression then you can
also turn on compression in your RMAN backups and decrease the amount of
data that has to be transferred across the link.
2. look at performing mixed incrementals and partial fulls to
spread a full backup over multiple days.
3. Use the Incremental forever, although I wouldn’t recommend this.
*Matthew Parker*
*Chief Technologist*
*Dimensional DBA*
*425-891-7934 <425-891-7934> (cell)*
*D&B *047931344
*CAGE *7J5S7
*Dimensional.dba@xxxxxxxxxxx <Dimensional.dba@xxxxxxxxxxx>*
*View Matthew Parker's profile on LinkedIn*
<http://www.linkedin.com/pub/matthew-parker/6/51b/944/>
www.dimensionaldba.com
*From:* oracle-l-bounce@xxxxxxxxxxxxx [mailto:
oracle-l-bounce@xxxxxxxxxxxxx] *On Behalf Of *Ilmar Kerm
*Sent:* Tuesday, July 12, 2016 3:20 PM
*To:* christopherdtaylor1994@xxxxxxxxx
*Cc:* ORACLE-L
*Subject:* Re: Looking for Suggestions - 5 TB DB WHSE Backup options
Hi
We do "incremental forever" backups using incrementally updated image
copies that are located on a separate disk storage than the primary
database. Before updating the image copy, we snapshot the backup filesystem
to provide backup history. Also store a second copy of the archivelogs in
that same filesystem where the image copy is located - LOG_ARCHIVE_DEST_x
parameter, so no separate archivelog backup is needed.
Tape is ruled out in this case and you also need either a storage system
that can do NFS, snapshots, compression (optional) and thin clones (Oracle
ZFSSA, Netapp, ...) or a filesystem that can do these things (ACFS, ZFS,
..?). But all the steps needed (except storage snapshots) can be done using
RMAN and no 3rd party libraries are needed - just need to write a script to
orchestrate the steps.
Ilmar
On Tue, Jul 12, 2016 at 7:42 PM, Chris Taylor <
christopherdtaylor1994@xxxxxxxxx> wrote:
Ok, guys & gals, I'm looking for suggestions for the following challenge.
I'm very familiar with RMAN fulls w/ incrementals writing either to disk or
directly to tape. What I'm NOT familiar with is other options that I may
not know of and that's where I need your help.
Objective:
Nightly backups of 5 TB Data Warehouse that is currently being snapshotted
weekly at the SAN Layer instead of tape or disk based backups.
Hardware/OS:
IBM XIV Storage (not sure of model #)
RedHat Linux OS (5.x)
Oracle 11.2.0.4
Netbackup is tape solution
Method Options:
1. RMAN Fulls on Weekend, (either to disk or direct to tape) with nightly
incrementals. I'm leaning toward disk based backups which are then written
to tape and using parallel threads for the disk based backup to prevent
overwhelming our tape library
2. Other options?
Chris
--
Ilmar Kerm
--
Mladen Gogala
Oracle DBA
Tel: (347) 321-1217
