Lots of good discussion here. I've done rather large numbers of
servers with cron and made it work nicely. I did use subversion for
config mgmt and ansible for automation, found these to work well for a
simple lightweight solution. Had a framework that automatically kept
a local copy of all scripts on the servers in sync with the central
subversion repo.
Scheduling tools and OEM have some nice functionality but they also
bring a lot of new complexity. Now you have to patch them, maintain
the servers they run on, etc. But if you've already got them, by all
means take advantage.
However I would actually agree that the best solution is to use the
scheduler built into your backup software. I used cronjobs widely in
an environment where backups were RMAN-based with DISK not SBT. My
biggest issue with scheduling was managing how many backup jobs launch
concurrently - even after I built in lockfile logic to prevent
multiple backups jobs from stepping on each other, there still was no
simple way to code awareness of jobs on other servers. Job management
software can do this. But most backup software can do even more: it
knows more than just whether jobs are still running; it knows things
like how many RMAN channels are open to the backup device and can do
things like throttle network throughput if needed.
I don't know about Netbackup specifically so maybe there are some
problems with their job management system. But my knee-jerk reaction
is that it'd be worth learning your backup product well and leveraging
it as much as you can.
-J
On Mon, Apr 13, 2015 at 8:16 AM, MARK BRINSMEAD
<mark.brinsmead@xxxxxxxxx> wrote:
Excellent suggestions.
I never thought about things like "puppet" and "chef", mainly because I have
(until recently) usually been restricted to working with what I find in the
customer's environment, and these are tools I pretty much never find -- or
they are never made available to me. Sysadmins should probably do the same
due diligence on these tools as any other, but their widespread use suggests
that they are probably pretty safe. (Oddly enough, sysadmins often seem to
be less skeptical of the tools *they* use -- I bet there will be no
objections here.)
Don't underestimate the value of monitoring in all of this. For example,
its great to schedule backups -- by whatever means you like -- but it is
dangerous if you start to assume that the scheduled backups are actually
running. (And with 30 or more database servers, such assumptions are very
tempting.) It can be terribly valuable to have a monitoring test in place
that verifies that backups are run on schedule (or, at the very least, any
given database is backed up once per XXXXX) and being alerted when this is
not the case. This way, if your scheduling mechanism does fail (or is
misused or misconfigured) you can learn about it before the damage is
irreparable.
