Re: Monitoring emails

  • From: Jeremy Schneider <jeremy.schneider@xxxxxxxxxxxxxx>
  • To: mshah@xxxxxxxxxxxxxxx
  • Date: Fri, 28 Aug 2015 16:03:25 -0400

i head up much of this in our dba group - and i think we're pretty
brutal when it comes to trimming down the pages that actually come out
to our DBA phones/pagers. we've had weeks go by that we don't get a
single page and then i'm relieved when i get a page and see that
everything is still working fine. :) this is also because we have a
great team and a solid infrastructure with infrequent major problems.

my DBA team is not global, so we generally work north american
business hours. when my phone beeps, i usually go look at it even if
i'm in the middle of dinner with the kids. i value my own evenings
and weekends - and i value the time and attention of other DBAs that i
work with. so we don't want our phones to beep for anything that
could have waited until the next morning when someone gets the office
and checks their email.

obviously we get paged if an important system is unavailable. we do
have some "non-production" systems which would need off-hours
attention if they are unavailable - of course this is really worked
out with the business. but we've ruthlessly trimmed down the noise and
our management goes to bat for us when everybody wants their stuff to
be critical. honestly, over the past few years, i can't think of many
issues we had where the business really needed a DBA to interrupt
their dinner or weekend to look at it immediately. there were a few,
but not many. just today i added a custom OEM metric on percentage of
processes used, because we became aware of an issue where something
could exhaust the processes on a database -- and we will now get a
page if process usage goes above a certain threshold. but we've
already taken so other steps to address the problem and i don't expect
many pages - if any.

now that was all just about pages. going back to the original
question about automated emails, it's another subject entirely. i like
to get emails and i have lots of server-side filters that move them
into folders that my email client doesn't even look at until i click
there. our SAs don't trim down their alerts like we do - so i get a
decent amount of traffic from their monitoring system. but i like
that. the key here is that it's all informational, and on the DBA team
we don't expect ourselves to read them - just what we're interested
in. people can setup filters to get rid of stuff they don't care
about. i don't usually check my email when i'm not at work, so it
doesn't bother me to get extra noise emails.

so far today i've got 4 pages (which also come as emails), 32
backup-related emails (there was a minor issue) and 33 miscellaneous
emails from monitoring systems that i actually watch - that is, i look
in the email folder occasionally and skim the subject lines and mark
them as read. of course i would give attention to anything that
really needs it, but all the critical stuff comes to our phones
anyway.

so far today i've got 192 "junk" emails from various other monitoring
systems which i don't watch at all - on rare occasions i'll dig into
one of those folders to look for something specific but otherwise i
completely ignore it.

-Jeremy

--
http://about.me/jeremy_schneider


On Fri, Aug 28, 2015 at 3:22 PM, Mayen Shah <mshah@xxxxxxxxxxxxxxx> wrote:

Your main goal should be to identify an act upon critical issues in your
environment. Of course there will be informational alerts/emails.



Imagine few hundred alerts (minimum 1 minute per alert) * n number of DBAs.
Is it really productive? And among all the noise likelihood of missing
critical alerts are very high. One can argue that he/she ignore or do not
act upon x% of alerts. I am of the opinion that if you ignore any alert, it
is not worth alerting on.



I have worked in environment where we will categorize alerts into
informative, warning, critical and emergency. Setup rules so emails are
organized and emergency and critical alerts are not missed.



Thanks

Mayen



From: oracle-l-bounce@xxxxxxxxxxxxx [mailto:oracle-l-bounce@xxxxxxxxxxxxx]
On Behalf Of Alfredo Abate
Sent: Friday, August 28, 2015 3:02 PM
To: veeeraman@xxxxxxxxx
Cc: ORACLE-L
Subject: Re: Monitoring emails



Ram,



The important question is how many are Critical alerts vs Warning alerts? I
would think if you are getting 100s of Ciritcal alerts there is a problem.
:) I can see getting Warning alerts frequently but perhaps you filter those
to go to different folder, etc that can be reviewed a few times per day.



For us we get Warning alerts throughout the day (maybe 25 - 100) and
Critical alerts very few if any per day. It all depends what is important
to you and the team managing the databases. This is where trying to find a
good balance between being proactive towards preventing something and
general "noise" can become an art as much as it is a science.





Alfredo



On Fri, Aug 28, 2015 at 1:10 PM, Ram Raman <veeeraman@xxxxxxxxx> wrote:

List,



How many automated emails do listers get from all the databases that are
being monitored on a daily basis?



We get a few hundred emails a day (<100 DBs), but some new members here feel
that is too many and want us to cut down on that. I personally feel that
most of the messages are relevant to us.



Thanks

Ram.

--






--
//www.freelists.org/webpage/oracle-l


Other related posts: