[haiku] Re: Service downtime announcement
- From: Oliver Tappe <zooey@xxxxxxxxxxxxxxx>
- To: haiku@xxxxxxxxxxxxx
- Date: Sat, 27 Apr 2013 01:44:41 +0200
Hi John,
On 2013-04-27 at 01:17:42 [+0200], John Scipione <jscipione@xxxxxxxxx> wrote:
[ ... ]
>
> Thanks for the update, the above sounds like a decent explanation of
> the problem analysis and solution.
>
> The critical factor here seems to be monitoring. It sounds like we got
> lucky that you noticed there was a problem in time, although I assume
> that if worse-came-to-worst we would have been able to restore from a
> backup.
Yes.
> Still, it would be nice to setup some sort of email notification on
> the server such as Nagios that could warn us (you?) when there is a
> problem in the future. At my workplace we use Nagios to monitor
> servers and it emails us when a disk is failing, if we are low on hard
> drive space or low on memory, or if the network become saturated. I
> assume that the server is running some sort of Linux (Suse IIRC) so
> Nagios should be a snap to setup, I can lend my expertise here if need
> be.
We are using something similar to Nagios, actually: collectd. That's where we
get the info from that's shown on the server-stats page. Additionally, smartd
would send out mails when it encounters disk problems. The problem this time
was just that smartd wasn't running anymore. The latter has been fixed by
telling systemd to restart it automatically, so next time around we should
get warned as soon as the disks themselves notice something fishy.
> We can set it up to email a @haiku-os.org email account such as
> monitoring@xxxxxxxxxxxx and then we can forward that address to you or
> setup a Google account or something. The point of using a
> @haiku-os.org email address is just so that we can easily forward the
> mails to multiple recipients and if you don't want to get the emails
> anymore we can just stop forwarding them.
We are already sending out those kind of mails to the mailing list
haiku-sysadmin at freelists.org. If you're interested in the admin-related
work, please subscribe there (or just look at the archives) and you'll get a
better idea about what's going on.
cheers,
Oliver
Other related posts:
