Re: new database server build - sanity check
- From: De DBA <dedba@xxxxxxxxxx>
- To: jifjif@xxxxxxxxx
- Date: Fri, 25 Jan 2013 11:43:45 +1000
Hi Jeff,
Having just completed a similar project, I would suggest if you're not bound to
Solaris, go Linux instead. Otherwise, avoid Solaris 11 and stick to Solaris 10
if possible. From a sysadmin and user perspective, Solaris 11 is a completely
new operating system that only superficially resembles earlier Solaris
versions. I worked with it for the last 10 months, both as the sysadmin and
DBA, and I honestly cannot recommend it.
Amongst the features that made Solaris 11 an unfortunate choice were:
- Solaris 11 local zones are integrated into the global operating system, much
like a chroot jail and completely unlike a VM. This means that when the global
zone is updated, all local zones are updated at the same time. This negates the
possibility to apply security patches to the test zones before rolling them out
into production.
- The new IPS package installation system is next to unusable, not to mention
badly documented and counter-intuitive.
- All system management is done via completely new tools and stored in a
Microsoft-like registry system, and badly documented. There is a lot to find
out, mostly via the Solaris Express developers forums. Some (but not all)
traditional configuration files are ignored, or even regenerated from the
registry.
- The familiar "recommended patch sets" no longer exist, all patching must be
done online from ( a downloaded copy of ) the Oracle support package repository
which requires either a direct internet connection or a dedicated server to
host a copy of the entire repository.
- Solaris 11 local zones are bare minimum installations and require a list of
other packages to be installed before they are usable. By default no user
interface is provided. Try installing gnome by hand.. :(
- Resource monitoring within the zone is confusing, to say the least, as zone
capped resources (CPU, memory) and system wide resources (swap, load) are
reported by tools in the zone. For instance, given total virtual memory of
160G, a capped memory allocation for the zone of 72G, monitoring inside the
zone may show total memory as the capped 72G, but available virtual memory as
160G! Trying to allocate beyond 72G fails due to the resource cap. Similarly
the load (runqueue), which even inside a zone is the global figure.
Only Oracle 11.2 from 11.2.0.3 upward is supported on Solaris 11.
If you must use Solaris 11 zones, be sure to use exclusive network interfaces.
When using shared interfaces, the routing table of the global zone is pushed
into the local zones, because the TCP stack is shared. This can lead to
difficult to debug connection failures if the local zones cannot see all
interfaces that the global zone can see. For instance, we had 6 NICs installed,
two of which had gigabit network and were to be used by the production zones.
The other NICs were connected to the same subnets, but via slower switches. The
production zones would intermittently loose connectivity because the global
zone would shift the default route to one of the NICs that were invisible to
the production zones.
The solution is to create a VNIC on the physical NIC or aggregate that you want
the zone to use and assign that VNIC as exclusive NIC to the zone. This works
also for Solaris10 branded zones in a Solaris 11 global zone.
We decided against using ZFS for any data systems, as ZFS (even more than VxFS)
is a first and foremost a volume manager, more so than a file system. It
duplicates functionality found in the Oracle instance (e.g. block caching,
predictive read-ahead) as well as the SAN (raid, snapshots, predictive
read-ahaed, write-behind). It is also wasteful, as it needs ~25% of free disk
space for internal workings, compared with only 1-5% for UFS. "Tuning" ZFS for
Oracle database involves disabling as good as possible all volume managing and
caching capabilities of ZFS, which makes implementing it rather a frustrating
and ultimately futile exercise.
Instead, we used ASM for the database and FRA, and UFS for RMAN backup sets.
UFS was chosen to avoid contention problems with RMAN's write sizes and the ZFS
preferred block size.
For the root partition, you have no choice: Solaris 11 must be installed on a
ZFS partition.
As always, your mileage may vary ;)
Cheers,
Tony
On 25/01/13 10:14, ~Jeff~ wrote:
> Hi all
> Your opinions please!
>
> We are embarking on a new database server build - the sysadmin is keen on
> Solaris 11.1 and ZFS on the T4-1 server. We will use zones for environment
> segregation and license mitigation.
>
> While I've used sol9 and 10 without absolutely no issues before, using such
> a new OS version with a relatively non-mainstream filesystem seems ...
> adventurous.
>
> Am I being too apprehensive, or is this a reasonable concern ? I'm
> interested in any real-life experience, rather than the marketing bullet
> points. Discuss. :)
>
> thanks in advance -
> Jeff
>
>
> --
> //www.freelists.org/webpage/oracle-l
>
>
>
--
//www.freelists.org/webpage/oracle-l
Other related posts:
