Re: High "global cache blocks lost" statistics on one RAC node
- From: Anand Rao <panandrao@xxxxxxxxx>
- To: racdba@xxxxxxxxxxxxx
- Date: Fri, 9 Sep 2005 09:50:24 +1000
Hi,
I believe you can do that by configuring and letting the cluster
manager know of only the VIP (used for RAC traffic). This means, even
though you could have other private IPs configured for other purposes,
they will not be under the clusterware management framework. The
Public IP may or may not be under cluster management, because that
depends on your application availability requirements.
Oracle always talks to the clusterware internally to figure the
correct private IP. If there are multiple private IPs, Oracle
considers the first one returned by the clusterware. Sometimes, people
override this using CLUSTER_INTERCONNECTS parameter which may be
required (occasionally) but restricts the HA features configured with
the card.
All these things can be easily configured at the cluster software
level. The VIP configuration, failover and load balancing feature are
all done at the clusterware level. So, the speak to the OS/cluster
vendor and they will clear these for you. You need to test your VIP
stuff carefully though, before going into production.
cheers
anand
On 09/09/05, Hameed, Amir <Amir.Hameed@xxxxxxxxx> wrote:
> Hi Anand,
> Thank you for all the useful information. One more question though; if
> there are multiple Ips available on the server:
> 1) A public IP
> 2)A VIP defined for UDP (bundled with multiple Ips just like KG and you
> have suggested)
> 3) A private IP between the backend and the middle-tier servers
>
> How would I tell Oracle to use the VIP defined for UDP and not any other
> IP? With LLT it is easy (even though you do not define Ips) because
> based upon the interfaces defined in the config file, LMX multiplexes
> the traffic.
>
> Thanks
> Amir
> -----Original Message-----
> From: racdba-bounce@xxxxxxxxxxxxx [mailto:racdba-bounce@xxxxxxxxxxxxx]
> On Behalf Of Anand Rao
> Sent: Thursday, September 08, 2005 7:26 PM
> To: racdba@xxxxxxxxxxxxx
> Subject: Re: High "global cache blocks lost" statistics on one RAC node
>
> Hi Amir,
>
> Yes, UDP does not have the inherent intelligence to recognise multiple
> network paths etc. For that matter, i don't think any protocol is
> built with that intelligence. That is something modeled at the higher
> level in the configuration/architecture layer in the cluster.
>
> You can use multiple Gigabit Ethernet cards and configure them for
> failover or load balancing. You will need to create a 'virtual ip
> address' (nothing to do with Oracle 10g stuff) which is exposed to
> Oracle. It is better you get this sorted out with the Hardware/Cluster
> vendor and perform a full test.
>
> Do not use CLUSTER_INTERCONNECTS parameter unless Oracle Support asks
> you to do so or if you have certain special requirements. Open a TAR
> and ask Oracle about it. Yes, even if you have configured private
> interconnect failover, setting this parameter disables the capability
> of Oracle to recognise the same and RAC will cease to work.
>
> On Sun, you need to change the valued i suggested earlier. On AIX,
> Tru64 and HP the parameter names are different but conceptually you
> need to change the receive and send buffers as Sudhi has mentioned.
>
> Regards
> anand
>
> On 09/09/05, K Gopalakrishnan <kaygopal@xxxxxxxxx> wrote:
> > Amir:
> >
> > That is where we use various techniques like IP MultiPathing or NIC
> > teaming etc. Basically we club bunch of network cards to a single
> > logical one and the load balancing is done at card level which is
> > transparent to TCP/UDP layer.
> >
> > Regards,
> > Gopal
> >
> >
> > On 9/8/05, Hameed, Amir <Amir.Hameed@xxxxxxxxx> wrote:
> > > I guess I just can't let go of this topic. When using LLT, the LMX
> > > module takes care of the load-balancing of packets across the
> available
> > > interconnects. How is it done in UDP and how does UDP recognized
> > > multiple interconnects. Would I need to use the
> "cluster_interconects"
> > > parameter with UDP? If yes then I believe it has its own problems in
> > > Solaris as stated by Oracle:
> > >
> > >
> > >
> ------------------------------------------------------------------------
> > > --------
> > > Note:
> > > When you set CLUSTER_INTERCONNECTS in Solaris configurations, the
> > > interconnect High Availability features are not available. In other
> > > words, an interconnect failure that is normally unnoticeable would
> > > instead cause an Oracle cluster failure.
> > >
> > >
> ------------------------------------------------------------------------
> > > --------
> > >
> > > Thanks
> > > Amir
> > > -----Original Message-----
> > > From: racdba-bounce@xxxxxxxxxxxxx
> [mailto:racdba-bounce@xxxxxxxxxxxxx]
> > > On Behalf Of Ravi_Kulkarni@xxxxxxxx
> > > Sent: Thursday, September 08, 2005 1:11 PM
> > > To: racdba@xxxxxxxxxxxxx
> > > Subject: RE: High "global cache blocks lost" statistics on one RAC
> node
> > >
> > > Following doc, a little-dated, covers for all Unix Flavors and
> should
> > > have what you are looking for
> > >
> > >
> > > Doc ID: Note:181489.1
> > > Subject: Tuning Inter-Instance Performance in RAC and OPS
> > > Type: BULLETIN
> > > Status: PUBLISHED
> > > Content Type: TEXT/X-HTML
> > > Creation Date: 25-MAR-2002
> > > Last Revision Date: 18-JAN-2004
> > >
> > >
> > > Thanks,
> > > Ravi.
> > >
> > >
> > > -----Original Message-----
> > > From: racdba-bounce@xxxxxxxxxxxxx
> [mailto:racdba-bounce@xxxxxxxxxxxxx]
> > > On Behalf Of Hameed, Amir
> > > Sent: Thursday, September 08, 2005 10:56 AM
> > > To: racdba@xxxxxxxxxxxxx
> > > Subject: RE: High "global cache blocks lost" statistics on one RAC
> node
> > >
> > > Thank you KG.
> > > This is really useful information. It will help me save time by not
> > > trying to spend time to resolve something that is not stable.
> > > Thanks for everyone's input and comments on this subject. I will
> change
> > > the protocol from LLT to UDP and then run the same test again and
> see if
> > > the stats of lost blocks changes.
> > > Does anyone have any guidelines on what to tune in UDP?
> > >
> > > Thanks
> > > Amir
> > > -----Original Message-----
> > > From: racdba-bounce@xxxxxxxxxxxxx
> [mailto:racdba-bounce@xxxxxxxxxxxxx]
> > > On Behalf Of K Gopalakrishnan
> > > Sent: Thursday, September 08, 2005 11:46 AM
> > > To: racdba@xxxxxxxxxxxxx
> > > Subject: Re: High "global cache blocks lost" statistics on one RAC
> node
> > >
> > > Amir:
> > >
> > > Not just Raj, Most of my customers (all are biggest banks/telcos)
> run
> > > their critical apps on UDP. Few guys tried Veritas LLT, Sun RSM and
> > > miserably failed. Anand was also part of that experiment. Sun RSM
> is
> > > totally unstable and we could't find a single reference customer
> from
> > > Sun who is on production with RSM on 15K.
> > >
> > > About LLT, one of the largest bank in India (they are the biggest
> OLTP
> > > database in the world) tried thier best and now they have gone back
> to
> > > UDP. I also have similar experience with HMP on Hyperfabric on one
> of
> > > the benchmarks and finally we have gone back to UDP.
> > >
> > > KG
> > >
> > > On 9/8/05, Hameed, Amir <Amir.Hameed@xxxxxxxxx> wrote:
> > > > It is good to know that there are folks who are running their
> mission
> > > > critical apps on UDP.
> > > >
> > > > ________________________________
> > > > From: racdba-bounce@xxxxxxxxxxxxx
> [mailto:racdba-bounce@xxxxxxxxxxxxx]
> > > On
> > > > Behalf Of rjamya
> > > > Sent: Thursday, September 08, 2005 7:12 AM
> > > > To: racdba@xxxxxxxxxxxxx
> > > > Subject: Re: High "global cache blocks lost" statistics on one RAC
> > > node
> > > >
> > > >
> > > > We too run mission critical databases (where downtime costs us
> > > thousands)
> > > > and they all use UDP and have always been reliable.
> > > >
> > > > YMMV
> > > > Raj
> > > >
> > > > On 9/7/05, Hameed, Amir <Amir.Hameed@xxxxxxxxx> wrote:
> > > > > This is a mission critical application and processes millions of
> $$
> > > of
> > > > > revenue and I am a little hesitant in going with UDP because of
> its
> > > > > not-so-reliable reputation. I may escalate this with VOS
> tomorrow
> > > and
> > > > > see what they have to say.
> > > > >
> > > > > Amir
> > > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > Best Regards,
> > > K Gopalakrishnan
> > > Co-Author: Oracle Wait Interface, Oracle Press 2004
> > > http://www.amazon.com/exec/obidos/tg/detail/-/007222729X/
> > >
> > >
> > >
> >
> >
> > --
> > Best Regards,
> > K Gopalakrishnan
> > Co-Author: Oracle Wait Interface, Oracle Press 2004
> > http://www.amazon.com/exec/obidos/tg/detail/-/007222729X/
> >
> >
>
>
>
Other related posts:
