RE: RAC private network IP's - 169.254.x.x

  • From: D'Hooge Freek <Freek.DHooge@xxxxxxxxx>
  • To: "cichomitiko@xxxxxxxxx" <cichomitiko@xxxxxxxxx>, goran bogdanovic <goran00@xxxxxxxxx>
  • Date: Fri, 6 Apr 2012 12:29:25 +0200


How can you assign a different vlan on network ports that are part of the same 
I would think this would break your bonding, meaning you tested on a bond that 
has only 1 working network port to begin with.

The reason why you need to use separate subnets when using HAIP with multiple 
nics is the os routing. When you have multiple ip addresses in the same subnet 
on a server, it is the routing table which determines the interface to be used 
for outgoing traffic.
If that interface fails, the routing table will not be changed and thus all 
communications break.

This is also the reason why one should not use ifdown to test network failures.
It does things to cleanly, like removing the entry for that interface from the 
routing table.

Kind regards,
Freek D'Hooge
Oracle Database Administrator
email: freek.dhooge@xxxxxxxxx
tel +32(0)3 451 23 82

-----Original Message-----
From: oracle-l-bounce@xxxxxxxxxxxxx [mailto:oracle-l-bounce@xxxxxxxxxxxxx] On 
Behalf Of Radoulov, Dimitre
Sent: vrijdag 6 april 2012 12:05
To: goran bogdanovic
Cc: krishna.setwin@xxxxxxxxx; oracle-l
Subject: Re: RAC private network IP's - 169.254.x.x

Thanks Goran!
One of our customers is using bonding for the private Interconnect 
(every node has 4 NICs - 2 in bond for the public
and 2 for the private network, GI sees bond0 and bond1).
Initially separate private vlans were used for the Interconnect and, 
during an HA test, disabling one of the NICs
in bond on one of the nodes completely broke the communication. After 
that we configured all interfaces used
for the Interconnect in the same pvlan and after rebooting the nodes, 
all worked well.

This issue is of course different as the customer is using bonding and 
not GI's HAIP directly
for Interconnect redundancy.


On 06/04/2012 11:24, goran bogdanovic wrote:
> pay attention on
> *
> *
> if you use more than one NIC for interconnect ...
> goran
> On Fri, Apr 6, 2012 at 10:10 AM, Radoulov, Dimitre 
> <cichomitiko@xxxxxxxxx <mailto:cichomitiko@xxxxxxxxx>> wrote:
>     Hi Krishna,
>     as already stated, the 169.254.*.* is used by GI automatically:
>     11gR2 Grid Infrastructure Redundant Interconnect and
>     ora.cluster_interconnect.haip [ID 1210883.1]
>     Grid automatically picks free link local addresses from reserved
>     169.254.*.* subnet for HAIP. According to RFC-3927, link local subnet
>     169.254.*.* should not be used for any other purpose. With HAIP, by
>     default, interconnect traffic will be load balanced across all active
>     interconnect interfaces, and corresponding HAIP address will be failed
>     over transparently to other adapters if one fails or becomes
>     non-communicative. .
>     Dimitre
>     On 06/04/2012 09:57, D'Hooge Freek wrote:
>     > I think the <> subnet is
>     used for autoconfiguration ip addresses.
>     > Which are addresses that are self assigned after verifying that
>     no one else is using it by sending out an arp ping (for instance,
>     in case when the dhcp server is not giving out an address).
>     >
>     > You should not use them when statistically assigning an ip address.
>     >
>     > Oracle is using ip addresses in this range for the private
>     network as part of the high available ip address (HAIP)
>     functionality (which is also the reason multicast's must be
>     allowed between the different nodes).
>     >
>     [...]
>     > Can we use as private IP's for RAC 2
>     node
>     > cluster?
>     > (basically any 169.254.x.x )
>     >
>     > we just have 1 private NIC for each node.
>     >
>     > version: rac
>     > RHEL 5.8
>     >
>     --



Other related posts: