RAC Infiniband Questions

  • From: "Carlson,Todd" <Todd.Carlson@xxxxxxx>
  • To: "oracle-l@xxxxxxxxxxxxx" <oracle-l@xxxxxxxxxxxxx>
  • Date: Tue, 31 Aug 2010 15:05:37 -0500

Hey Guys,

We are building out our RAC environment on 11.2.0.1.2 with RHEL 5.5 on Sun 
x4270 servers. We are running into problems trying to bind our Infiniband 
connections together. In SAND & DEV, we have 2 node clusters. Each node has 2 
HCA's (Sun Dual Port 40Gb/sec 4x Infiniband QDR Host Channel Adapter), 1 & 2 
with 2 ports, A & B. We have 2 Sun 36 port Infiniband switches. What we are 
trying to do is bind 1.A with 2.A and bind 1.B with 2.B. However, there is very 
little documentation from Sun/Oracle or RedHat on how to do this and all of our 
attempts have failed. So, right now these environments are using one connection 
between the nodes using UDP.

We are currently building out TEST and we need to have this working by the 10th 
of Sept. We will then rebuild DEV & SAND to have the binding working correctly. 
So, I am hoping/praying that you can guide us here on how we go about 
configuring the binding of the IB ports. Is there some documentation that you 
would send me? We have read "RAC Support for RDS Over Infiniband [ID 751343.1]" 
and are essentially stuck at step #2.

On a Similar vein, once we get the channels bound, we would then have 2 
private, bound interfaces to use. In the cluster install (step 6 of 16), would 
we select both of them or just one?

Thanks for your help here, I really appreciate it!

Todd Carlson
Manager - DBA/ERP & EUC Teams
World Wide Technology
(314) 301-2788

Other related posts: