Re: SCAN lookup request goes to old DNS server

  • From: Stalin Subbiah <stalinsk@xxxxxxxxx>
  • To: Mark Bobak <Mark.Bobak@xxxxxxxxxxxx>
  • Date: Wed, 19 Mar 2014 19:06:09 -0700

Apparently we ended up restarting cluster stack one node at a time to clear up 
DNS request going to old DNS server. Very strange. Oracle support also 
recommended the same as they couldn't find anything wrong with the setup we had.

On Mar 11, 2014, at 4:22 PM, Stalin Subbiah <stalinsk@xxxxxxxxx> wrote:

> Thanks Mark.
> 
> nscd is not being used in this environment. That's seems to be the first one 
> we have checked.
> 
> Sorry i should have mentioned O/S in the first place. It is RHEL 5.9
> 
> We are looking to restart Application server, an IBM WAS container, to see if 
> that helps. Will keep posted.
> 
> Stalin
> 
> On Mar 11, 2014, at 4:14 PM, Mark Bobak <Mark.Bobak@xxxxxxxxxxxx> wrote:
> 
>> I wonder if nscd caches the name server IP?
>> 
>> Try:
>> $ service nscd stop
>> 
>> (You didn’t mention O/S or O/S version.  That will work for RHEL6/OL6)
>> 
>> -Mark
>> 
>> From: Stalin Subbiah <stalinsk@xxxxxxxxx>
>> Reply-To: "stalinsk@xxxxxxxxx" <stalinsk@xxxxxxxxx>
>> Date: Tuesday, March 11, 2014 at 5:47 PM
>> To: "<oracle-l@xxxxxxxxxxxxx>" <oracle-l@xxxxxxxxxxxxx>
>> Subject: SCAN lookup request goes to old DNS server
>> 
>> All,
>> 
>> We recently made some changes to names server which hosts SCAN IPs for a RAC 
>> cluster. There is no change in domain name, SCAN Ips or anything, it's just 
>> we added another secondary DNS server and added it's IP (10.124.x.x) as the 
>> first names server in resolve.conf in all DB nodes, so that transition 
>> becomes seamless when we phase out older names server (10.119.x.x).
>> 
>> DB Node --
>> -bash-3.2$ more /etc/resolv.conf
>> search domain.com
>> nameserver 10.124.x.x
>> nameserver 10.119.x.x
>> 
>> -bash-3.2$ nslookup dbcluster-scan.domain.com
>> Server: 10.124.x.x
>> Address: 10.124.x.x#x
>> 
>> Name: dbcluster-scan.domain.com
>> Address: 10.119.x.x
>> Name: dbcluster-scan.domain.com
>> Address: 10.119.x.x
>> Name: dbcluster-scan.domain.com
>> Address: 10.119.x.x
>> 
>> As you can see lookup for SCAN in OS seems to pick the right names server 
>> for the requests, however network team is seeing scan lookup traffic to 
>> 10.119.x.x names server from db node(s). We had 10.119.x.x as the first 
>> entry in resolve.conf before and that's the names server we are phasing it 
>> out.
>> 
>> Does anyone know if clusterware is caching names server somewhere? I didn't 
>> expect this to be case at first but saw this behavior after making this 
>> change to the cluster. I also tried dumping OCR but didn't see any info 
>> there.
>> 
>> Thanks,
>> Stalin
>> 
>> 
> 

Other related posts: