Re: 11.2.0.3 Grid causing high load on servers
- From: "Shastry(DBA)" <shastry17@xxxxxxxxx>
- To: oracle-l <oracle-l@xxxxxxxxxxxxx>, ora-apps-dba@xxxxxxxxxxxxx
- Date: Sat, 11 Aug 2012 12:32:45 +0530
Apologies on earlier blank email which was sent accidentally due to key
board short-cuts:
Hi Gurus,
Recently, 2 months back we upgraded the 11g CRS from 11.2.0.1
to 11.2.0.3 version. It was running absolutely fine and couple of days back
the load on linux box went to 600+ from 25 avg load causing the server to
hung. We are working with Oracle SR support too but not recieved any inputs
so far. Below are some of the observations by SA. please suggest
Oracle DB - 11.2.0.3
Oracle CRS/Grid - 11.2.0.3
Platform - Linux
oracle 32659 1 0 Aug08 ? 00:00:01
/oracle/product/grid_home/jdk/bin/java -Djava.net.preferIPv4Stack=true
?classpath
/oracle/product/grid_home/jdk/lib/rt.jar:/oracle/product/grid_home/jlib/cvu.jar:/oracle/product/grid_home/jlib/srvm.jar:/oracle/product/grid_home/oui/jlib/OraInstaller.jar:/oracle/product/grid_home/oui/jlib/OraPrereq.jar:/oracle/product/grid_home/oui/jlib/prov_fixup.jar:/oracle/product/grid_home/oui/jlib/xmlparserv2.jar:/oracle/product/grid_home/oui/jlib/share.jar:/oracle/product/grid_home/oui/jlib/orai18nmapping.jar:/oracle/product/grid_home/jlib/srvmhas.jar:/oracle/product/grid_home/jdbc/lib/ojdbc5.jar:/oracle/product/grid_home/jlib/netcfg.jar
-DCV_DESTLOC=/tmp -DCV_HOME=/oracle/product/grid_home
oracle.ops.verification.client.CluvfyDriver comp crs -display_status
[8833@testdb] [main] [ 2012-08-09 05:53:15.233 PDT ]
[RuntimeExec.runCommand:75] Calling Runtime.exec() with the command
[8833@testdb] [main] [ 2012-08-09 05:53:15.233 PDT ]
[RuntimeExec.runCommand:77] /bin/rm
[8833@testdb] [main] [ 2012-08-09 05:53:15.233 PDT ]
[RuntimeExec.runCommand:77] -f
[8833@testdb] [main] [ 2012-08-09 05:53:15.234 PDT ]
[RuntimeExec.runCommand:77] /tmp/localnode.olr.loc
[8833@testdb] [Thread-21] [ 2012-08-09 05:53:15.237 PDT ]
[StreamReader.run:61] In StreamReader.run
[8833@testdb] [main] [ 2012-08-09 05:53:15.237 PDT ]
[RuntimeExec.runCommand:142] runCommand: Waiting for the process
[8833@testdb] [Thread-20] [ 2012-08-09 05:53:15.237 PDT ]
[StreamReader.run:61] In StreamReader.run
[7329@testdb] [main] [ 2012-08-09 05:53:21.340 PDT ]
[RuntimeExec.runCommand:142] runCommand: Waiting for the process
[7329@testdb] [Thread-30] [ 2012-08-09 05:53:21.340 PDT ]
[StreamReader.run:61] In StreamReader.run
[7329@testdb] [Thread-31] [ 2012-08-09 05:53:21.340 PDT ]
[StreamReader.run:61] In StreamReader.run
SA suspected that the "file got deleted:
/var/tmp/.oracle/sprocr_local_conn_0_PROC"
But there are no crons or manual removal of the files in /var/tmp/.oracle
area by anyone, what else can cause this issue since we are not able to
stop and start the CRS services and reboot of all cluster nodes is fixing
it, rather we are looking for the actual cause and fix. Please share your
views and suggestions.
[root@testdb ~]# uptime
13:12:33 up 1 day, 5:23, 1 user, load average: *627.15, 624.81, 619.83*
oracle 32742 1 0 Aug09 ? 00:00:01
/oracle/product/grid_home/jdk/bin/java -Djava.net.preferIPv4Stack=tru
-classpath
/oracle/product/grid_home/jdk/lib/rt.jar:/oracle/product/grid_home/jlib/cvu.jar:/oracle/product/grid_homejlib/srvm.jar:/oracle/product/grid_home/oui/jlib/OraInstaller.jar:/oracle/product/grid_home/oui/jlib/OraPrereq.jar:/oacle/product/grid_home/oui/jlib/prov_fixup.jar:/oracle/product/grid_home/oui/jlib/xmlparserv2.jar:/oracle/product/gri_home/oui/jlib/share.jar:/oracle/product/grid_home/oui/jlib/orai18n-mapping.jar:/oracle/product/grid_home/jlib/srvmha.jar:/oracle/product/grid_home/jdbc/lib/ojdbc5.jar:/oracle/product/grid_home/jlib/netcfg.jar
-DCV_DESTLOC=/tmp -DCV_HME=/oracle/product/grid_home
oracle.ops.verification.client.CluvfyDriver comp crs -display_status
[root@testdb ~]# strace -p 32742
Process 32742 attached - interrupt to quit
futex(0x2aaaac06ad14, FUTEX_WAIT_PRIVATE, 59, NULL <unfinished ...>
Process 32742 detached
Thanks,
Shastry
On Sat, Aug 11, 2012 at 12:23 PM, Shastry(DBA) <shastry17@xxxxxxxxx> wrote:
> Hi gurus,
>
>
--
//www.freelists.org/webpage/oracle-l
Other related posts:
