RE: Slow GC response for x64 bit 2008

  • From: "Taylor, Chris David" <ChrisDavid.Taylor@xxxxxxxxxxxxxxx>
  • To: "'Storey, Robert (DCSO)'" <RStorey@xxxxxxxxxxxxxxxxxx>, "'Mark W. Farnham'" <mwf@xxxxxxxx>
  • Date: Wed, 7 Dec 2011 09:30:50 -0600

If you have the Microsoft SysInternals Suite downloaded to your GC box, I think 
you can tell what options Weblogic is running with:


1.) Download Sysinternals Suite (if you don't have it)

2.) Execute PROCEXP.EXE

3.) Right click on one of the COLUMN HEADERS and choose "SELECT COLUMNS"

[cid:image001.png@01CCB4C1.A266B6B0]





4.) Choose PROCESS IMAGE tab

5.) Then CHECK the COMMAND LINE box

[cid:image002.png@01CCB4C1.A266B6B0]



6. CLICK OK

7. Now, SORT by COMMAND LINE and scroll down till you see your Weblogic Drive & 
Directory

8. You should see something like DRIVE:\MIDDLE~1\...\...\...

9. Look for Process Type Java.Exe and Command Line like EMGC_OMS1 and 
EMGC_ADMINSERVER  - These are the 2 Weblogic Domains (You may have to expand 
the COMMAND LINE column to see it)

[cid:image003.png@01CCB4C2.970739B0]

[cid:image004.png@01CCB4C2.970739B0]



10. RIGHT CLICK on each of the Processes and choose properties

[cid:image005.png@01CCB4C2.970739B0]



11. SELECT the COMMAND LINE text (shift-click with mouse at the beginning of 
the line and then press END key to quickly select entire line)

12. Copy the text

13. Paste it to an editor such as notepad etc

14. examine the command line for the debug switch I believe

15. Here's my FULL command line

F:\Oracle\MIDDLE~1\JDK160~1.5-3\bin\java  -client   -Xms256m -Xmx1024m 
-XX:MaxPermSizeQ2m -XX:CompileThreshold?00 -XX:PermSize8m 
-Dweblogic.Name=EMGC_OMS1 
-Djava.security.policy=F:\Oracle\MIDDLE~1\WLSERV~1.3\server\lib\weblogic.policy 
-Dweblogic.system.BootIdentityFile=F:\Oracle\gc_inst\user_projects\domains\GCDomain\servers\EMGC_OMS1\data\nodemanager\boot.properties
 -Dweblogic.nodemanager.ServiceEnabled=true 
-Dweblogic.security.SSL.ignoreHostnameVerification=true 
-Dweblogic.ReverseDNSAllowedúlse -DINSTANCE_HOME=F:\Oracle\gc_inst\em\EMGC_OMS1 
-DORACLE_HOME=F:\Oracle\Middleware\oms11g 
-Ddomain.home=F:\Oracle\gc_inst\user_projects\domains\GCDomain 
-Djava.awt.headless=true -Ddomain.name=GCDomain 
-Docm.repeater.home=F:\Oracle\Middleware\oms11g -Xrs  -Xverify:none  -da 
-Dplatform.home=F:\Oracle\MIDDLE~1\WLSERV~1.3 
-Dwls.home=F:\Oracle\MIDDLE~1\WLSERV~1.3\server 
-Dweblogic.home=F:\Oracle\MIDDLE~1\WLSERV~1.3\server  
-Ddomain.home=F:\Oracle\gc_inst\USER_P~1\domains\GCDomain 
-Dcommon.components.home=F:\Oracle\MIDDLE~1\ORACLE~1 -Djrf.version.1.1 
-Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.Jdk14Logger 
-Djrockit.optfile=F:\Oracle\MIDDLE~1\ORACLE~1\modules\oracle.jrf_11.1.1\jrocket_optfile.txt
 
-Doracle.domain.config.dir=F:\Oracle\gc_inst\USER_P~1\domains\GCDomain\config\FMWCON~1
 
-Doracle.server.config.dir=F:\Oracle\gc_inst\USER_P~1\domains\GCDomain\config\FMWCON~1\servers\EMGC_OMS1
 
-Doracle.security.jps.config=F:\Oracle\gc_inst\USER_P~1\domains\GCDomain\config\fmwconfig\jps-config.xml
 -Djava.protocol.handler.pkgs=oracle.mds.net.protocol  
-Digf.arisidbeans.carmlloc=F:\Oracle\gc_inst\USER_P~1\domains\GCDomain\config\FMWCON~1\carml
  
-Digf.arisidstack.home=F:\Oracle\gc_inst\USER_P~1\domains\GCDomain\config\FMWCON~1\arisidprovider
 
-Dweblogic.alternateTypesDirectory=\modules\oracle.ossoiap_11.1.1,\modules\oracle.oamprovider_11.1.1
  -Dweblogic.jdbc.remoteEnabledúlse  -Dweblogic.management.discoverúlse 
-Dweblogic.management.server=https://spobmgrd01:7101  -Dwlw.iterativeDev= 
-Dwlw.testConsole= -Dwlw.logErrorsToConsole= 
-Dweblogic.ext.dirs=F:\Oracle\MIDDLE~1\patch_wls1032\profiles\default\sysext_manifest_classpath
  weblogic.Server





16. Notice near the end are some switches:

-Dwlw.iterativeDev= -Dwlw.testConsole= -Dwlw.logErrorsToConsole


These look to be not set on mine.  I think there is an ADDITIONAL switch for 
DEBUGGING that shows up when DEBUG is enabled but I might be mistaken



Chris Taylor

Sr. Oracle DBA

Ingram Barge Company

Nashville, TN 37205



"Quality is never an accident; it is always the result of intelligent effort."

-- John Ruskin (English Writer 1819-1900)



CONFIDENTIALITY NOTICE: This e-mail and any attachments are confidential and 
may also be privileged. If you are not the named recipient, please notify the 
sender immediately and delete the contents of this message without disclosing 
the contents to anyone, using them for any purpose, or storing or copying the 
information on any medium.





-----Original Message-----
From: Storey, Robert (DCSO) [mailto:RStorey@xxxxxxxxxxxxxxxxxx]
Sent: Wednesday, December 07, 2011 9:11 AM
To: Taylor, Chris David; 'Mark W. Farnham'
Subject: RE: Slow GC response for x64 bit 2008



I would agree but I can't find a good way to test that.



Let me check the process.



-----Original Message-----

From: Taylor, Chris David [mailto:ChrisDavid.Taylor@xxxxxxxxxxxxxxx]

Sent: Wednesday, December 07, 2011 9:09 AM

To: Storey, Robert (DCSO); 'Mark W. Farnham'; 'oracle-l@xxxxxxxxxxxxx'

Subject: RE: Slow GC response for x64 bit 2008



I still think the performance issue is in the Weblogic rendering BUT:



Examine the task manager of the Windows 2008 target box (particularly the 
EMAGENT.EXE process) and run your tests while observing the task on the 
monitored host. If the emagent.exe process begins chewing up CPU then you're 
more likely on the right track.  However, if the emagent process on the 
monitored host remains mostly idle, then the problem is on the GC host itself 
(and most likely in the Weblogic 10 layer).



I would bet the Production Mode (from Development Mode) switch didn't work as 
it should have in Weblogic and it is still running with DEBUG set.



Chris Taylor

Sr. Oracle DBA

Ingram Barge Company

Nashville, TN 37205



"Quality is never an accident; it is always the result of intelligent effort."

-- John Ruskin (English Writer 1819-1900)



CONFIDENTIALITY NOTICE: This e-mail and any attachments are confidential and 
may also be privileged. If you are not the named recipient, please notify the 
sender immediately and delete the contents of this message without disclosing 
the contents to anyone, using them for any purpose, or storing or copying the 
information on any medium.



-----Original Message-----

From: oracle-l-bounce@xxxxxxxxxxxxx [mailto:oracle-l-bounce@xxxxxxxxxxxxx] On 
Behalf Of Storey, Robert (DCSO)

Sent: Wednesday, December 07, 2011 7:35 AM

To: 'Mark W. Farnham'; oracle-l@xxxxxxxxxxxxx

Subject: RE: Slow GC response for x64 bit 2008



Since I'm new to GC, I'm still not fully understanding what is happening when 
you select a Host target homepage.  Is the agent being asked to provide a huge 
amount of data all over?  Is it reading from the repository.



I definitely agree it's something wacky on the 2008 machines.



Ping was the first thing I did.  No network latency. Less than 1ms response.  
The two machines are on the same switch, so no weird routing.



If I understood the sequence of steps better for building that Target page, I 
could do some tracing.



Which tracing level would be the one to set?  One for the OMS?  OMA?  Agent on 
the box?  Since I'm not (as best I can tell) seeing an error in the process, it 
leads me to believe that it is taking the agent that long to gather and send 
the info.





-----Original Message-----

From: Mark W. Farnham [mailto:mwf@xxxxxxxx]

Sent: Wednesday, December 07, 2011 7:22 AM

To: Storey, Robert (DCSO); oracle-l@xxxxxxxxxxxxx

Subject: RE: Slow GC response for x64 bit 2008



I'd start with divide and conquer regarding where in the stack the long elapsed 
time is.



If you open a command window and use ping you should be able to do a traceroute 
to see if the network is going some stupid way to your targets.

If you examine the objects on each displayed page, can you verify you're not 
reloading wallpaper, logos, or pictures from a remote source?



Is there anything funky about the DNS and/or security on W2008 (haven't had the 
displeasure yet) that might be taking a long time to resolve?



22 seconds is approximately forever, but not failing, so I think we're looking 
for some strange latency here.



Is it possible W2008R2 is trying to refresh the whole network topology for the 
workgroup, or something like that?



Good luck.



-----Original Message-----

From: oracle-l-bounce@xxxxxxxxxxxxx [mailto:oracle-l-bounce@xxxxxxxxxxxxx]

On Behalf Of Storey, Robert (DCSO)

Sent: Tuesday, December 06, 2011 1:19 PM

To: oracle-l@xxxxxxxxxxxxx

Subject: Slow GC response for x64 bit 2008



Afternoon all,

I've been working with Oracle on an issue rendering of the target homepage in 
grid control for 3, x64bit boxes running 64bit W2008R2. But, since its Pri4, I 
should hear from them in the next 30 days or so.



I'm running GC 11 on W2008 x32. I have 22 targets.  Two of my targets are 
x32bit.  A W2003 x32 running 32bit os and my GC box that is running W2008

x32 bit OS on a x64bit hardware.  Rendering these Host target homepages takes 
about 1.2 seconds.  My other 3 boxes are all W2008R2 x64 on 64bit Dell 
Powedges.  When I click on the link for their Host homepage, its takes 22 
seconds to render.



I have looked through every log that I can find but can find no errors.  I 
installed the agent on a 64 bit workstation running Windows7 64bit and it works 
just fine.  Renders in about a second.



Once suggestion I got from another DBA was that I was in Demo mode vice 
Production on Weblogic.  I reset and rebooted that so it is now in production 
mode.



But, I can't find any reason as to why the render takes so long on these boxes.



Anyone run up against this or have a thought?  (lol  moving to linux is not an 
option).



--

//www.freelists.org/webpage/oracle-l





--

//www.freelists.org/webpage/oracle-l












--
//www.freelists.org/webpage/oracle-l


Other related posts: