RE: Know anyone?
- From: Magnus Andersen <Magnus.Andersen@xxxxxxxxxxxxxxx>
- To: 'Denny Koovakattu' <oracle-l@xxxxxxxxxxxxxx>, Magnus Andersen <Magnus.Andersen@xxxxxxxxxxxxxxx>
- Date: Fri, 22 Jul 2005 15:30:36 -0400
Denny,
What kernel version are you running?
Magnus
-----Original Message-----
From: Denny Koovakattu [mailto:oracle-l@xxxxxxxxxxxxxx]
Sent: Friday, July 22, 2005 3:13 PM
To: Magnus Andersen
Cc: Oracle-L (E-mail)
Subject: RE: Know anyone?
We had similar problems, but they were caused by kswapd. If the CPU usage
is
mostly system with zero or close to zero CPU being used by the user
processes,
it indicates the kswapd bug. I don't think changing pagecache eliminates the
bug, but just moves the point at which it gets triggered. We also suspect
that
more physical memory increases the chances of encountering the bug. Hugetlb
might help for these systems. Mostly we have been hit by this bug when we
establish few Oracle connections at the same time. We have also had issues
running find or du. It's unpredictable. The last response we got from Redhat
was that it's fixed in the exact kernel version we are running ;)
Thanks,
Denny
--
Denny Koovakattu
Quoting Magnus Andersen <Magnus.Andersen@xxxxxxxxxxxxxxx>:
> Denny,
>
> I'm seeing alot of idel CPU during the spikes. I rarley see more tahn 30%
> usage at any time. Kswapd was a problem, but it got a lot better when I
> modifeid the pagecache parameters. I don't see any swapping on the system
> at all. I'm running RH EL 3 As Update 4, kernel is 2.4.21-32.0.1.ELsmp.
> I'm not using hugetbls because I ran into problems with the Oracle patch
> (see below). I'm hoping that I can figure that out over the weekend.
> Regarding the sar stats. Which of them would you like to see. I hesitate
> to post all of them since they are many...
>
> Thanks,
> Magnus
>
>
> OPATCH error:
> - Linking Oracle
> rm -f /opt/oracle/ora92/rdbms/lib/oracle
> gcc -o /opt/oracle/ora92/rdbms/lib/oracle -L/opt/oracle/ora92/rdbms/lib/
> -L/opt
> /oracle/ora92/lib/ -L/opt/oracle/ora92/lib/stubs/ -Wl,-E `test -f
> /opt/oracle/
> ora92/rdbms/lib/skgaioi.o && echo /opt/oracle/ora92/rdbms/lib/skgaioi.o`
> /opt/or
> acle/ora92/rdbms/lib/opimai.o /opt/oracle/ora92/rdbms/lib/ssoraed.o
> /opt/oracle/
> ora92/rdbms/lib/ttcsoi.o /opt/oracle/ora92/lib/nautab.o
> /opt/oracle/ora92/lib/na
> eet.o /opt/oracle/ora92/lib/naect.o /opt/oracle/ora92/lib/naedhs.o
> /opt/oracle/o
> ra92/rdbms/lib/config.o -lserver9 -lodm9 -lskgxp9 -lskgxn9 -lclient9
> -lvsn9 -l
> wtcserver9 -lcommon9 -lgeneric9 /opt/oracle/ora92/rdbms/lib/defopt.o
> -lknlopt `i
> f /usr/bin/ar tv /opt/oracle/ora92/rdbms/lib/libknlopt.a | grep xsyeolap.o
>
> /de
> v/null 2>&1 ; then echo "-loraolap9" ; fi` -lslax9 -lpls9 -lplp9
-lserver9
> -lc
> lient9 -lvsn9 -lwtcserver9 -lcommon9 -lgeneric9 -lknlopt -lslax9 -lpls9
> -lplp9
> -ljox9 -lserver9 -locijdbcst9 -lwwg9 `cat /opt/oracle/ora92/lib/ldflags`
> -l
> nsslb9 -lncrypt9 -lnsgr9 -lnzjs9 -ln9 -lnl9 -lnro9 `cat
> /opt/oracle/ora92/lib/ld
> flags` -lnsslb9 -lncrypt9 -lnsgr9 -lnzjs9 -ln9 -lnl9 -lmm -lnls9
-lcore9
> -ln
> ls9 -lcore9 -lnls9 -lxml9 -lcore9 -lunls9 -lnls9 `cat
> /opt/oracle/ora92/lib/ldfl
> ags` -lnsslb9 -lncrypt9 -lnsgr9 -lnzjs9 -ln9 -lnl9 -lnro9 `cat
> /opt/oracle/or
> a92/lib/ldflags` -lnsslb9 -lncrypt9 -lnsgr9 -lnzjs9 -ln9 -lnl9
-ltrace9
> -lnl
> s9 -lcore9 -lnls9 -lcore9 -lnls9 -lxml9 -lcore9 -lunls9 -lnls9 `if
> /usr/bin/ar
> tv /opt/oracle/ora92/rdbms/lib/libknlopt.a | grep "kxmnsd.o" > /dev/null
> 2>&1 ;
> then echo " " ; else echo "-lordsdo9"; fi` -lctxc9 -lctx9 -lzx9 -lgx9
-lctx9
> -lz
> x9 -lgx9 -lordimt9 -lnls9 -lcore9 -lnls9 -lcore9 -lnls9 -lxml9 -lcore9
> -lunls9
> -lnls9 -lsnls9 -lunls9 -lxsd9 -lnls9 -lcore9 -lnls9 -lcore9 -lnls9 -lxml9
> -lcor
> e9 -lunls9 -lnls9 `cat /opt/oracle/ora92/lib/sysliblist`
> -Wl,-rpath,/opt/oracl
> e/ora92/lib:/lib/i686:/lib:/usr/lib -lm `cat
> /opt/oracle/ora92/lib/sysliblis
> t` -ldl -lm `test -f /opt/oracle/ora92/rdbms/lib/skgaioi.o && echo
-laio`
> /opt/oracle/ora92/lib//libserver9.a(krd.o)(.text+0xcc92): In function
> `krda1fbuf
> ':
> : undefined reference to `skgmmap_ro'
> /opt/oracle/ora92/lib//libserver9.a(kcl.o)(.text+0x17a7): In function
> `kclpto':
> : undefined reference to `skgmmap_ro'
> /opt/oracle/ora92/lib//libserver9.a(kcl.o)(.text+0xa737): In function
> `kclbr':
> : undefined reference to `skgmmap_ro'
> /opt/oracle/ora92/lib//libserver9.a(kcl.o)(.text+0xaf6c): In function
> `kclswrite
> ':
> : undefined reference to `skgmmap_ro'
> /opt/oracle/ora92/lib//libserver9.a(kcl.o)(.text+0xb8f9): In function
> `kcllwr':
> : undefined reference to `skgmmap_ro'
> /opt/oracle/ora92/lib//libserver9.a(kcl.o)(.text+0x157ee): more undefined
> refere
> nces to `skgmmap_ro' follow
> collect2: ld returned 1 exit status
> make: *** [/opt/oracle/ora92/rdbms/lib/oracle] Error 1
>
> OPatch encounters the following issues during Make:
> There were problems running make commands. The errors encounted were:
> Error for command: cd /opt/oracle/ora92/rdbms/lib
> /usr/bin/make -f ins_rdbms.mk ioracle 2>&1
>
> Do you want to stop?
> Please respond Y|N >
> Y
> /usr/local/IT/oracle/9idb_linux/patches/db/OPatch/opatch.pl version:
> 1.0.0.0.51
> Copyright (c) 2001-2004 Oracle Corporation. All Rights Reserved.
>
> Make Error!
> ERROR: OPatch failed during patching, possibly due to missing files.
>
>
> -----Original Message-----
> From: Denny Koovakattu [mailto:oracle-l@xxxxxxxxxxxxxx]
> Sent: Friday, July 22, 2005 11:37 AM
> To: Magnus.Andersen@xxxxxxxxxxxxxxx
> Cc: Oracle-L (E-mail)
> Subject: Re: Know anyone?
>
>
>
> During these spikes, what is the CPU usage ? Is most of the CPU usage
> reported
> as system ? (sar output during a spike would be helpful) Is the kswapd
> process
> using a excessive CPU ? How much memory do you have on this system ?
What's
> the
> RH patch level ? Are you using hugetlb ?
>
> Thanks,
> Denny
> --
> Denny Koovakattu
>
> Quoting Magnus Andersen <Magnus.Andersen@xxxxxxxxxxxxxxx>:
>
> > Hi All,
> >
> > I am in need of a consultant that is heavy on RedHat Linux real world
> > experience for an emergency situation.
> >
> > Here is the situation: I have recently gone live with our production
> > database on RedHat Linux 3 AS and I am having a few issues. My biggest
> > problem right now is that I am seeing spikes in my load average,
resulting
> > in my system slowing down, throughout the day as soon as I have 6 - 7 or
> > more two tier connections to the database.
> >
> > Hardware: HP DL 580, 4 3 GHZ CPUs with 512k L1 Cache and 4 MB L2 Cache,
> HDDs
> > is in RAID 10 config, all Ultra 320 36 GB 15000RPM.
> >
> > Thanks in advance,
> >
> > Magnus Andersen
> > Systems Administrator / Oracle DBA
> > Walker & Associates, Inc.
>
>
> ----------------------------------------------------------------
> This message was sent using IMP, the Internet Messaging Program.
>
----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.
--
//www.freelists.org/webpage/oracle-l
Other related posts:
