stuck dll as in dll is locked by a previous backup process.
%systemroot%\system32 does not have the orasbt.dll. The only place where I
found it was in c:\program files\commvault\simpana\base.
Have a question open with commvault if post backup process can be refined to
ensure locks to dlls are released.
Thanks
On Jan 11, 2016, at 3:02 PM, Mladen Gogala <gogala.mladen@xxxxxxxxx> wrote:--
Hi MJ,
What is a "stuck DLL"? Was there a recent change of DLL files? The allocate
channel command usually goes like this:
allocate channel c0 device type SBT;
The problems can happen if there is orasbt.dll in the path, because Windows
will use the first orasbt.dll which it can find. Simpana expects to find the
library in C:\Program Files\\Simpana\Base. If it encounters the same library
in a wrong directory, like C:\Windows\System32, the whole thing will croak.
Regards
On 01/11/2016 03:12 PM, MJ Mody wrote:
Actual error CommVault engineer pulled from the logs is 'connection with the
OraSbtThread failed, possibly due to a stuck DLL'
I think this could be the smoking gun
On Jan 9, 2016, at 4:46 PM, Mladen Gogala <gogala.mladen@xxxxxxxxx> wrote:Was there anything in the ClOraAgent.log? Did you open a case with
On 01/08/2016 11:21 PM, MJ Mody wrote:
Suspected resource contention of sorts. Follow-up with backup vendor,
CommVault, yielded a file lock with cv/oracle library (i.e. oci.dll) from
an earlier backup. Will need to review the Oracle shadow copy interface.
Still a mystery as to why nothing was written to alert log.
CommVault support? Do you have any other backup system on the machine?
Simpana client library is called ORASBT.DLL and usually resides in
C:\Program Files\Commvault\Simpana\Base. If there was another backup
software on the machine, it can put ORASBT.DLL either in %ORACLE_HOME%\lib
or in C:\Windows\system32. Please, search the entire machine for the
ORASBT.DLL libraries, since the library clash can be quite serious problem.
The name of the library is mandated by Oracle Corp. and all backup suites
(Commvault, TSM, NetBackup, Avamar) must use the same name.
Regards
Cheers
MJ
On Jan 8, 2016, at 9:26 PM, De DBA <dedba@xxxxxxxxxx> wrote:--
Usually I'd suspect an out-of-memory situation. The scenario reminds me
of a case I had where the backup software (Tivoli, I think), would use
Windows shadow copy to execute an RMAN backup. The shadow copy process
never exited and after a (large) number of backups the Oracle process
memory was exhausted and the database hang. From memory that was also
W2K8R2/11.2.0.3. I believe there was a bug in the Oracle shadow copy
interface.
You did check the Windows Event Logs (Application and System) I presume?
Hth,
Tony
On 09/01/16 11:03, MJ Mody wrote:
Nothing like ringing in a new year than with a hung database.
In this case, running 11.2.0.3 EE with ASO/TDE on W2K8-R2 with last CPU.
From what is known, external backups (tape writes) were taking place in
backup software, both rman and data pump (the .dmp files were created
early in the evening). Additionally, nightly EOD processing was also
taking place.
Performed a server reboot to bring the database online. Point of concern
is from 9pm onwards, until the database came back online (~1:30am),
there is no entries in the alert log.
Feel free to share if anyone has come across this or have some words of
wisdom for troubleshooting and remediation.
Thank you in advance.
Best
MJ--
//www.freelists.org/webpage/oracle-l
//www.freelists.org/webpage/oracle-l
--
Mladen Gogala
Oracle DBA
http://mgogala.freehostia.com
--
//www.freelists.org/webpage/oracle-l
--
Mladen Gogala
Oracle DBA
http://mgogala.freehostia.com
