You set the PATH twice
[root@uat-db-002 .ssh]# export PATH=$PATH:$ORACLE_HOME/OPatch
[root@uat-db-002 .ssh]# export PATH=$PATH:/u01/g01/grid/12.1/OPatch
Was the original $ORACLE_HOME = /u01/g01/grid/12.1/OPatch or something else
Run
Env | grep PATH
Does the path have different paths to OPatch?
Whatever is the first one in the PATH variable will be used first.
Is the version of opatchauto/opatch the same on each node and the required
version?
From: oracle-l-bounce@xxxxxxxxxxxxx <oracle-l-bounce@xxxxxxxxxxxxx> On Behalf
Of Sameer Choudhari
Sent: Thursday, October 3, 2019 8:41 PM
To: Oracle-L@xxxxxxxxxxxxx
Subject: OPatchAuto failed
Team
I am applying JULY 2019 Proactive Bunble Patch on our UAT env.
Env :
DB : 12.1.0.2.0 (2 node rac)
OS : Oracle Linux 7
I am able to patch node-1 i.e. uat-db-001 without any issue but got below error
while patching node -2 i.e. uat-db-002 .
[root@uat-db-002 .ssh]# export PATH=$PATH:$ORACLE_HOME/OPatch
[root@uat-db-002 .ssh]# export PATH=$PATH:/u01/g01/grid/12.1/OPatch
[root@uat-db-002 .ssh]# cd /u01/stage/COMBO_JVM_DBBP_JUL2019/29699255
[root@uat-db-002 29699255]# $ORACLE_HOME/OPatch/opatchauto apply
/u01/stage/COMBO_JVM_DBBP_JUL2019/29699255/29698629
OPatchauto session is initiated at Fri Oct 4 13:13:54 2019
System initialization log file is
/u01/g01/grid/12.1/cfgtoollogs/opatchautodb/systemconfig2019-10-04_01-14-00PM.log.
Session log file is
/u01/g01/grid/12.1/cfgtoollogs/opatchauto/opatchauto2019-10-04_01-14-44PM.log
The id for this session is LBTP
Remote command execution failed due to
*** Authorized uses only. All activity may be monitored and reported. ***
Can't locate Cwd.pm in @INC (@INC contains:
/u01/g01/grid/12.1/perl/lib/site_perl/5.14.1/x86_64-linux-thread-multi
/u01/g01/grid/12.1/perl/lib/site_perl/5.14.1
/u01/g01/grid/12.1/perl/lib/5.14.1/x86_64-linux-thread-multi
/u01/g01/grid/12.1/perl/lib/5.14.1 .) at
/u01/g01/grid/12.1/OPatch/auto/database/bin/RemoteHostExecutor.pl line 2.
BEGIN failed--compilation aborted at
/u01/g01/grid/12.1/OPatch/auto/database/bin/RemoteHostExecutor.pl line 2.
Command output:
oracle.dbsysmodel.driver.sdk.productdriver.ProductDriverException: Execution
failed for host uat-db-001 due to : java.lang.Exception: Remote command
execution failed due to
*** Authorized uses only. All activity may be monitored and reported. ***
Can't locate Cwd.pm in @INC (@INC contains:
/u01/g01/grid/12.1/perl/lib/site_perl/5.14.1/x86_64-linux-thread-multi
/u01/g01/grid/12.1/perl/lib/site_perl/5.14.1
/u01/g01/grid/12.1/perl/lib/5.14.1/x86_64-linux-thread-multi
/u01/g01/grid/12.1/perl/lib/5.14.1 .) at
/u01/g01/grid/12.1/OPatch/auto/database/bin/RemoteHostExecutor.pl line 2.
BEGIN failed--compilation aborted at
/u01/g01/grid/12.1/OPatch/auto/database/bin/RemoteHostExecutor.pl line 2.
Command output:
OPatchAuto failed.
OPatchauto session completed at Fri Oct 4 13:15:08 2019
Time taken to complete the session 1 minute, 14 seconds
opatchauto failed with error code 42
[root@uat-db-002 29699255]#
Checked MOS and Google but no help. I have shared partial session log below
i.e.
/u01/g01/grid/12.1/cfgtoollogs/opatchauto/opatchauto2019-10-04_01-14-44PM.log .
Could you please let me know what could be the issue.
Oct 04, 2019 1:15:06 PM com.oracle.cie.remote.linux.LinuxCommandExecutionHelper
getSSHCmd
INFO: SSH path::/bin/ssh
Oct 04, 2019 1:15:06 PM
com.oracle.cie.remote.linux.LinuxSSHUserEquivalenceConnHandlerImpl
executeCommand
INFO: Command execution command: /bin/ssh -o FallBackToRsh=no -o
PasswordAuthentication=no -o StrictHostKeyChecking=no -o
NumberOfPasswordPrompts=0 uat-db-001 ls /u01/g01/grid/12.1 > /dev/null
Oct 04, 2019 1:15:07 PM
com.oracle.cie.remote.linux.LinuxSSHUserEquivalenceConnHandlerImpl
executeCommand
INFO: Command execution return code: 0
Oct 04, 2019 1:15:07 PM
com.oracle.cie.remote.linux.LinuxSSHUserEquivalenceConnHandlerImpl
executeCommand
INFO: command execution message:
Oct 04, 2019 1:15:07 PM
com.oracle.cie.remote.linux.LinuxSSHUserEquivalenceConnHandlerImpl
executeCommand
INFO: command executon failure message:
*** Authorized uses only. All activity may be monitored and reported. ***
Oct 04, 2019 1:15:07 PM
com.oracle.helper.remote.ssh.SSHEquivalenceCommandExecutor executeAndNotify
INFO: Executing command on host uat-db-001 : /u01/g01/grid/12.1//perl/bin/perl
/u01/g01/grid/12.1/OPatch/auto/database/bin/RemoteHostExecutor.pl
-GRID_HOME=/u01/g01/grid/12.1 -OBJECTLOC=/u01/g01/grid/
12.1//cfgtoollogs/opatchautodb/hostdata.obj -CRS_ACTION=check
-CLUSTERNODES=uat-db-001,uat-db-002
-JVM_HANDLER=oracle/dbsysmodel/driver/sdk/productdriver/remote/RemoteOperationHelper
Oct 04, 2019 1:15:07 PM com.oracle.cie.remote.linux.LinuxCommandExecutionHelper
getSSHCmd
INFO: SSH path::/bin/ssh
Oct 04, 2019 1:15:07 PM
com.oracle.cie.remote.linux.LinuxSSHUserEquivalenceConnHandlerImpl
executeCommand
INFO: Command execution command: /bin/ssh -o FallBackToRsh=no -o
PasswordAuthentication=no -o StrictHostKeyChecking=no -o
NumberOfPasswordPrompts=0 uat-db-001 /u01/g01/grid/12.1//perl/bin/perl /u01/g0
1/grid/12.1/OPatch/auto/database/bin/RemoteHostExecutor.pl
-GRID_HOME=/u01/g01/grid/12.1
-OBJECTLOC=/u01/g01/grid/12.1//cfgtoollogs/opatchautodb/hostdata.obj
-CRS_ACTION=check -CLUSTERNODES=uat-db-001
,uat-db-002
-JVM_HANDLER=oracle/dbsysmodel/driver/sdk/productdriver/remote/RemoteOperationHelper
Oct 04, 2019 1:15:07 PM
com.oracle.cie.remote.linux.LinuxSSHUserEquivalenceConnHandlerImpl
executeCommand
INFO: Command execution return code: 2
Oct 04, 2019 1:15:07 PM
com.oracle.cie.remote.linux.LinuxSSHUserEquivalenceConnHandlerImpl
executeCommand
INFO: command execution message:
Oct 04, 2019 1:15:07 PM
com.oracle.cie.remote.linux.LinuxSSHUserEquivalenceConnHandlerImpl
executeCommand
INFO: command executon failure message:
*** Authorized uses only. All activity may be monitored and reported. ***
Can't locate Cwd.pm in @INC (@INC contains:
/u01/g01/grid/12.1/perl/lib/site_perl/5.14.1/x86_64-linux-thread-multi
/u01/g01/grid/12.1/perl/lib/site_perl/5.14.1
/u01/g01/grid/12.1/perl/lib/5.14.1/x86_6
4-linux-thread-multi /u01/g01/grid/12.1/perl/lib/5.14.1 .) at
/u01/g01/grid/12.1/OPatch/auto/database/bin/RemoteHostExecutor.pl line 2.
BEGIN failed--compilation aborted at
/u01/g01/grid/12.1/OPatch/auto/database/bin/RemoteHostExecutor.pl line 2.
Oct 04, 2019 1:15:07 PM
com.oracle.helper.remote.ssh.SSHEquivalenceCommandExecutor executeAndNotify
WARNING: Error while executing command "/u01/g01/grid/12.1//perl/bin/perl
/u01/g01/grid/12.1/OPatch/auto/database/bin/RemoteHostExecutor.pl
-GRID_HOME=/u01/g01/grid/12.1 -OBJECTLOC=/u01/g01/grid/12.1/
/cfgtoollogs/opatchautodb/hostdata.obj -CRS_ACTION=check
-CLUSTERNODES=uat-db-001,uat-db-002
-JVM_HANDLER=oracle/dbsysmodel/driver/sdk/productdriver/remote/RemoteOperationHelper
" on host uat-db-001 :
Remote command execution failed due to
*** Authorized uses only. All activity may be monitored and reported. ***
Can't locate Cwd.pm in @INC (@INC contains:
/u01/g01/grid/12.1/perl/lib/site_perl/5.14.1/x86_64-linux-thread-multi
/u01/g01/grid/12.1/perl/lib/site_perl/5.14.1
/u01/g01/grid/12.1/perl/lib/5.14.1/x86_6
4-linux-thread-multi /u01/g01/grid/12.1/perl/lib/5.14.1 .) at
/u01/g01/grid/12.1/OPatch/auto/database/bin/RemoteHostExecutor.pl line 2.
BEGIN failed--compilation aborted at
/u01/g01/grid/12.1/OPatch/auto/database/bin/RemoteHostExecutor.pl line 2.
Command output:
Oct 04, 2019 1:15:07 PM
com.oracle.helper.remote.ssh.SSHEquivalenceCommandExecutor executeAndNotify
WARNING: Exception while executing command "" on host uat-db-001 : Remote
command execution failed due to
*** Authorized uses only. All activity may be monitored and reported. ***
Can't locate Cwd.pm in @INC (@INC contains:
/u01/g01/grid/12.1/perl/lib/site_perl/5.14.1/x86_64-linux-thread-multi
/u01/g01/grid/12.1/perl/lib/site_perl/5.14.1
/u01/g01/grid/12.1/perl/lib/5.14.1/x86_6
4-linux-thread-multi /u01/g01/grid/12.1/perl/lib/5.14.1 .) at
/u01/g01/grid/12.1/OPatch/auto/database/bin/RemoteHostExecutor.pl line 2.
BEGIN failed--compilation aborted at
/u01/g01/grid/12.1/OPatch/auto/database/bin/RemoteHostExecutor.pl line 2.
Command output:
Oct 04, 2019 1:15:07 PM
com.oracle.helper.remote.ssh.SSHEquivalenceThreadHandler$SSHEquiCommandListener
notifyResult
INFO: Received command execution result for host uat-db-001
Oct 04, 2019 1:15:07 PM
oracle.dbsysmodel.driver.sdk.productdriver.remote.HostDataCollector notifyResult
INFO: Consolidating result from host : uat-db-001
Oct 04, 2019 1:15:07 PM
oracle.dbsysmodel.driver.sdk.productdriver.remote.HostDataCollector notifyResult
INFO: Execution failed for host uat-db-001 due to : Remote command execution
failed due to
*** Authorized uses only. All activity may be monitored and reported. ***
Can't locate Cwd.pm in @INC (@INC contains:
/u01/g01/grid/12.1/perl/lib/site_perl/5.14.1/x86_64-linux-thread-multi
/u01/g01/grid/12.1/perl/lib/site_perl/5.14.1
/u01/g01/grid/12.1/perl/lib/5.14.1/x86_6
4-linux-thread-multi /u01/g01/grid/12.1/perl/lib/5.14.1 .) at
/u01/g01/grid/12.1/OPatch/auto/database/bin/RemoteHostExecutor.pl line 2.
BEGIN failed--compilation aborted at
/u01/g01/grid/12.1/OPatch/auto/database/bin/RemoteHostExecutor.pl line 2.
Command output:
Oct 04, 2019 1:15:07 PM
oracle.dbsysmodel.driver.sdk.productdriver.remote.HostDataCollector
performRemoteOperForActions
INFO: Remote node list for home /u01/g01/grid/12.1 : []
2019-10-04 13:15:07,859 INFO [1] oracle.dbsysmodel.driver.sdk.util.OsysUtility
- Output message :::
2019-10-04 13:15:07,870 INFO [1]
com.oracle.cie.common.util.reporting.CommonReporter - Reporting console output
: Message{id='null', message='Remote command execution failed due to
*** Authorized uses only. All activity may be monitored and reported. ***
Can't locate Cwd.pm in @INC (@INC contains:
/u01/g01/grid/12.1/perl/lib/site_perl/5.14.1/x86_64-linux-thread-multi
/u01/g01/grid/12.1/perl/lib/site_perl/5.14.1
/u01/g01/grid/12.1/perl/lib/5.14.1/x86_6
4-linux-thread-multi /u01/g01/grid/12.1/perl/lib/5.14.1 .) at
/u01/g01/grid/12.1/OPatch/auto/database/bin/RemoteHostExecutor.pl line 2.
BEGIN failed--compilation aborted at
/u01/g01/grid/12.1/OPatch/auto/database/bin/RemoteHostExecutor.pl line 2.
Command output: '}
2019-10-04 13:15:07,873 INFO [1]
com.oracle.glcm.patch.auto.db.integration.model.productsupport.DBBaseProductSupport
- Space available after session: 33580 MB
2019-10-04 13:15:07,996 SEVERE [1] com.oracle.glcm.patch.auto.OPatchAuto -
OPatchAuto failed.
com.oracle.glcm.patch.auto.OPatchAutoException:
oracle.dbsysmodel.driver.sdk.productdriver.ProductDriverException: Execution
failed for host uat-db-001 due to : java.lang.Exception: Remote command exe
cution failed due to
*** Authorized uses only. All activity may be monitored and reported. ***
Can't locate Cwd.pm in @INC (@INC contains:
/u01/g01/grid/12.1/perl/lib/site_perl/5.14.1/x86_64-linux-thread-multi
/u01/g01/grid/12.1/perl/lib/site_perl/5.14.1
/u01/g01/grid/12.1/perl/lib/5.14.1/x86_6
4-linux-thread-multi /u01/g01/grid/12.1/perl/lib/5.14.1 .) at
/u01/g01/grid/12.1/OPatch/auto/database/bin/RemoteHostExecutor.pl line 2.
BEGIN failed--compilation aborted at
/u01/g01/grid/12.1/OPatch/auto/database/bin/RemoteHostExecutor.pl line 2.
Command output:
at
com.oracle.glcm.patch.auto.OPatchAuto.prepareOrchestration(OPatchAuto.java:775)
at
com.oracle.glcm.patch.auto.OPatchAuto.orchestrate(OPatchAuto.java:397)
at
com.oracle.glcm.patch.auto.OPatchAuto.orchestrate(OPatchAuto.java:344)
at com.oracle.glcm.patch.auto.OPatchAuto.main(OPatchAuto.java:212)
Caused by: oracle.dbsysmodel.driver.sdk.productdriver.ProductDriverException:
Execution failed for host uat-db-001 due to : java.lang.Exception: Remote
command execution failed due to
*** Authorized uses only. All activity may be monitored and reported. ***
Can't locate Cwd.pm in @INC (@INC contains:
/u01/g01/grid/12.1/perl/lib/site_perl/5.14.1/x86_64-linux-thread-multi
/u01/g01/grid/12.1/perl/lib/site_perl/5.14.1
/u01/g01/grid/12.1/perl/lib/5.14.1/x86_6
4-linux-thread-multi /u01/g01/grid/12.1/perl/lib/5.14.1 .) at
/u01/g01/grid/12.1/OPatch/auto/database/bin/RemoteHostExecutor.pl line 2.
BEGIN failed--compilation aborted at
/u01/g01/grid/12.1/OPatch/auto/database/bin/RemoteHostExecutor.pl line 2.
Command output:
at
oracle.dbsysmodel.driver.sdk.productdriver.OUIDriver.checkForException(OUIDriver.java:846)
at
oracle.dbsysmodel.driver.sdk.productdriver.OUIDriver.handleExceptions(OUIDriver.java:832)
at
oracle.dbsysmodel.driver.sdk.productdriver.OUIDriver.fetchRemoteHostData(OUIDriver.java:791)
at
oracle.dbsysmodel.driver.sdk.productdriver.OUIDriver.fetchRemoteNodeData(OUIDriver.java:690)
at
com.oracle.glcm.patch.auto.db.integration.model.productsupport.topology.DBPatchingHelper.fetchPreInitiatedNodes(DBPatchingHelper.java:899)
at
com.oracle.glcm.patch.auto.db.product.validation.validators.RemoteGIStatusValidation.isFirstNodeOfPatching(RemoteGIStatusValidation.java:274)
at
com.oracle.glcm.patch.auto.db.product.validation.validators.RemoteGIStatusValidation.validate(RemoteGIStatusValidation.java:185)
at
com.oracle.glcm.patch.auto.db.product.validation.DBValidationController.invoke(DBValidationController.java:91)
at
com.oracle.glcm.patch.auto.db.integration.model.productsupport.topology.CRSTopologyBuilder.validate(CRSTopologyBuilder.java:367)
at
com.oracle.glcm.patch.auto.db.integration.model.productsupport.topology.AbstractTopologyBuilder.buildTopology(AbstractTopologyBuilder.java:117)
at
com.oracle.glcm.patch.auto.db.integration.model.productsupport.topology.CRSTopologyBuilder.buildTopology(CRSTopologyBuilder.java:112)
at
com.oracle.glcm.patch.auto.db.integration.model.productsupport.DBBaseProductSupport.loadTopology(DBBaseProductSupport.java:192)
at
com.oracle.glcm.patch.auto.db.integration.model.productsupport.DBProductSupport.loadTopology(DBProductSupport.java:69)
at
com.oracle.glcm.patch.auto.OPatchAuto.loadTopology(OPatchAuto.java:1732)
at
com.oracle.glcm.patch.auto.OPatchAuto.prepareOrchestration(OPatchAuto.java:730)
... 3 more
2019-10-04 13:15:07,999 INFO [1]
com.oracle.cie.common.util.reporting.CommonReporter - Reporting console output
: Message{id='null',
message='oracle.dbsysmodel.driver.sdk.productdriver.ProductDriverE
xception: Execution failed for host uat-db-001 due to : java.lang.Exception:
Remote command execution failed due to
*** Authorized uses only. All activity may be monitored and reported. ***
Can't locate Cwd.pm in @INC (@INC contains:
/u01/g01/grid/12.1/perl/lib/site_perl/5.14.1/x86_64-linux-thread-multi
/u01/g01/grid/12.1/perl/lib/site_perl/5.14.1
/u01/g01/grid/12.1/perl/lib/5.14.1/x86_6
4-linux-thread-multi /u01/g01/grid/12.1/perl/lib/5.14.1 .) at
/u01/g01/grid/12.1/OPatch/auto/database/bin/RemoteHostExecutor.pl line 2.
BEGIN failed--compilation aborted at
/u01/g01/grid/12.1/OPatch/auto/database/bin/RemoteHostExecutor.pl line 2.
Command output: '}
2019-10-04 13:15:08,003 INFO [1]
com.oracle.cie.common.util.reporting.CommonReporter - Reporting console output
: Message{id='null', message='OPatchAuto failed.'}
[root@uat-db-002 29699255]#
So far , I tried
For "Can't locate Cwd.pm in @INC" , I checked it's there. Also, checked the
permission for /u01/g01/grid/12.1/cfgtoollogs/opatchautodb and its owned by
grid home user.
[root@uat-db-002 29699255]# find / -name Cwd.pm
/usr/lib64/perl5/vendor_perl/Cwd.pm
/u01/g01/grid/12.1/perl/lib/5.14.1/x86_64-linux-thread-multi/Cwd.pm
/u01/g01/grid/12.1/perl/lib/5.14.1/Cwd.pm
/u01/d01/db/12.1.0/perl/lib/5.14.1/x86_64-linux-thread-multi/Cwd.pm
/u01/em_agent/agent_13.2.0.0.0/perl/lib/5.14.4/x86_64-linux-thread-multi/Cwd.pm
[root@uat-db-002 29699255]#
Cheers
Sameer
This is an email from Ezidebit Pty Ltd. The email and any attachments may be
confidential, legally privileged and/or subject to copyright. If you are not
the intended recipient, you must not disclose or use the information contained
in this e-mail. If you have received this email in error, please notify us
immediately and delete the email and all copies. We do not guarantee that this
email and any attachments are free from virus or other errors. We will not be
responsible for loss or damage resulting (either directly or indirectly) from
any such virus or error. We will only be responsible for a change to a document
if we made the change. The content of and opinions expressed in non-business
emails are not that of Ezidebit Pty Ltd and are that of the sender.
