Re: Backups failing
- From: Oliver <ofabelo@xxxxxxxxx>
- To: Niall Litchfield <niall.litchfield@xxxxxxxxx>
- Date: Tue, 30 Aug 2011 08:56:35 +0100
I'm looking emagent.trc, just when it crashes ... The log output is:
2011-08-30 00:11:12 Thread-4039011216 ERROR scheduler: Collection
oracle_listener:LISTENER_xxx.xxx.xx:Response has not finished in 721 minutes
(allowed: 720).
Exiting agent.
2011-08-30 00:11:12 Thread-4039011216 ERROR : Signalled to Exit with status
3
2011-08-30 00:11:12 Thread-4136433344 ERROR main: EMAgent abnormal
terminating
2011-08-30 00:11:41 Thread-4136175296 WARN command: Job Subsystem Timeout
set at 600 seconds
2011-08-30 00:11:41 Thread-4136175296 WARN upload: Upload manager has no
Failure script: disabled
2011-08-30 00:11:41 Thread-4136175296 WARN upload: Recovering left over xml
files in upload directory
2011-08-30 00:11:42 Thread-4136175296 WARN upload: Recovered 0 left over
xml files in upload directory
2011-08-30 00:11:42 Thread-4136175296 WARN metadata: Metric Disk_Path does
not have any data columns
2011-08-30 00:11:42 Thread-4136175296 WARN metadata: Metric
osm_diskGroupPolicies does not have any data columns
2011-08-30 00:12:42 Thread-4136175296 ERROR TargetManager: TIMEOUT when
compute dynamic properties for target xxx.xxx.xxx
2011-08-30 00:12:42 Thread-4136175296 ERROR TargetManager: target
{xxx.xxx.xxx, oracle_database} is broken: cannot compute dynamic properties
in time.
2011-08-30 00:12:43 Thread-4136175296 WARN collector: the column name
DiskActivityavwait in this condition does not exist
2011-08-30 00:12:43 Thread-4119530384 ERROR upload: Error in
uploadXMLFiles. Trying again in 60.00 seconds.
I'm "googling" but without success for now ... I don't find the solution.
Thanks beforehand.
2011/8/30 Oliver <ofabelo@xxxxxxxxx>
> Same error today, initialization failure in status of the job and in the
> error log, "Step aborted because agent went down".
>
>
>
>
> 2011/8/29 Oliver <ofabelo@xxxxxxxxx>
>
>> This is the output:
>>
>> Oracle Enterprise Manager 10g Database Control Release 10.2.0.1.0
>> Copyright (c) 1996, 2005 Oracle Corporation. All rights reserved.
>> ---------------------------------------------------------------
>> Agent Version : 10.1.0.4.1
>> OMS Version : 10.1.0.4.0
>> Protocol Version : 10.1.0.2.0
>> Agent Home : /u01/app/oracle/product/10.2.0/db_1/xxx_ora10g
>> Agent binaries : /u01/app/oracle/product/10.2.0/db_1
>> Agent Process ID : 19051
>> Parent Process ID : 18114
>> Agent URL : http://xxx:3938/emd/main
>> Started at : 2011-08-29 12:09:00
>> Started by user : oracle
>> Last Reload : 2011-08-29 12:14:14
>> Last successful upload : 2011-08-29 12:14:23
>> Total Megabytes of XML files uploaded so far : 4.40
>> Number of XML files pending upload : 0
>> Size of XML files pending upload(MB) : 0.00
>> Available disk space on upload filesystem : 87.14%
>> ---------------------------------------------------------------
>> Agent is Running and Ready
>>
>> I see it all ok, isn't it?
>> Jobs are many hours for seeing it has failed ... I will look tomorrow, but
>> I see that it doesn't writing in the backup dir ...
>>
>>
>> 2011/8/29 Niall Litchfield <niall.litchfield@xxxxxxxxx>
>>
>>> Hi
>>>
>>> what is the full output on the target machine of emctl status agent?
>>>
>>>
>>>
>>> On Mon, Aug 29, 2011 at 11:49 AM, Oliver <ofabelo@xxxxxxxxx> wrote:
>>>
>>>> When I do "emctl upload agent" step, it shows:
>>>>
>>>> EMD upload error: uploadXMLFiles skipped :: OMS version not checked
>>>> yet..
>>>>
>>>> Jus the error of that page ...
>>>>
>>>>
>>>> 2011/8/29 Oliver <ofabelo@xxxxxxxxx>
>>>>
>>>>> Yes, I did crosscheck in rman ...
>>>>> I'm going to do the steps of:
>>>>>
>>>>>
>>>>> http://jhdba.wordpress.com/2008/12/10/clearing-an-enterprise-manager-agent-that-fails-to-upload/
>>>>>
>>>>> Is it ok?
>>>>>
>>>>> Thanks beforehand.
>>>>>
>>>>>
>>>>> 2011/8/29 Pavel <ocp.pauler@xxxxxxxxx>
>>>>>
>>>>>> Hi
>>>>>> Did you crosscheck after delete some backups? Clear state don't remove
>>>>>> created jobs.
>>>>>>
>>>>>>
>>>>>> Regards,
>>>>>> Pavel.
>>>>>>
>>>>>> 2011/8/29 Oliver <ofabelo@xxxxxxxxx>
>>>>>>
>>>>>>> Thanks, but I'm sorry, I'm a newbie, the dba is on vacation and I
>>>>>>> have to look it because I'm the person with more knowledges of Oracle.
>>>>>>> I've tried stop and start agent/dbconsole but it is failing again. I
>>>>>>> understand your commands, except remove all uploaded files :-?
>>>>>>> Clear state or remove uploads files remove created jobs?
>>>>>>> Ah, one thing, I think that it started to fail when it surpassed the
>>>>>>> flash recovery quota. I had 300GB, and it had 330GB written, in
>>>>>>> destination
>>>>>>> of backup dir. I deleted some backups and I will increase to 500GB the
>>>>>>> flash
>>>>>>> recovery quota (I've not restarted database, is it necessary?)
>>>>>>> Thanks beforehand.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2011/8/29 Pavel <ocp.pauler@xxxxxxxxx>
>>>>>>>
>>>>>>>> Hi
>>>>>>>> Could you try these one:
>>>>>>>>
>>>>>>>> Issue emctl stop agent, emctl clear state agent, emctl start agent.
>>>>>>>>
>>>>>>>> And I think remove all uploads files.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Pavel.
>>>>>>>>
>>>>>>>>
>>>>>>>> 2011/8/29 Oliver <ofabelo@xxxxxxxxx>
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>> backups of EM are failing suddenly. It shows "Initialization
>>>>>>>>> failure" in status and error log shows "Step aborted because agent
>>>>>>>>> went
>>>>>>>>> down". The error comes after many hours, I see that it doesn't write
>>>>>>>>> nothing
>>>>>>>>> in the oputut dir of the backup.
>>>>>>>>> If I do "$ tail -f emagent.trc", it shows some warns and the
>>>>>>>>> errors:
>>>>>>>>>
>>>>>>>>> 2011-08-29 09:43:25 Thread-4136277696 ERROR TargetManager: TIMEOUT
>>>>>>>>> when compute dynamic properties for target
>>>>>>>>> ora10g.localhost.localdomain
>>>>>>>>> 2011-08-29 09:43:25 Thread-4136277696 ERROR TargetManager: target
>>>>>>>>> {ora10g.localhost.localdomain, oracle_database} is broken: cannot
>>>>>>>>> compute
>>>>>>>>> dynamic properties in time.
>>>>>>>>> 2011-08-29 09:43:25 Thread-4136277696 WARN collector: the column
>>>>>>>>> name DiskActivityavwait in this condition does not exist
>>>>>>>>> 2011-08-29 09:43:25 Thread-4119632784 ERROR upload: Error in
>>>>>>>>> uploadXMLFiles. Trying again in 60.00 seconds.
>>>>>>>>>
>>>>>>>>> It is 10.2.0.1.0 on Linux. Someone can help me?
>>>>>>>>> Thanks beforehand.
>>>>>>>>>
>>>>>>>>> Cheers...
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> Niall Litchfield
>>> Oracle DBA
>>> http://www.orawin.info
>>>
>>
>>
>
Other related posts:
