Re: OEM reporting listener down but listener not actually down

  • From: Mladen Gogala <gogala.mladen@xxxxxxxxx>
  • To: Jack van Zanen <jack@xxxxxxxxxxxx>
  • Date: Sun, 1 Dec 2019 19:54:05 -0500

It is not reporting status to OMS. If it was reporting status to OMS, OMS could do something similar to "srvctl status listener" and we have our answer. However, you cannot get a TNS error doing "svrctl status listener". If it was OMS, you could check your local listener log using adrci and find the error. It looks like the OEM trying external connection from the OEM host to your RAC. I may be wrong. Please check the listener log using the ADRCI tool.

Regards

On 12/1/19 7:46 PM, Jack van Zanen wrote:

Hang on, I am confused now.

I thought that the agent on the database host reports the status of the listener to OMS. So I read this as the agent on database host could not connect to listener on database host.
Are you saying that we can get this because the OMS server can not connect to the listener?



Jack van Zanen


-------------------------
This e-mail and any attachments may contain confidential material for the sole use of the intended recipient. If you are not the intended recipient, please be aware that any disclosure, copying, distribution or use of this e-mail or any attachment is prohibited. If you have received this e-mail in error, please contact the sender and delete all copies.
Thank you for your cooperation


On Mon, Dec 2, 2019 at 11:35 AM Mladen Gogala <gogala.mladen@xxxxxxxxx <mailto:gogala.mladen@xxxxxxxxx>> wrote:

    Hi Jack,

    That can also be a problem with DNS or a network route. If the
    host hosting the OEM cannot reach listener for whatever reason,
    you will get this error. It can be something as trivial as a bad
    port on the switch or change in DNS server.

    Regards

    On 12/1/19 7:19 PM, Jack van Zanen wrote:
    intermittently we get some alert from OEM about listener being down.

    Target name=*LISTENER_SCAN3_telstraexa1-000_1*
    
<https://aus01.safelinks.protection.outlook.com/?url=https%3A%2F%2FNSW-ATP-GRID.snl.7net.com.au%3A7799%2Fem%2Fredirect%3FpageType%3DTARGET_HOMEPAGE%26targetName%3DLISTENER_SCAN3_telstraexa1-000_1%26targetType%3Doracle_listener&data=01%7C01%7CJVanZanen%40Seven.com.au%7C5514208b3d2848d8fe9f08d774d613ee%7Cb359291241554399b790752c894d2935%7C0&sdata=Pww2ktObyuQu4vBRxuWvcmAGkXXJIozv%2F8nHJP1ABHE%3D&reserved=0>

    Categories=*Availability*
    Message=*The listener is down: TNS-12545: Connect failed because
    target host or object does not exist .*
    
<https://aus01.safelinks.protection.outlook.com/?url=https%3A%2F%2FNSW-ATP-GRID.snl.7net.com.au%3A7799%2Fem%2Fredirect%3FpageType%3Dsdk-core-event-console-detailEvent%26issueID%3DE43696ECA26F42148CCCD6EC4862F4A9&data=01%7C01%7CJVanZanen%40Seven.com.au%7C5514208b3d2848d8fe9f08d774d613ee%7Cb359291241554399b790752c894d2935%7C0&sdata=z%2FQDPTS5qbEkx1MB8kdz3JKvyYGToJ4%2BUbcptqfRc7I%3D&reserved=0>



    Checking the server we see no issue with the listener being down.


    grid      24040      1  0 Nov23 ?        00:03:48
    /u01/app/12.2.0.1/grid/bin/tnslsnr
    <http://12.2.0.1/grid/bin/tnslsnr> ASMNET1LSNR_ASM -no_crs_notify
    -inherit
    grid      24402      1  0 Nov23 ?        00:04:58
    /u01/app/12.2.0.1/grid/bin/tnslsnr
    <http://12.2.0.1/grid/bin/tnslsnr> LISTENER -no_crs_notify -inherit
    grid     234773      1  0 Nov23 ?        00:02:16
    /u01/app/12.2.0.1/grid/bin/tnslsnr
    <http://12.2.0.1/grid/bin/tnslsnr> LISTENER_SCAN2 -no_crs_notify
    -inherit
    grid     234791      1  0 Nov23 ?        00:02:13
    /u01/app/12.2.0.1/grid/bin/tnslsnr
    <http://12.2.0.1/grid/bin/tnslsnr> LISTENER_SCAN3 -no_crs_notify
    -inherit


    This is Oracle 12.2 on Exadata Cloud at Customer solution .
    Anyone else seen this behaviour before?

    Oracle suggested to turn on client and server sql net tracing but
    I am a bit hesitant due to the sheer volume of tracing it will
    likely generate before I get the issue again.

    To me it looks like a DNS issue, but Oracle looked at this and
    there is no issues there.




    Jack van Zanen


    -------------------------
    This e-mail and any attachments may contain confidential material
    for the sole use of the intended recipient. If you are not the
    intended recipient, please be aware that any disclosure, copying,
    distribution or use of this e-mail or any attachment is
    prohibited. If you have received this e-mail in error, please
    contact the sender and delete all copies.
    Thank you for your cooperation

-- Mladen Gogala
    Database Consultant
    Tel: (347) 321-1217

--
Mladen Gogala
Database Consultant
Tel: (347) 321-1217

Other related posts: