Hi all,
This is resolved. Found out that SAML was being used and it was creating
numerous connections for each customer DB connection. I keystore cache was
created that relieved the pressure.
tnsping saw delays up to 3000ms and many time "no listener" yet listener
was continuously up.
AWR show logins per second of 400+
Found a perl script that mined the listener.log for connections per minute
that made a cool Excel graph the leadership team could read.
Oracle support told me that the listener couldn't handle more than 20
connections per second and to create additional listeners if needed.
I told the support and app dev teams of the connection rate and they hunted
till they found the SAML issue.
Thanks
On Tue, Oct 31, 2017 at 12:03 PM, Bill Ferguson <wbfergus@xxxxxxxxx> wrote:
Hi Woody,
Check the event log on the Windows server. This sometimes gives a bit more
useful information. One possible explanation might be with the password
having expired? I've had this problem a few times on Windows and the error
messages are terrible, so I can't remember exactly what an Error 61 is, but
hopefully there is a better error message in the event log. Have the system
Admin check the Event Viewer. From there (it's been so long I can't
remember exactly), I think it is "Windows Logs" and then "Application".
Hope this helps,
Bill Ferguson
On Thu, Oct 26, 2017 at 8:08 AM, Woody McKay <woody.mckay@xxxxxxxxx>
wrote:
Hello all,
Got a production server that has been having listener problems this week,
each morning. Hosted at rackspace.
lsnrctl status some times works and other times gives no listener error.
Windows event log shows the listener last came up yesterday, yet lsnrctl
show it available and not available this morning.
The listener.log shows no errors, only established connections.
The listener log grows fast. Connections to DB were about 300, with AWR
showing a login rate of about 136/second.
Earlier in the morning, when everything worked fine, the login rate was
81/second.
The DB alert log shows no errors.
Here is the status error:
LSNRCTL> status ncdb226
Connecting to (ADDRESS=(PROTOCOL=TCP)(HOST=172.24.12.59)(PORT=1521))
TNS-12541: TNS:no listener
TNS-12560: TNS:protocol adapter error
TNS-00511: No listener
64-bit Windows Error: 61: Unknown error
I can't figure out what this means: 64-bit Windows Error: 61: Unknown
error
All of this worked fine last week. Could there be a windows port time out
or something?
Thoughts?
Woody
--
-- Bill Ferguson
