Lei, Average message queue time in AWR is derived from (G)V$GES_STATISTICS. This number is the time taken to put an indirect message in to the message queue and sending it. We would need to see the KSXP message stats time to identify if there is a problem in IPC layer. IPC times are typilcaly < 1ms and you can query the X$KSXPCLIENT to get more data. . If you want me to speculate, I would assume either your machine is extremely busy (100% CPU) during that time or inter connect is peaked out. You can confirm this by checking the OSWatcher archives. Best Regards, K Gopalakrishnan Oracle Database 10g Real Application Clusters Handbook http://www.amazon.com/gp/product/007146509X/ Oracle Wait Interface: A Practical Guide to Performance Diagnostics & Tuning http://www.amazon.com/exec/obidos/tg/detail/-/007222729X/ On Wed, Apr 28, 2010 at 3:34 PM, Lei Zeng <lzeng@xxxxxxxxxxxxx> wrote: > Hi: > > > > One instance of our 8-node RAC database had a 2-minute hiccup yesterday ( I > mean it was totally frozen up and I could not even ‘sqlplus as sysdba’ on > the box itself). I could go to other nodes and no issue there. From the AWR > reports, I can only see one anomaly: > > Global Cache and Enqueue Services - Messaging Statistics > > Avg message sent queue time (ms): > > 1,289.3 > > > > Did some research but could not find much about this metric. Anybody helps > me to get some clarification on it? > > > > Thanks in advance, > > Lei >