Hi Amir, Are you able to replicate a situation with > 1s for single read ? If yes enable trace on Oracle and setup a wireshark on dNFS interface. Using a tim from trace you should be able to find corresponding logs from network sniffer and maybe you will find a reason why it take too long. It could be a network retransmission so your network guys will have to find a root cause or it could be a long read request on EMC so problem will be redirected to storage guys. Here is a example of output from 10046 trace, strace command and wireshark http://oracleprof.blogspot.ie/2013/01/how-dnfs-database-clone-works-part-1.html regards, Marcin Przepiorowski On Tue, Aug 20, 2013 at 3:14 AM, Hameed, Amir <Amir.Hameed@xxxxxxxxx> wrote: > Hi Austin, > I really appreciate your feedback. I did go through the document that you > have mentioned below a few times in the past but most of the patches listed > at that time were either released for Linux or Exadata for Solaris; we are > SPARC Solaris. After we provided pstack dumps to Oracle today, Oracle > confirmed that we are hitting the same bug that you have mentioned below. > But since we are at 11.2.0.3.6 and the patch for Solaris is available for > 11.2.0.3.5, Oracle is in the process of back-porting the patch. > > You are also right about the 32k chunk size because of the rsize and wsize > mount options and I confirmed it by running truss. How has your experience > been with dNFS so far? > > Thanks > -----Original Message----- > From: oracle-l-bounce@xxxxxxxxxxxxx [mailto:oracle-l-bounce@xxxxxxxxxxxxx] > On Behalf Of Austin Hackett > Sent: Monday, August 19, 2013 5:39 PM > To: oracle-l digest users > Subject: Re: Latency of "direct path reads" > > Hi Amir > > I've recently deployed a 5 node 11.2.0.3.2 RAC with dNFS and a NetApp > FAS6290 into production. > > Are you familiar with MOS Doc ID 1495104.1, which has a list of > Recommended Patches for Direct NFS? If you don't have these patches > applied, I would definitely suggest getting them installed. For example, > during my initial lab tests I kept hitting bug 15987992 under heavy > sequential read load. I/O would completely hang for a seemingly indefinite > period. Admittedly, this doesn't sound like what you're seeing. However, > there are other bugs mentioned which sound like they could be of interest. > You may need to request back ports of the patches for your exact PSU-level. > I did. > > Apologies if this is old news. > > Regarding your question about sending > 32K chunks of data, the Oracle > recommended value for the rsize mount option is 32K, so I think a 1MB multi > block read would be split into 32K chunks under KNFS anyway. I don't know > if DNFS uses a larger transfer buffer size? > > Austin > > -- > //www.freelists.org/webpage/oracle-l > > > -- > //www.freelists.org/webpage/oracle-l > > > -- Marcin Przepiorowski http://oracleprof.blogspot.com -- //www.freelists.org/webpage/oracle-l