dNFS beats kNFS hands down; there is no question about it. dNFS also
does a good job of load-balancing load between the dNFS interfaces. It
is the various layers that are involved in the NAS infrastructure that
makes implementing dNFS very challanging. We have had our storage
vendor engaged for a long time now but have not had much luck which
leads me to believe that NAS itself has its limitations and does not
scale well when compared to FC based connectivity.
*From:*Kevin Jernigan [mailto:kevin.jernigan@xxxxxxxxxx]
*Sent:* Friday, October 17, 2014 1:06 PM
*To:* Hameed, Amir; Don Seiler
*Cc:* fuzzy.graybeard@xxxxxxxxx; oracle-l@xxxxxxxxxxxxx
*Subject:* Re: Is filesystemio_options relevant when the database is
on ASM ?
Direct NFS (dNFS) can lead to challenges for the underlying NAS
infrastructure, because it eliminates the performance bottlenecks
inherent in most OS implementations of the NFS client (what we usually
call "kernel" NFS or kNFS). For example, dNFS will allow each Oracle
process to make a direct connection to the NAS server, rather than
funneling all I/O requests through a single connection. This means
that the NAS server(s) need to be able to handle the increase in I/O
traffic that the database will request from the storage.
So, it's not a flaw in dNFS that creates problems for some NAS setups
- there are often performance limitations in the NAS server that were
masked by kNFS limitations, and dNFS removes those limitations. dNFS
does extremely well in environments that have the right setup to
handle higher IOPS etc - if the application workload on top of the
database requires a certain level of I/O throughput and latency, it's
much easier to get to those levels with dNFS than with kNFS.
-KJ
--
Kevin Jernigan
Senior Director Product Management
Advanced Compression, Hybrid Columnar
Compression (HCC), Database File System
(DBFS), SecureFiles, Database Smart Flash
Cache, Total Recall, Database Resource
Manager (DBRM), Direct NFS Client (dNFS),
Continuous Query Notification (CQN),
Index Organized Tables (IOT), Information
Lifecycle Management (ILM)
+1-650-607-0392 (o)
+1-415-710-8828 (m)
On 10/17/14, 8:13 AM, Hameed, Amir wrote:
We started using DNFS about 1 ½ years ago and have been bleeding
ever since. One of the issues is with the NAS technology from our
vendor which we have found unstable. The IO timings fluctuate too
much and the 10046 traces can easily prove it. The other issue is
most likely with our network infrastructure, which we were told
initially both by our data center folks and by our network vendor
that we did not need a separate dedicate network for the private
NAS traffic and that the existing switches would provide the QoS
that is needed but now both are proposing a dedicated set of
switches to try and resolve the issue.
DNFS works fine for environments that are not very demanding on
the IOs and latency but based on our experience, it does not do
well with high IOPS and low latency requirements.
*From:*Don Seiler [mailto:don@xxxxxxxxx]
*Sent:* Thursday, October 16, 2014 4:20 PM
*To:* Kevin Jernigan
*Cc:* Hameed, Amir; fuzzy.graybeard@xxxxxxxxx
<mailto:fuzzy.graybeard@xxxxxxxxx>; oracle-l@xxxxxxxxxxxxx
<mailto:oracle-l@xxxxxxxxxxxxx>
*Subject:* Re: Is filesystemio_options relevant when the database
is on ASM ?
For what it's worth, we have the 7420 (two of them). My criticism
of them doesn't come lightly. It's been a long year+ and the fact
that these problems have kept us from migrating to them has
resulted us having to deal with other emergencies from our current
aging storage that we hoped to be off of nearly a year ago.
Don.
On Thu, Oct 16, 2014 at 3:12 PM, Kevin Jernigan
<kevin.jernigan@xxxxxxxxxx <mailto:kevin.jernigan@xxxxxxxxxx>> wrote:
Don,
Based on your experiences I think anyone would understand your
perspective. My areas of responsibility within Oracle are all
within the database team, so I am not an expert on ZFSSA (ZS3)
storage, but I will contact the ZS3 product team to see if
they have any comments.
Thanks for your feedback,
-Kevin J
--
Kevin Jernigan
Senior Director Product Management
Advanced Compression, Hybrid Columnar
Compression (HCC), Database File System
(DBFS), SecureFiles, Database Smart Flash
Cache, Total Recall, Database Resource
Manager (DBRM), Direct NFS Client (dNFS),
Continuous Query Notification (CQN),
Index Organized Tables (IOT), Information
Lifecycle Management (ILM)
+1-650-607-0392 <tel:%2B1-650-607-0392> (o)
+1-415-710-8828 <tel:%2B1-415-710-8828> (m)
On 10/16/14, 12:59 PM, Don Seiler wrote:
Of course, it's entirely possible that the two ZFSSA units
that we received are the only two lemons off of the
assembly line. That would explain by the ZFSSA support
techs had so much trouble finding/fixing the problems
(most of which are still not fixed).
Last summer I couldn't have been more excited to get these
units installed and start using them. But it was clear
from the start that things were going wrong, and it was a
series of problems from the start in various components of
the ZFSSA. Even better was the fact that the two units
each had unique problems, failing differently than their
counterpart.
So, given the problems we've seen, considering both the
quantity and severity, and how completely unimpressed we
were with the "one-stop shop" for support that was one of
the big selling points, we can in no way consider moving
our production databases onto it, and are looking for
alternative storage to remove our staging and unit test
databases off of it. Even if/when the open bugs get fixed
and someone discovers the cause of the NFS hangs, the
trust is completely gone in these systems.
Don.
On Thu, Oct 16, 2014 at 2:41 PM, Don Seiler <don@xxxxxxxxx
<mailto:don@xxxxxxxxx>> wrote:
We've had nothing but problems with them from
installation misconfigurations to hardware failures
(two silent NIC failures in addition to the disk
failures we get warnings about, and one instance of
the entire appliance IO module crashing). The last few
months we'd see intermittent NFS hanging for 5-6
minutes to all mounts from the ZFSSA, resulting in
those databases crashing. That is the most worrisome.
If the storage can't stay online, then we have bigger
problems than worrying about speed.
Add on to that the less than stellar support
recommendations we've gotten flip-flopping around
Infiniband recommendations, write-bias setttings,
bonding configurations, etc. and it's been a complete
nightmare that's left us still on our old storage that
is starting to fail. I have zero confidence in the
ZFSSA (at least the two machines that we've been sold)
to run our database. This is 14 months after
installation and they're still not in production and
never will be.
You say you have many customers on it, but we found
this hard to believe given all of the bugs that we
tripped over along the way, including a couple that
were apparently discovered by us. We would have
expected support to give us a heads-up about all of
the needed patches if they have so many customers that
have done the same thing. Other than the recommended
Direct NFS patches MOS note, we've been basically
stumbling around in the dark. At various points,
support suggests patches that only *might* fix the
problem ... as if they aren't sure themselves. And
those patches don't ever fix the problem.
Don.
On Thu, Oct 16, 2014 at 2:18 PM, Kevin Jernigan
<kevin.jernigan@xxxxxxxxxx
<mailto:kevin.jernigan@xxxxxxxxxx>> wrote:
Don,
Why do you recommend not using ZFSSA for live
databases yet? We (Oracle) have many customers
using ZFSSA for live production environments -
including our own IT department, with >200PBs of
ZFSSA storage in place for applications that
support all aspects of our business...
-Kevin J
--
Kevin Jernigan
Senior Director Product Management
Advanced Compression, Hybrid Columnar
Compression (HCC), Database File System
(DBFS), SecureFiles, Database Smart Flash
Cache, Total Recall, Database Resource
Manager (DBRM), Direct NFS Client (dNFS),
Continuous Query Notification (CQN),
Index Organized Tables (IOT), Information
Lifecycle Management (ILM)
+1-650-607-0392 <tel:%2B1-650-607-0392> (o)
+1-415-710-8828 <tel:%2B1-415-710-8828> (m)
On 10/16/14, 11:32 AM, Don Seiler wrote:
Yes in 12c DNFS works on NFSv4. In fact, NFSv4
is required if you plan to use OISP (Oracle
Intelligent Storage Protocol) to talk to their
ZFSSA. Although I would not suggest using the
ZFSSA to run live databases yet. Should be OK
for FRA uses.
Don.
On Thu, Oct 16, 2014 at 10:46 AM, Kevin
Jernigan <kevin.jernigan@xxxxxxxxxx
<mailto:kevin.jernigan@xxxxxxxxxx>> wrote:
As of Oracle Database 12c, dNFS works with
both NFSv3 and NFSv4...KJ
--
Kevin Jernigan
Senior Director Product Management
Advanced Compression, Hybrid Columnar
Compression (HCC), Database File System
(DBFS), SecureFiles, Database Smart Flash
Cache, Total Recall, Database Resource
Manager (DBRM), Direct NFS Client (dNFS),
Continuous Query Notification (CQN),
Index Organized Tables (IOT), Information
Lifecycle Management (ILM)
+1-650-607-0392 <tel:%2B1-650-607-0392> (o)
+1-415-710-8828 <tel:%2B1-415-710-8828> (m)
On 10/16/14, 8:37 AM, Hameed, Amir wrote:
I don’t believe DNFS is certified to
work with NFSv4.
*From:*oracle-l-bounce@xxxxxxxxxxxxx
<mailto:oracle-l-bounce@xxxxxxxxxxxxx>
[mailto:oracle-l-bounce@xxxxxxxxxxxxx]
*On Behalf Of *Hans Forbrich
*Sent:* Thursday, October 16, 2014
11:30 AM
*To:* oracle-l@xxxxxxxxxxxxx
<mailto:oracle-l@xxxxxxxxxxxxx>
*Subject:* Re: Is filesystemio_options
relevant when the database is on ASM ?
On 16/10/2014 3:35 AM, Frits Hoogland
wrote:
When using NFS underneath ASM,
I've witnessed
filesystemio_options being
honoured by the database, which
means it needs setting it to
'setall' for the combination
AIO+DIO. Which makes sense,
because you need to create a file
on a (NFS) filesystem to be used
as ASM disk device.
Then it becomes important to know
which NFS?
I believe DNFS behaves different than
standard NFSv3 which may be different
again from NFSv4
/Hans
--
Don Seiler
http://www.seiler.us
--
Don Seiler
http://www.seiler.us
--
Don Seiler
http://www.seiler.us
--
Don Seiler
http://www.seiler.us
