[SI-LIST] Re: Fibre channel interconnect margins

  • From: Alan.Hiltonnickel@xxxxxxx
  • To: Chris.Cheng@xxxxxxxxxxxx
  • Date: Mon, 03 Jul 2006 12:14:45 -0700

That's a novel idea, Chris - doing the math. Unfortunately I thought you 
were the math whiz. My math sucks.

I think your calculation is correct, but let's lay it out so we can be 
sure we're using the same numbers. I'll use PCI Express, since I seem to 
be focusing on that lately. This way I can pretend to be working.

PCIe data rate: 2.5 Gbps = 2.5x10e9 bits/second
PCIe maximum BER: 1.0x10e-12 errors per bit

Theoretical error interval at max BER: 2.5x10e9 * 1x10e-12 = 2.5x10e-3 
errors per second, or one error every 400 seconds.

400 seconds is about 6.7 minutes. On a PCIe x16 graphics card, that 
means an error every 25 seconds, which might result in dropped frames on 
a video stream. I'm working on an 8-channel card and I assume I'll be 
working late nights if I get a single-bit error every minute.

But a low BER means a LOT of testing. Just investigating a BER of 10e-12 
means monitoring a single channel for an event that only happens every 
six minutes. Achieving a BER of, say, 10e-15 means testing each channel 
for 4.6 days! So yes Chris, discussions of fault tolerance aside (and a 
typical PC system can probably easily tolerate the PCIe spec of one bit 
error in a trillion), I seriously doubt anyone deliberately designs only 
to the spec and ships with a BER of 10e-12. In fact, a BER of exactly 
10e-12 might indeed prove to be difficult to achieve. But you DO need to 
know your error rate (and it may be very low but it won't be zero) in 
order to know how much margin you have.

Based on raw testing with PCI Express, you would need 67 minutes to test 
a BER of 10e-13, 11 hours for a BER of 10e-14, and 4.6 days for a BER of 
10e-15. The makers of BERTS have algorithms to shorten this period by 
stressing the eye and estimating the BER, but I think full testing of 
the final product is still necessary.

I haven't yet quantified the target BER in my design. I imagine I'd want 
to see a BER of at least 10e-14, possibly even 10e-15, for shipping, 
although debugging a BER of 10e-13 would be challenging. I'd be 
interested in knowing what others consider an acceptable margin.

Alan


Chris Cheng wrote On 07/01/06 12:25,:

>My apology to Chris and Alan. My math is horrible. At 2+Gb/s, a BERT of 10e-12 
>is hardly a matter of days but hours or even minutes. With a typical dual loop 
>JBOD with a few hundred disks, we are talking about something you can see VERY 
>often. Are you still saying you are shipping product like that ?
>________________________________
>
>From: si-list-bounce@xxxxxxxxxxxxx on behalf of Chris Cheng
>Sent: Sat 7/1/2006 12:01 AM
>To: Alan.Hiltonnickel@xxxxxxx
>Cc: si-list@xxxxxxxxxxxxx
>Subject: [SI-LIST] Re: Fibre channel interconnect margins
>
>
>
>I know exactly what those expected and acceptable level you are talking about. 
>And I have adjust and monitor them at both production and prototype level. It 
>is either humping along or fall on its face and nothing in between. If you 
>tell me you have indeed log those errors and get 10e-12 BERT my hats off to 
>you. I have been puzzled by this 10e-xx claims for a long time and indeed very 
>interested in what others do.
>________________________________
>
>From: Alan.Hiltonnickel@xxxxxxx [mailto:Alan.Hiltonnickel@xxxxxxx]
>Sent: Fri 6/30/2006 10:57 PM
>To: Chris Cheng
>Cc: si-list@xxxxxxxxxxxxx
>Subject: Re: [SI-LIST] Re: Fibre channel interconnect margins
>
>You might want to re-check that. Yes, we have the same CRC checks and flags. 
>But after error rates get to the "expected" or "acceptable" levels, most folks 
>turn off the error reporting and let the software and hardware do what it's 
>designed to do, because the system is tolerant of errors. That's the purpose 
>of CRCs - to make systems more robust, not to make the customers help you 
>debug your systems.
>
>Otherwise, it sounds like you're telling us that you have invented a way to 
>design equipment where randomness does not happen. Hope you got a patent on 
>that! ;-)
>
>Alan
>
>
>
>
>------------------------------------------------------------------
>To unsubscribe from si-list:
>si-list-request@xxxxxxxxxxxxx with 'unsubscribe' in the Subject field
>
>or to administer your membership from a web page, go to:
>http://www.freelists.org/webpage/si-list
>
>For help:
>si-list-request@xxxxxxxxxxxxx with 'help' in the Subject field
>
>List FAQ wiki page is located at:
>                http://si-list.org/wiki/wiki.pl?Si-List_FAQ
>
>List technical documents are available at:
>                http://www.si-list.org
>
>List archives are viewable at:    
>                http://www.freelists.org/archives/si-list
>or at our remote archives:
>                http://groups.yahoo.com/group/si-list/messages
>Old (prior to June 6, 2001) list archives are viewable at:
>                http://www.qsl.net/wb6tpu
> 
>
>
>
>
>------------------------------------------------------------------
>To unsubscribe from si-list:
>si-list-request@xxxxxxxxxxxxx with 'unsubscribe' in the Subject field
>
>or to administer your membership from a web page, go to:
>http://www.freelists.org/webpage/si-list
>
>For help:
>si-list-request@xxxxxxxxxxxxx with 'help' in the Subject field
>
>List FAQ wiki page is located at:
>                http://si-list.org/wiki/wiki.pl?Si-List_FAQ
>
>List technical documents are available at:
>                http://www.si-list.org
>
>List archives are viewable at:     
>               http://www.freelists.org/archives/si-list
>or at our remote archives:
>               http://groups.yahoo.com/group/si-list/messages
>Old (prior to June 6, 2001) list archives are viewable at:
>               http://www.qsl.net/wb6tpu
>  
>
>  
>

-- 
Alan Hilton-Nickel
Signal Integrity Engineer
Sun Microsystems Inc.
Netra Systems and Networking
Newark, CA

------------------------------------------------------------------
To unsubscribe from si-list:
si-list-request@xxxxxxxxxxxxx with 'unsubscribe' in the Subject field

or to administer your membership from a web page, go to:
http://www.freelists.org/webpage/si-list

For help:
si-list-request@xxxxxxxxxxxxx with 'help' in the Subject field

List FAQ wiki page is located at:
                http://si-list.org/wiki/wiki.pl?Si-List_FAQ

List technical documents are available at:
                http://www.si-list.org

List archives are viewable at:     
                http://www.freelists.org/archives/si-list
or at our remote archives:
                http://groups.yahoo.com/group/si-list/messages
Old (prior to June 6, 2001) list archives are viewable at:
                http://www.qsl.net/wb6tpu
  

Other related posts: