[School-GNU-Linux] Re The battle for wireless network drivers

Vimal,
Nice revelation. In fact the newly announced HP-Microsoft strategic tie-up also 
is a pointer to the growing nexus between hardware manufacturers and Microsoft.

See http://www.expressindia.com/fullstory.php?newsid=78167 

CK Raju
----------
Strategies are invoked when there is a real, substantial threat.
Is Microsoft now in a state of panic ?


--------Original Message -------
 The battle for wireless network drivers
 <javascript:void
window.open('http://www.thejemreport.com/mambo/index2.php?option=com_content&do_pdf=1&id=293',
'win2',
'status=no,toolbar=no,scrollbars=yes,titlebar=no,menubar=no,resizable=yes,width=640,height=480,directories=no,location=no');>
 <javascript:void
window.open('http://www.thejemreport.com/mambo/index2.php?option=com_content&task=view&id=293',
'win2',
'status=no,toolbar=no,scrollbars=yes,titlebar=no,menubar=no,resizable=yes,width=640,height=480,directories=no,location=no');>
 <javascript:void
window.open('http://www.thejemreport.com/mambo/index2.php?option=com_content&task=emailform&id=293',
'win2',
'status=no,toolbar=no,scrollbars=yes,titlebar=no,menubar=no,resizable=yes,width=400,height=250,directories=no,location=no');>
  Written by Jem Matzan


  Dec 20, 2006 at 08:52 PM

  BSD and Linux programmers have had a lot of success in creating drivers
for new computer hardware in a timely manner, but much of their effort has
been without the support of major hardware manufacturers. Intel, Marvell,
Texas Instruments and Broadcom, though separate and competing entities, seem
by one consent to prevent non-Microsoft operating systems from working
properly with some of their most widely-used network chips. To find out more
about this situation, I interviewed representatives from network chip
manufacturers and programmers from free software operating systems. Their
answers are below.
<http://digg.com/submit?phase=2&url=http://www.thejemreport.com/mambo/content/view/293&title=The%20battle%20for%20wireless%20network%20chip%20drivers&bodytext=Why%20is%20it%20so%20difficult%20to%20get%20good%20drivers%20for%20wireless%20network%20cards?%20This%20article%20offers%20some%20explanations,%20including%20comments%20from%20wireless%20driver%20developers%20and%20wireless%20chip%20manufacturers.&topic=linux_unix>
<http://www.thejemreport.com/phpads/adclick.php?n=a2c863ca> The problem

If you have not had any trouble with network drivers recently, you probably
aren't aware of the problem that this article addresses, so here's a quick
overview: The increasing need for higher network chip performance and lower
cost of manufacturing has encouraged companies that create computer network
chips to abstract software that was previously permanently stored on the
chip. When this software is integrated with the hardware, you don't have to
think about it -- it works without any extra effort, and all you need is a
driver so that your operating system can interact with it. If hardware
specifications are not provided, the device can be reverse-engineered to
create a driver from scratch. Programmers generally do this by playing with
the hardware registers until they figure out how to interact with it.

When the software is abstracted from the hardware, it changes from an
invisible program stored on a memory chip into a file that must be loaded
into the network chip's memory through the driver. Generally this software
(no matter where it resides) is called *firmware*. In this case the driver
does not interface with the hardware directly; it only does so through the
firmware. In this scenario it is impossible to reverse-engineer the hardware
because it is essentially brain-dead without its firmware -- all it knows
how to do is load the firmware. Once the file is loaded into the network
chip, then the hardware knows how to be a network device. Essentially the
firmware is its own device-specific operating system. Programmers are unable
to reverse-engineer this kind of hardware because the only registers they
can play with are the ones that load the firmware; they can, however,
reverse-engineer the firmware interface once it is loaded. That's still not
an easy thing to do, but even if it weren't so complicated, many operating
system developers don't want to write their own device firmware -- all they
want to do is write a driver that can load and interface with it.

The first challenge for operating system developers is obtaining the right
to distribute the firmware file, which some manufacturers will not allow
without significant restriction. Firmware is not operating system-specific,
so the same firmware file can be used with any OS, provided it has a driver
that can interface with it. So the second challenge is creating such a
driver, which requires firmware interface documentation. Not surprisingly,
many of the same manufacturers that prohibit or restrict firmware
distribution also won't provide interface documentation.

This problem is most prevalent with wireless network chips, but it's also
creeping into the wired variety as well. Most notably, Broadcom has
developed a new PCI Express 10/100/1000 LAN
card<http://www.broadcom.com/products/Enterprise-Networking/Gigabit-Ethernet-Controllers/BCM5751>that
uses several discrete MIPS processors, all of which require
proprietary
firmware to be loaded. According to a network driver programmer I spoke
with, they are employed as such: One of the processors is responsible for
loading the firmware onto the other chips; one sends network packets; one
receives packets; one tracks packet state to perform TCP offload assistance
and various other things; one handles negotiating with the host CPU as it
puts/removes packets in/out of the descriptor rings. In this case, all of
the firmware fits into an 87k file, but firmware files can be considerably
larger: Intel's firmware files are just under half a megabyte, and the
firmware for the Alteon Networks Tigon II network controller is in the
vicinity of 2MB.

Unrestricted redistribution of firmware files is satisfactory for some open
source operating system projects like OpenBSD, FreeBSD, and many varieties
of GNU/Linux, but others like Fedora Core and Debian demand an entirely free
software environment, so redistribution of the firmware without the ability
to modify and distribute the source code is prohibited. The standard
response to this from the Free Software Foundation is to reverse-engineer
the device and provide free firmware. So even though it is very difficult --
almost impossible -- to do it in the absence of documentation, could such
firmware eventually be reverse-engineered? The development team that works
on the OpenBSD operating system has a lot of experience with
reverse-engineering, but both project leader Theo de Raadt and OpenBSD
network driver programmer Jonathan Gray agree that such work would be
impractical. Of reverse-engineering firmware and the hardware that it runs
on, de Raadt told me, "We can sometimes reverse-engineer how to talk to a
device... some are worse than others... but imagine reverse engineering the
firmware of 300-400 devices on the market today! Behind their little
ARM/MIPS buses, they are a *no man's land* of undocumented-ness and bugs;
hundreds and hundreds of bugs created almost all by the realities of
'time-to-market pressures.'" The issue, in other words, is made worse by
hastily-designed hardware that doesn't work as it should, and requires
specific workarounds in the firmware and/or driver. Sometimes manufacturers
provide patches or documentation for driver programmers; sometimes they
don't.

So instead of lobbying for documentation to write open source firmware, de
Raadt would prefer to simply have the right to freely distribute necessary
proprietary firmware files with his operating system, along with correct
firmware interface documentation so that a driver can be created, and
information from the manufacturer regarding bug workarounds. Many network
chip manufacturers stubbornly refuse to grant these requests, however. Theo
de Raadt told me in an email, "Our efforts to do more wireless involves a
few approaches. We reverse-engineer what we can. We borrow from other
people's reverse-engineering lessons where we can, for instance,
prism54.org<http://www.prism54.org/>is a Linux team, but their
reverse-engineer work has resulted in knowledge
which we can obviously use to write a BSD driver. And finally, we dialogue
directly with vendors to get more free access to documentation, early access
to hardware, or firmware distribution rights (sometimes there is just no
other way). Some vendors (in particular Ralink or Realtek) will even give us
hardware before it goes on sale. Some give us documentation, some give us
code. But largely a lot of American vendors are still stupidly resisting
giving anything. In any case, all these efforts together now mean that we
have more wireless support in OpenBSD than all the Linux distributions.
Maybe even combined!"

According to Jonathan Gray, the drivers that OpenBSD currently has that
require firmware that vendors won't let the OpenBSD Project distribute are:

   - *acx* (4) - TI ACX100/ACX111 IEEE 802.11a/b/g wireless network
   device
   - *an* (4) - Aironet Communications 4500/4800 IEEE 802.11FH/b
   - *atw* (4) - ADMtek ADM8211 IEEE 802.11b
   - *awi* (4) - BayStack 650 IEEE 802.11FH PCMCIA
   - *bcw* (4) - Broadcom IEEE 802.11b/g (this driver is still under
   development and does not currently work)
   - *ipw* (4) - Intel PRO/Wireless 2100 IEEE 802.11b wireless network
   device
   - *iwi* (4) - Intel PRO/Wireless 2200BG/2225BG/2915ABG IEEE 802.11a/b/g
   wireless network device
   - *wpi* (4) - Intel PRO/Wireless 3945ABG IEEE 802.11a/b/g
   - *malo* (4) - Marvell Libertas IEEE 802.11b/g
   - *pgt* (4) - Connexant/Intersil Prism GT Full-MAC IEEE 802.11a/b/g
   wireless network device
   - *uath* (4) - Atheros USB IEEE 802.11a/b/g
   - *wpi* (4) - Intel PRO/Wireless 3945ABG IEEE 802.11a/b/g wireless
   network device

And the wireless devices that either do not require firmware, or that have
runtime firmware that OpenBSD is allowed to distribute:

   - *ath* (4) - Atheros IEEE 802.11a/b/g
   - *atu* (4) - Atmel AT76C50x USB IEEE 802.11b wireless network device
   - *ral, ural* (4) - Ralink Technology IEEE 802.11a/b/g wireless
   network device (2nd gen 802.11 Ralink)
   - *rum* (4) - Ralink Technology USB IEEE 802.11a/b/g wireless network
   device
   - *ray* (4) - Raytheon Raylink/WebGear Aviator IEEE 802.11FH
   - *rtw* (4) - Realtek RTL8180L IEEE 802.11b
   - *wi* (4) - WaveLAN/IEEE, PRISM 2-3, and Spectrum24 IEEE 802.11b
   - *zyd* (4) - Zydas ZD1211 USB IEEE 802.11b/g wireless network device

Intel requires that people who use its Centrino wireless firmware submit to
a lengthy license
agreement<http://ipw2100.sourceforge.net/firmware.php?fid=4>before
downloading and using it. While the majority of end-users may just
scroll down and click the "I Agree" link to get past it, the process is not
quite so simple for free software operating systems, which would have to
provide the same license hurdle for every one of their users, and agree not
to modify any of the driver header code that Intel provides. Commercial
desktop GNU/Linux distributions like SUSE, Mandriva, and Linspire already do
this, or have independent distribution agreements with Intel.
When reverse-engineering works

Reyk Floeter and various other OpenBSD programmers managed to write a free
replacement for the proprietary Atheros hardware abstraction layer (HAL)
called ar5k <http://www.openbsd.org/cgi-bin/cvsweb/src/sys/dev/ic/ar5xxx.h>or
"OpenHAL." The HAL isn't firmware; it is a form of abstracted device
driver that actually loads into the operating system kernel through a small
amount of driver code. The big difference between a HAL and a firmware is
where the code resides; if it's on the device, it's firmware, and if it is
loaded into the operating system's kernel, it's HAL.

ar5k works with many Atheros-based wireless cards and has been examined and
recently given a clean bill of copyright health from the Software Freedom
Law Center <http://lkml.org/lkml/2006/11/14/357>. Mysteriously, it has
virtually no support from the Madwifi Project <http://madwifi.org/>, which
is the development team responsible for creating Atheros drivers for
Linux-based operating systems. Madwifi continues to primarily support the
proprietary Atheros HAL, though there is an old and uninformative page in
the Madwifi Wiki <http://madwifi.org/wiki/OpenHAL> about it. The email
addresses listed for the Madwifi developers either bounce or elicit no
response to requests for comments on why there is no apparent effort to use
the open source HAL in Madwifi. So if it exists, it's free-as-in-rights, and
it works, then why doesn't Madwifi use ar5k instead of the proprietary HAL?
It may very well be pressure from Atheros that keeps ar5k out of Madwifi,
but no one at Madwifi or Atheros would talk to me about it.
The manufacturers speak -- or don't

In order to find out why network chip manufacturers are so polarized in
their support of free software operating systems, I made contact with
company representatives at Atheros, Intel, Marvell, Atmel, Ralink, Texas
Instruments, Broadcom, and Realtek. Not surprisingly, the manufacturers who
shun operating system programmers also seem to be reluctant to talk to the
press.
Atheros

After weeks of repeated requests and pleas for any kind of response just to
verify that the PR email address works, an Atheros representative told me
that she was unable to find anyone at the company who was qualified and
willing to comment for this article.
Intel

Intel punted me to different people a few times, then after a short delay
and a bizarre inquisition into my professional background and "intentions"
in writing this article, told me that the company had nothing to say on the
matter of wireless firmware distribution rights and interface documentation.
Considering Intel's outstanding PR record and its general willingness to
provide hardware documentation for the PCI chipsets and drive controllers
that it makes, this behavior is unusual. One of the questions I asked Intel
was if it felt its uncooperativeness with free software developers was in
direct conflict with this
presentation<http://developer.osdl.org/dev/opendrivers/summit2006/james_ketrenos.pdf>about
the importance of participating in and supporting open source software
development that Intel engineer James Ketrenos gave last summer. In it,
Ketrenos says several things in favor of corporate cooperation with open
source software developers, including these points:


   - Enable the community to do as much as possible
   - Only keep internal those things that the community can not
   contribute to (Example: Certification testing)
   - If you need to keep IP closed source (for example some whiz-bang
   algorithm), document the hardware sufficiently that the community can
   provide their own.
   - Treat the community as if they were a member of your internal team

Broadcom

The Broadcom press relations person I spoke with very much wanted to help
me, but could not find anyone at Broadcom who felt qualified to comment.
Marvell

I have already written about the substandard responsiveness of Marvell's
outsourced PR agency
<http://www.thejemreport.com/mambo/content/view/286/>(see the bottom
of the linked article).
Texas Instruments

An internal Texas Instruments press relations person was very responsive to
my requests for comment at first, then punted me to an outsourced PR agency
which, in turn, ignored several emails asking for information. After more
than two weeks of repeated inquiries, a representative of the PR agency
informed me that she was unable to find anyone at Texas Instruments who was
qualified to comment on TI's policies on providing hardware documentation
and firmware redistribution agreements.
Ralink Technologies

I had no trouble getting through to Ralink, where I spoke with company
representative Lillian Chiu.

*Ralink has been responsive to requests for hardware documentation without
requiring an NDA. Why can Ralink do this when competing manufacturers such
as Intel and Marvell require non-disclosure agreements?*

*Lillian Chiu:* It's our philosophy to spread the technology without border,
along with high performance and low cost.

*Because of Ralink's cooperativeness with projects like OpenBSD and Linux
kernel developers, Ralink's products tend to be very well supported in
so-called "alternative" operating systems. Do you see this as a competitive
advantage? Does Ralink sell more network products as a result?*

*LC:* Our customers have often provided positive feedback for our
best-in-class device support. We see it as a win-win situation where
advanced users get more flexibility while we sell more products.

*If a programmer needs to access Ralink hardware documentation, whom at
Ralink should they contact?*

*LC:* Please visit Ralink Web site at www.ralinktech.com for details under
/support/forums.
Atmel Inc.

Richard Bisset, product marketing director of the multimedia and
communications group, offered several minutes of his time to answer some
questions about wireless drivers and firmware.

*Atmel has been responsive to requests for hardware documentation without
requiring an NDA. Why can Atmel do this when competing manufacturers such as
Intel and Marvell require non-disclosure agreements?*

*Richard Bisset:* For some things we do require non-disclosure agreements,
but we are generally able to provide the API documentation and the firmware
driver interface specifications for our hardware. As to why others may not
be able to do this... well, our software is developed in-house, but others
might out outsource their driver development to third-party companies, so
they may not even have the documentation that a programmer requests.

As to why Intel might be so secretive with this information if they do have
it, I can't say. Perhaps because more and more features are actually being
moved from the firmware to the driver, some companies are being more guarded
than others. Additionally, as the Centrino laptop processor includes
wireless, it may be they don't want to give up any potential IP that could
help their competition -- I don't really know for sure.

*Because of Atmel's cooperativeness with projects like OpenBSD and Linux
kernel developers, Atmel's products tend to be very well supported in
so-called "alternative" operating systems. Do you see this as a competitive
advantage? Does Atmel sell more network products as a result?*
*RB:* Well, if you look at the success of Intel, Marvell, Broadcom, etc. --
the big players in the wireless industry -- they are successful with the OEM
and peripheral markets. As the embedded markets generally require more
technical support, they probably don't care much about getting another
50,000 or 100,000 units sold and can be tight with their APIs. We were late
to market with our 802.11g products, so we missed the OEM opportunities, and
with wireless, you've only got three real chances for success: you can be
first to market with a technology, or you can have valuable and unique
features that no one else has and the market wants, or you can have the
lowest price. Atmel wasn't first, didn't have any new unique features, and
wasn't the cheapest, either. With the PC and OEM markets being somewhat
locked out, we repositioned to focus on the embedded space where the market
was experiencing and predicting large growth. In the embedded market, if you
don't get documentation to developers, then you both fail.

*What's the big secret with device firmware, then?*

*RB:* Firmware is generally running on the chip itself, as opposed to the
host. The code is written tightly coupled to the chip it is running on and
implements/enables functions that the hardware supports. Therefore, there
generally is a lot of IP within the firmware. Our firmware is loaded from an
external flash into internal SRAM, but more and more semiconductor
manufacturers are moving firmware functions to the driver.

We usually provide driver source code, and we try to put it under the GPL if
possible, so that's usually good enough if you want to write your own
driver. If you want to see more than that, we generally require an NDA, or
if you're an embedded customer, we provide reference platforms.

*So the firmware is so secret that you have to sign an NDA to see it, but
not secret enough that some stranger who works on an obscure operating
system can see it with merely a signature?*

*RB:* It is kind of a strange situation, yes, and sometimes we run into
people who are reluctant to sign NDAs. It really depends on the customer and
target application. We don't require an NDA to see hardware APIs, and I
think that's what you're talking about with regard to documentation.

*If a programmer needs to access Atmel hardware documentation, whom at Atmel
should they contact?*

*RB:* We have contact forms through our Web site
<http://www.atmel.com/>where you can make requests, and they are
directed to the proper person at
Atmel.
Realtek Inc.

After a few emails, I made contact with Tracy Ho, a Taiwan-based
representative for Realtek.

*Realtek has reportedly been responsive to requests for hardware
documentation without requiring a non-disclosure agreement (NDA). Why can
Realtek do this when competing manufacturers such as Intel and Marvell
require NDAs?*

*Tracy Ho:* For some years Realtek has been one of the largest NIC/LOM
solution providers. One of the major reasons for this is that Realtek takes
customer service as one of its top priorities. We aim to work with our
customers as partners in a mutually cooperative environment. Our product
sales and technical support teams are well-recognized by our customers and
we strive to provide them with the convenience of flexibility and real-time
support. One of the ways we do this is by releasing general hardware
documents without requiring a NDA.

*Because of Realtek's cooperativeness with projects like OpenBSD and Linux
kernel developers, Realtek's products tend to be very well supported in
so-called "alternative" operating systems. Do you see this as a competitive
advantage? Does Realtek sell more network products as a result?*

*TH:* Realtek has been working very closely with various operating system
providers pretty much since the company was first started, and we do
consider this to be one of our competitive advantages. Over the last decade,
the great expansion of networked devices throughout the home and work
environment, and the use of open-source operating systems in many such
devices, has definitely helped us expand market share.

*If a programmer needs to access Realtek hardware documentation, whom at
Realtek should they contact?*

*TH:* For most hardware documentation, programmers can access Realtek's
website <http://www.realtek.com.tw/>. Any other questions can be directed to
our technical support teams (contact information available from the "Contact
Us" link on our website).
All we need to do is make sure we keep talking

Some of the non-responsiveness of manufacturers may just be bad PR work, but
the same companies that wouldn't talk to me have also refused to reply to
free software programmers who have requested the same information. The
impression I got from most of these companies (excepting Intel) was that
they were not at all prepared to deal with the issues of firmware
redistribution rights and hardware API documentation requests. That they
have ignored free software programmers' requests is not necessarily a sign
of unwillingness to participate, but perhaps a general sense of confusion as
to *how* they are able to help. No one seems to know whom to talk to at the
company, and in some cases the proper documentation may not exist -- or it
may belong to yet another company that the hardware manufacturer outsourced
the firmware development to.

On the other hand, it was just as difficult to contact and get comments from
the Madwifi developers, and it seems that each individual wireless driver in
the Linux kernel has different people working on it. Even if one of the
above companies wanted to provide the appropriate materials to create free
drivers and firmware, how would they know whom to contact? The irony in this
story is that the heart of the problem lies in a lack of communication, but
not between operating systems and network devices -- between hardware
manufacturers and the software developers who are trying to support their
devices.

Discuss this article or get technical support on our
forum.<http://www.tjrforum.com/>

*Copyright 2006 Jem Matzan.*
 [image: Creative Commons
License]<http://creativecommons.org/licenses/by-nc-nd/2.5/>

This work is licensed under a Creative Commons
Attribution-NonCommercial-NoDerivs 2.5
License<http://creativecommons.org/licenses/by-nc-nd/2.5/>


-- 
Free Software, Free Society <http://fsfs.hipatia.net>
<table class="contentpaneopen"><tbody><tr><td class="contentheading" 
width="100%"><font size="4">
                                The battle for wireless network drivers</font>  
                                                        </td>
                                                        <td 
class="buttonheading" align="right">
                        <a href="javascript:void 
window.open('http://www.thejemreport.com/mambo/index2.php?option=com_content&do_pdf=1&id=293',
 'win2', 
'status=no,toolbar=no,scrollbars=yes,titlebar=no,menubar=no,resizable=yes,width=640,height=480,directories=no,location=no');"
 title="PDF">

                        <br>                    </a>
                        </td>
                                                        <td 
class="buttonheading" align="right">
                                <a href="javascript:void 
window.open('http://www.thejemreport.com/mambo/index2.php?option=com_content&task=view&id=293',
 'win2', 
'status=no,toolbar=no,scrollbars=yes,titlebar=no,menubar=no,resizable=yes,width=640,height=480,directories=no,location=no');"
 title="Print">

                                <br>                            </a>
                                </td>
                                                        <td 
class="buttonheading" align="right">
                        <a href="javascript:void 
window.open('http://www.thejemreport.com/mambo/index2.php?option=com_content&task=emailform&id=293',
 'win2', 
'status=no,toolbar=no,scrollbars=yes,titlebar=no,menubar=no,resizable=yes,width=400,height=250,directories=no,location=no');"
 title="E-mail">

                        <br>                    </a>
                        </td>
                                                </tr>
                        </tbody></table>
                        
                
                                <table class="contentpaneopen"><tbody><tr>
                        <td colspan="2" align="left" valign="top" width="70%">
                        <span class="small">
                        Written by Jem Matzan                   </span>
                          <br><br>
                        </td>
                </tr>
                                        <tr>
                                <td colspan="2" class="createdate" valign="top">
                                Dec 20, 2006 at 08:52 PM<br><br>                
                </td>
                        </tr>
                                        <tr>
                        <td colspan="2" valign="top">
                        <p>BSD
and Linux programmers have had a lot of success in creating drivers for
new computer hardware in a timely manner, but much of their effort has
been without the support of major hardware manufacturers. Intel,
Marvell, Texas Instruments and Broadcom, though separate and competing
entities, seem by one consent to prevent non-Microsoft operating
systems from working properly with some of their most widely-used
network chips. To find out more about this situation, I interviewed
representatives from network chip manufacturers and programmers from
free software operating systems. Their answers are below.</p>
<a 
href="http://digg.com/submit?phase=2&url=http://www.thejemreport.com/mambo/content/view/293&title=The%20battle%20for%20wireless%20network%20chip%20drivers&bodytext=Why%20is%20it%20so%20difficult%20to%20get%20good%20drivers%20for%20wireless%20network%20cards?%20This%20article%20offers%20some%20explanations,%20including%20comments%20from%20wireless%20driver%20developers%20and%20wireless%20chip%20manufacturers.&topic=linux_unix";>
</a>

<table style="margin-left: 0px; float: left;"><tbody><tr><td>

<noscript><a href="http://www.thejemreport.com/phpads/adclick.php?n=a2c863ca"; 
target="_blank"><img 
src="http://www.thejemreport.com/phpads/adview.php?what=zone:2&n=a2c863ca"; 
border="0" alt=""></a></noscript>
</td></tr></tbody></table>

<h1>The problem</h1>

<p>If you have not had any trouble with network drivers recently, you
probably aren't aware of the problem that this article addresses, so
here's a quick overview: The increasing need for higher network chip
performance and lower cost of manufacturing has encouraged companies
that create computer network chips to abstract software that was
previously permanently stored on the chip. When this software is
integrated with the hardware, you don't have to think about it -- it
works without any extra effort, and all you need is a driver so that
your operating system can interact with it. If hardware specifications
are not provided, the device can be reverse-engineered to create a
driver from scratch. Programmers generally do this by playing with the
hardware registers until they figure out how to interact with it.</p>

<p>When the software is abstracted from the hardware, it changes from
an invisible program stored on a memory chip into a file that must be
loaded into the network chip's memory through the driver. Generally
this software (no matter where it resides) is called <em>firmware</em>.
In this case the driver does not interface with the hardware directly;
it only does so through the firmware. In this scenario it is impossible
to reverse-engineer the hardware because it is essentially brain-dead
without its firmware -- all it knows how to do is load the firmware.
Once the file is loaded into the network chip, then the hardware knows
how to be a network device. Essentially the firmware is its own
device-specific operating system. Programmers are unable to
reverse-engineer this kind of hardware because the only registers they
can play with are the ones that load the firmware; they can, however,
reverse-engineer the firmware interface once it is loaded. That's still
not an easy thing to do, but even if it weren't so complicated, many
operating system developers don't want to write their own device
firmware -- all they want to do is write a driver that can load and
interface with it.</p>

<p>The first challenge for operating system developers is obtaining the
right to distribute the firmware file, which some manufacturers will
not allow without significant restriction. Firmware is not operating
system-specific, so the same firmware file can be used with any OS,
provided it has a driver that can interface with it. So the second
challenge is creating such a driver, which requires firmware interface
documentation. Not surprisingly, many of the same manufacturers that
prohibit or restrict firmware distribution also won't provide interface
documentation.</p>

<p>This problem is most prevalent with wireless network chips, but it's
also creeping into the wired variety as well. Most notably, Broadcom
has developed <a 
href="http://www.broadcom.com/products/Enterprise-Networking/Gigabit-Ethernet-Controllers/BCM5751";
 rel="nofollow" target="_blank">a new PCI Express 10/100/1000 LAN card</a>
that uses several discrete MIPS processors, all of which require
proprietary firmware to be loaded. According to a network driver
programmer I spoke with, they are employed as such: One of the
processors is responsible for loading the firmware onto the other
chips; one sends network packets; one receives packets; one tracks
packet state to perform TCP offload assistance and various other
things; one handles negotiating with the host CPU as it puts/removes
packets in/out of the descriptor rings. In this case, all of the
firmware fits into an 87k file, but firmware files can be considerably
larger: Intel's firmware files are just under half a megabyte, and the
firmware for the Alteon Networks Tigon II network controller is in the
vicinity of 2MB.</p>

<p>Unrestricted redistribution of firmware files is satisfactory for
some open source operating system projects like OpenBSD, FreeBSD, and
many varieties of GNU/Linux, but others like Fedora Core and Debian
demand an entirely free software environment, so redistribution of the
firmware without the ability to modify and distribute the source code
is prohibited. The standard response to this from the Free Software
Foundation is to reverse-engineer the device and provide free firmware.
So even though it is very difficult -- almost impossible -- to do it in
the absence of documentation, could such firmware eventually be
reverse-engineered? The development team that works on the OpenBSD
operating system has a lot of experience with reverse-engineering, but
both project leader Theo de Raadt and OpenBSD network driver programmer
Jonathan Gray agree that such work would be impractical. Of
reverse-engineering firmware and the hardware that it runs on, de Raadt
told me, "We can sometimes reverse-engineer how to talk to a device...
some are worse than others... but imagine reverse engineering the
firmware of 300-400 devices on the market today! Behind their little
ARM/MIPS buses, they are a <em>no man's land</em> of undocumented-ness
and bugs; hundreds and hundreds of bugs created almost all by the
realities of 'time-to-market pressures.'" The issue, in other words, is
made worse by hastily-designed hardware that doesn't work as it should,
and requires specific workarounds in the firmware and/or driver.
Sometimes manufacturers provide patches or documentation for driver
programmers; sometimes they don't.</p>

<p>So instead of lobbying for documentation to write open source
firmware, de Raadt would prefer to simply have the right to freely
distribute necessary proprietary firmware files with his operating
system, along with correct firmware interface documentation so that a
driver can be created, and information from the manufacturer regarding
bug workarounds. Many network chip manufacturers stubbornly refuse to
grant these requests, however. Theo de Raadt told me in an email, "Our
efforts to do more wireless involves a few approaches. We
reverse-engineer what we can. We borrow from other people's
reverse-engineering lessons where we can, for instance, <a 
href="http://www.prism54.org/"; rel="nofollow" target="_blank">prism54.org</a>
is a Linux team, but their reverse-engineer work has resulted in
knowledge which we can obviously use to write a BSD driver. And
finally, we dialogue directly with vendors to get more free access to
documentation, early access to hardware, or firmware distribution
rights (sometimes there is just no other way). Some vendors (in
particular Ralink or Realtek) will even give us hardware before it goes
on sale. Some give us documentation, some give us code. But largely a
lot of American vendors are still stupidly resisting giving anything.
In any case, all these efforts together now mean that we have more
wireless support in OpenBSD than all the Linux distributions. Maybe
even combined!"</p>

<p>According to Jonathan Gray, the drivers that OpenBSD currently has
that require firmware that vendors won't let the OpenBSD Project
distribute are:</p>

<ul><li><strong>acx</strong> (4) - TI ACX100/ACX111 IEEE 802.11a/b/g wireless 
network device</li><li><strong>an</strong> (4) - Aironet Communications 
4500/4800 IEEE 802.11FH/b</li><li><strong>atw</strong> (4) - ADMtek ADM8211 
IEEE 
802.11b</li><li><strong>awi</strong> (4) - BayStack 650 IEEE 802.11FH 
PCMCIA</li><li><strong>bcw</strong> (4) - Broadcom IEEE 802.11b/g (this driver 
is still under development and does not currently work)</li><li><strong>
ipw</strong> (4) - Intel PRO/Wireless 2100 IEEE 802.11b wireless network 
device</li><li><strong>iwi</strong> (4) - Intel PRO/Wireless 
2200BG/2225BG/2915ABG IEEE 802.11a/b/g wireless network 
device</li><li><strong>wpi</strong>
 (4) - Intel PRO/Wireless 3945ABG IEEE 
802.11a/b/g</li><li><strong>malo</strong> (4) - Marvell Libertas IEEE 
802.11b/g</li><li><strong>pgt</strong> (4) - Connexant/Intersil Prism GT 
Full-MAC IEEE 802.11a/b/g wireless network device
</li><li><strong>uath</strong> (4) - Atheros USB IEEE 
802.11a/b/g</li><li><strong>wpi</strong> (4) - Intel PRO/Wireless 3945ABG IEEE 
802.11a/b/g wireless network device</li></ul>

<p>And the wireless devices that either do not require firmware, or
that have runtime firmware that OpenBSD is allowed to distribute:</p>

<ul><li><strong>ath</strong> (4) - Atheros IEEE 
802.11a/b/g</li><li><strong>atu</strong> (4) - Atmel AT76C50x USB IEEE 802.11b 
wireless network device</li><li><strong>ral, ural</strong> (4) - Ralink 
Technology IEEE 802.11a
/b/g wireless network device (2nd gen 802.11 
Ralink)</li><li><strong>rum</strong> (4) - Ralink Technology USB IEEE 
802.11a/b/g wireless network device</li><li><strong>ray</strong> (4) - Raytheon 
Raylink/WebGear Aviator IEEE 
802.11FH</li><li><strong>rtw</strong> (4) - Realtek RTL8180L IEEE 
802.11b</li><li><strong>wi</strong> (4) - WaveLAN/IEEE, PRISM 2-3, and 
Spectrum24 IEEE 802.11b</li><li><strong>zyd</strong> (4) - Zydas ZD1211 USB 
IEEE 802.11b
/g wireless network device</li></ul>

<p>Intel requires that people who use its Centrino wireless firmware submit to 
a <a href="http://ipw2100.sourceforge.net/firmware.php?fid=4"; rel="nofollow" 
target="_blank">lengthy license agreement</a>
before downloading and using it. While the majority of end-users may
just scroll down and click the "I Agree" link to get past it, the
process is not quite so simple for free software operating systems,
which would have to provide the same license hurdle for every one of
their users, and agree not to modify any of the driver header code that
Intel provides. Commercial desktop GNU/Linux distributions like SUSE,
Mandriva, and Linspire already do this, or have independent
distribution agreements with Intel.</p>

<h1>When reverse-engineering works</h1>

<p>Reyk Floeter and various other OpenBSD programmers managed to write
a free replacement for the proprietary Atheros hardware abstraction
layer (HAL) called <a 
href="http://www.openbsd.org/cgi-bin/cvsweb/src/sys/dev/ic/ar5xxx.h"; 
rel="nofollow" target="_blank">ar5k</a>
or "OpenHAL." The HAL isn't firmware; it is a form of abstracted device
driver that actually loads into the operating system kernel through a
small amount of driver code. The big difference between a HAL and a
firmware is where the code resides; if it's on the device, it's
firmware, and if it is loaded into the operating system's kernel, it's
HAL.</p>

<p>ar5k works with many Atheros-based wireless cards and has been examined and 
recently given <a href="http://lkml.org/lkml/2006/11/14/357"; rel="nofollow" 
target="_blank">a clean bill of copyright health from the Software Freedom Law 
Center
</a>. Mysteriously, it has virtually no support from the <a 
href="http://madwifi.org/"; rel="nofollow" target="_blank">Madwifi Project</a>,
which is the development team responsible for creating Atheros drivers
for Linux-based operating systems. Madwifi continues to primarily
support the proprietary Atheros HAL, though there is an old and
uninformative <a href="http://madwifi.org/wiki/OpenHAL"; rel="nofollow" 
target="_blank">page in the Madwifi Wiki</a>
about it. The email addresses listed for the Madwifi developers either
bounce or elicit no response to requests for comments on why there is
no apparent effort to use the open source HAL in Madwifi. So if it
exists, it's free-as-in-rights, and it works, then why doesn't Madwifi
use ar5k instead of the proprietary HAL? It may very well be pressure
from Atheros that keeps ar5k out of Madwifi, but no one at Madwifi or
Atheros would talk to me about it.</p>

<h1>The manufacturers speak -- or don't</h1>

<p>In order to find out why network chip manufacturers are so polarized
in their support of free software operating systems, I made contact
with company representatives at Atheros, Intel, Marvell, Atmel, Ralink,
Texas Instruments, Broadcom, and Realtek. Not surprisingly, the
manufacturers who shun operating system programmers also seem to be
reluctant to talk to the press.</p>

<h2>Atheros</h2>

<p>After weeks of repeated requests and pleas for any kind of response
just to verify that the PR email address works, an Atheros
representative told me that she was unable to find anyone at the
company who was qualified and willing to comment for this article.</p>

<h2>Intel</h2>

<p>Intel punted me to different people a few times, then after a short
delay and a bizarre inquisition into my professional background and
"intentions" in writing this article, told me that the company had
nothing to say on the matter of wireless firmware distribution rights
and interface documentation. Considering Intel's outstanding PR record
and its general willingness to provide hardware documentation for the
PCI chipsets and drive controllers that it makes, this behavior is
unusual. One of the questions I asked Intel was if it felt its
uncooperativeness with free software developers was in direct conflict
with <a 
href="http://developer.osdl.org/dev/opendrivers/summit2006/james_ketrenos.pdf"; 
rel="nofollow" target="_blank">this presentation</a>
about the importance of participating in and supporting open source
software development that Intel engineer James Ketrenos gave last
summer. In it, Ketrenos says several things in favor of corporate
cooperation with open source software developers, including these
points:</p>

<blockquote><ul><li>Enable the community to do as much as possible</li><li>Only 
keep internal those things that the community can not contribute to (Example: 
Certification testing)</li><li>If you need to keep IP closed source (for 
example some whiz-bang
algorithm), document the hardware sufficiently that the community can
provide their own.</li><li>Treat the community as if they were a member of your 
internal team</li></ul></blockquote>

<h2>Broadcom</h2>

<p>The Broadcom press relations person I spoke with very much wanted to
help me, but could not find anyone at Broadcom who felt qualified to
comment.</p>

<h2>Marvell</h2>

<p><a href="http://www.thejemreport.com/mambo/content/view/286/"; rel="nofollow" 
target="_blank">I have already written about the substandard responsiveness of 
Marvell's outsourced PR agency</a> (see the bottom of the linked article).
</p>

<h2>Texas Instruments</h2>

<p>An internal Texas Instruments press relations person was very
responsive to my requests for comment at first, then punted me to an
outsourced PR agency which, in turn, ignored several emails asking for
information. After more than two weeks of repeated inquiries, a
representative of the PR agency informed me that she was unable to find
anyone at Texas Instruments who was qualified to comment on TI's
policies on providing hardware documentation and firmware
redistribution agreements.</p>

<h2>Ralink Technologies</h2>

<p>I had no trouble getting through to Ralink, where I spoke with company 
representative Lillian Chiu.</p>

<p><strong><em>Ralink has been responsive to requests for hardware
documentation without requiring an NDA. Why can Ralink do this when
competing manufacturers such as Intel and Marvell require
non-disclosure agreements?</em></strong></p>

<p><strong>Lillian Chiu:</strong> It's our philosophy to spread the technology 
without border, along with high performance and low cost.</p>

<p><strong><em>Because of Ralink's cooperativeness with projects like
OpenBSD and Linux kernel developers, Ralink's products tend to be very
well supported in so-called "alternative" operating systems. Do you see
this as a competitive advantage? Does Ralink sell more network products
as a result?</em></strong></p>

<p><strong>LC:</strong> Our customers have often provided positive
feedback for our best-in-class device support. We see it as a win-win
situation where advanced users get more flexibility while we sell more
products.</p>

<p><strong><em>If a programmer needs to access Ralink hardware documentation, 
whom at Ralink should they contact?</em></strong></p>

<p><strong>LC:</strong> Please visit Ralink Web site at <a 
href="http://www.ralinktech.com";>www.ralinktech.com</a> for details under 
/support/forums.</p>

<h2>Atmel Inc.</h2>

<p>Richard Bisset, product marketing director of the multimedia and
communications group, offered several minutes of his time to answer
some questions about wireless drivers and firmware.</p>

<p><strong><em>Atmel has been responsive to requests for hardware
documentation without requiring an NDA. Why can Atmel do this when
competing manufacturers such as Intel and Marvell require
non-disclosure agreements?</em></strong></p>

<p><strong>Richard Bisset:</strong> For some things we do require
non-disclosure agreements, but we are generally able to provide the API
documentation and the firmware driver interface specifications for our
hardware. As to why others may not be able to do this... well, our
software is developed in-house, but others might out outsource their
driver development to third-party companies, so they may not even have
the documentation that a programmer requests.</p>

<p>As to why Intel might be so secretive with this information if they
do have it, I can't say. Perhaps because more and more features are
actually being moved from the firmware to the driver, some companies
are being more guarded than others. Additionally, as the Centrino
laptop processor includes wireless, it may be they don't want to give
up any potential IP that could help their competition -- I don't really
know for sure.</p>

<p><strong><em>Because of Atmel's cooperativeness with projects like
OpenBSD and Linux kernel developers, Atmel's products tend to be very
well supported in so-called "alternative" operating systems. Do you see
this as a competitive advantage? Does Atmel sell more network products
as a result?</em></strong></p>

<strong>RB:</strong> Well, if you look at the success of Intel,
Marvell, Broadcom, etc. -- the big players in the wireless industry --
they are successful with the OEM and peripheral markets. As the
embedded markets generally require more technical support, they
probably don't care much about getting another 50,000 or 100,000 units
sold and can be tight with their APIs. We were late to market with our
802.11g products, so we missed the OEM opportunities, and with
wireless, you've only got three real chances for success: you can be
first to market with a technology, or you can have valuable and unique
features that no one else has and the market wants, or you can have the
lowest price. Atmel wasn't first, didn't have any new unique features,
and wasn't the cheapest, either. With the PC and OEM markets being
somewhat locked out, we repositioned to focus on the embedded space
where the market was experiencing and predicting large growth. In the
embedded market, if you don't get documentation to developers, then you
both fail.
<p><strong><em>What's the big secret with device firmware, 
then?</em></strong></p>

<p><strong>RB:</strong> Firmware is generally running on the chip
itself, as opposed to the host. The code is written tightly coupled to
the chip it is running on and implements/enables functions that the
hardware supports. Therefore, there generally is a lot of IP within the
firmware. Our firmware is loaded from an external flash into internal
SRAM, but more and more semiconductor manufacturers are moving firmware
functions to the driver.</p>

<p>We usually provide driver source code, and we try to put it under
the GPL if possible, so that's usually good enough if you want to write
your own driver. If you want to see more than that, we generally
require an NDA, or if you're an embedded customer, we provide reference
platforms.</p>

<p><strong><em>So the firmware is so secret that you have to sign an
NDA to see it, but not secret enough that some stranger who works on an
obscure operating system can see it with merely a signature?</em></strong></p>

<p><strong>RB:</strong> It is kind of a strange situation, yes, and
sometimes we run into people who are reluctant to sign NDAs. It really
depends on the customer and target application. We don't require an NDA
to see hardware APIs, and I think that's what you're talking about with
regard to documentation.</p>

<p><strong><em>If a programmer needs to access Atmel hardware documentation, 
whom at Atmel should they contact?</em></strong></p>

<p><strong>RB:</strong> We have contact forms through <a 
href="http://www.atmel.com/"; target="_blank" rel="nofollow">our Web site</a> 
where you can make requests, and they are directed to the proper person at 
Atmel.</p>

<h2>Realtek Inc.</h2>

<p>After a few emails, I made contact with Tracy Ho, a Taiwan-based 
representative for Realtek.</p>

<p><strong><em>Realtek has reportedly been responsive to requests for
hardware documentation without requiring a non-disclosure agreement
(NDA). Why can Realtek do this when competing manufacturers such as
Intel and Marvell require NDAs?</em></strong></p>

<p><strong>Tracy Ho:</strong> For some years Realtek has been one of
the largest NIC/LOM solution providers. One of the major reasons for
this is that Realtek takes customer service as one of its top
priorities. We aim to work with our customers as partners in a mutually
cooperative environment. Our product sales and technical support teams
are well-recognized by our customers and we strive to provide them with
the convenience of flexibility and real-time support. One of the ways
we do this is by releasing general hardware documents without requiring
a NDA.</p>

<p><strong><em>Because of Realtek's cooperativeness with projects like
OpenBSD and Linux kernel developers, Realtek's products tend to be very
well supported in so-called "alternative" operating systems. Do you see
this as a competitive advantage? Does Realtek sell more network
products as a result?</em></strong></p>

<p><strong>TH:</strong> Realtek has been working very closely with
various operating system providers pretty much since the company was
first started, and we do consider this to be one of our competitive
advantages. Over the last decade, the great expansion of networked
devices throughout the home and work environment, and the use of
open-source operating systems in many such devices, has definitely
helped us expand market share.</p>

<p><strong><em>If a programmer needs to access Realtek hardware documentation, 
whom at Realtek should they contact?</em></strong></p>

<p><strong>TH:</strong> For most hardware documentation, programmers can access 
<a href="http://www.realtek.com.tw/"; rel="nofollow" target="_blank">Realtek's 
website</a>.
Any other questions can be directed to our technical support teams
(contact information available from the "Contact Us" link on our
website).</p>

<h1>All we need to do is make sure we keep talking</h1>

<p>Some of the non-responsiveness of manufacturers may just be bad PR
work, but the same companies that wouldn't talk to me have also refused
to reply to free software programmers who have requested the same
information. The impression I got from most of these companies
(excepting Intel) was that they were not at all prepared to deal with
the issues of firmware redistribution rights and hardware API
documentation requests. That they have ignored free software
programmers' requests is not necessarily a sign of unwillingness to
participate, but perhaps a general sense of confusion as to <em>how</em>
they are able to help. No one seems to know whom to talk to at the
company, and in some cases the proper documentation may not exist -- or
it may belong to yet another company that the hardware manufacturer
outsourced the firmware development to.</p>

<p>On the other hand, it was just as difficult to contact and get
comments from the Madwifi developers, and it seems that each individual
wireless driver in the Linux kernel has different people working on it.
Even if one of the above companies wanted to provide the appropriate
materials to create free drivers and firmware, how would they know whom
to contact? The irony in this story is that the heart of the problem
lies in a lack of communication, but not between operating systems and
network devices -- between hardware manufacturers and the software
developers who are trying to support their devices.</p>

<p><a href="http://www.tjrforum.com/"; target="_blank">Discuss this article or 
get technical support on our forum.</a></p>

<p><strong>Copyright 2006 Jem Matzan.</strong></p>
<table style="width: auto; height: auto; margin-left: auto; margin-right: 
auto;"><tbody><tr><td>

<a rel="license" href="http://creativecommons.org/licenses/by-nc-nd/2.5/";><img 
alt="Creative Commons License" style="border: 0px none ;" 
src="http://creativecommons.org/images/public/somerights20.png";></a>
</td></tr></tbody></table>
<table style="width: auto; height: auto; margin-left: auto; margin-right: 
auto;"><tbody><tr><td>
<p>This work is licensed under a <a rel="license" 
href="http://creativecommons.org/licenses/by-nc-nd/2.5/";>Creative Commons 
Attribution-NonCommercial-NoDerivs 2.5 
License</a></p></td></tr></tbody></table></td></tr></tbody>
</table><br><br>-- <br>Free Software, Free Society <<a 
href="http://fsfs.hipatia.net";>http://fsfs.hipatia.net</a>><br><br>

Other related posts: