[School-GNU-Linux] The battle for wireless network drivers

  • From: "Vimal Joseph" <vimal@xxxxxxxxxxxxxxxx>
  • To: schoolgnu@xxxxxxxxxxxxx
  • Date: Thu, 21 Dec 2006 14:54:04 +0530

The battle for wireless network drivers
<javascript:void
window.open('http://www.thejemreport.com/mambo/index2.php?option=com_content&do_pdf=1&id=293',
'win2',
'status=no,toolbar=no,scrollbars=yes,titlebar=no,menubar=no,resizable=yes,width=640,height=480,directories=no,location=no');>
<javascript:void
window.open('http://www.thejemreport.com/mambo/index2.php?option=com_content&task=view&id=293',
'win2',
'status=no,toolbar=no,scrollbars=yes,titlebar=no,menubar=no,resizable=yes,width=640,height=480,directories=no,location=no');>
<javascript:void
window.open('http://www.thejemreport.com/mambo/index2.php?option=com_content&task=emailform&id=293',
'win2',
'status=no,toolbar=no,scrollbars=yes,titlebar=no,menubar=no,resizable=yes,width=400,height=250,directories=no,location=no');>
 Written by Jem Matzan


 Dec 20, 2006 at 08:52 PM

 BSD and Linux programmers have had a lot of success in creating drivers
for new computer hardware in a timely manner, but much of their effort has
been without the support of major hardware manufacturers. Intel, Marvell,
Texas Instruments and Broadcom, though separate and competing entities, seem
by one consent to prevent non-Microsoft operating systems from working
properly with some of their most widely-used network chips. To find out more
about this situation, I interviewed representatives from network chip
manufacturers and programmers from free software operating systems. Their
answers are below.
<http://digg.com/submit?phase=2&url=http://www.thejemreport.com/mambo/content/view/293&title=The%20battle%20for%20wireless%20network%20chip%20drivers&bodytext=Why%20is%20it%20so%20difficult%20to%20get%20good%20drivers%20for%20wireless%20network%20cards?%20This%20article%20offers%20some%20explanations,%20including%20comments%20from%20wireless%20driver%20developers%20and%20wireless%20chip%20manufacturers.&topic=linux_unix>
<http://www.thejemreport.com/phpads/adclick.php?n=a2c863ca> The problem

If you have not had any trouble with network drivers recently, you probably
aren't aware of the problem that this article addresses, so here's a quick
overview: The increasing need for higher network chip performance and lower
cost of manufacturing has encouraged companies that create computer network
chips to abstract software that was previously permanently stored on the
chip. When this software is integrated with the hardware, you don't have to
think about it -- it works without any extra effort, and all you need is a
driver so that your operating system can interact with it. If hardware
specifications are not provided, the device can be reverse-engineered to
create a driver from scratch. Programmers generally do this by playing with
the hardware registers until they figure out how to interact with it.

When the software is abstracted from the hardware, it changes from an
invisible program stored on a memory chip into a file that must be loaded
into the network chip's memory through the driver. Generally this software
(no matter where it resides) is called *firmware*. In this case the driver
does not interface with the hardware directly; it only does so through the
firmware. In this scenario it is impossible to reverse-engineer the hardware
because it is essentially brain-dead without its firmware -- all it knows
how to do is load the firmware. Once the file is loaded into the network
chip, then the hardware knows how to be a network device. Essentially the
firmware is its own device-specific operating system. Programmers are unable
to reverse-engineer this kind of hardware because the only registers they
can play with are the ones that load the firmware; they can, however,
reverse-engineer the firmware interface once it is loaded. That's still not
an easy thing to do, but even if it weren't so complicated, many operating
system developers don't want to write their own device firmware -- all they
want to do is write a driver that can load and interface with it.

The first challenge for operating system developers is obtaining the right
to distribute the firmware file, which some manufacturers will not allow
without significant restriction. Firmware is not operating system-specific,
so the same firmware file can be used with any OS, provided it has a driver
that can interface with it. So the second challenge is creating such a
driver, which requires firmware interface documentation. Not surprisingly,
many of the same manufacturers that prohibit or restrict firmware
distribution also won't provide interface documentation.

This problem is most prevalent with wireless network chips, but it's also
creeping into the wired variety as well. Most notably, Broadcom has
developed a new PCI Express 10/100/1000 LAN
card<http://www.broadcom.com/products/Enterprise-Networking/Gigabit-Ethernet-Controllers/BCM5751>that
uses several discrete MIPS processors, all of which require
proprietary
firmware to be loaded. According to a network driver programmer I spoke
with, they are employed as such: One of the processors is responsible for
loading the firmware onto the other chips; one sends network packets; one
receives packets; one tracks packet state to perform TCP offload assistance
and various other things; one handles negotiating with the host CPU as it
puts/removes packets in/out of the descriptor rings. In this case, all of
the firmware fits into an 87k file, but firmware files can be considerably
larger: Intel's firmware files are just under half a megabyte, and the
firmware for the Alteon Networks Tigon II network controller is in the
vicinity of 2MB.

Unrestricted redistribution of firmware files is satisfactory for some open
source operating system projects like OpenBSD, FreeBSD, and many varieties
of GNU/Linux, but others like Fedora Core and Debian demand an entirely free
software environment, so redistribution of the firmware without the ability
to modify and distribute the source code is prohibited. The standard
response to this from the Free Software Foundation is to reverse-engineer
the device and provide free firmware. So even though it is very difficult --
almost impossible -- to do it in the absence of documentation, could such
firmware eventually be reverse-engineered? The development team that works
on the OpenBSD operating system has a lot of experience with
reverse-engineering, but both project leader Theo de Raadt and OpenBSD
network driver programmer Jonathan Gray agree that such work would be
impractical. Of reverse-engineering firmware and the hardware that it runs
on, de Raadt told me, "We can sometimes reverse-engineer how to talk to a
device... some are worse than others... but imagine reverse engineering the
firmware of 300-400 devices on the market today! Behind their little
ARM/MIPS buses, they are a *no man's land* of undocumented-ness and bugs;
hundreds and hundreds of bugs created almost all by the realities of
'time-to-market pressures.'" The issue, in other words, is made worse by
hastily-designed hardware that doesn't work as it should, and requires
specific workarounds in the firmware and/or driver. Sometimes manufacturers
provide patches or documentation for driver programmers; sometimes they
don't.

So instead of lobbying for documentation to write open source firmware, de
Raadt would prefer to simply have the right to freely distribute necessary
proprietary firmware files with his operating system, along with correct
firmware interface documentation so that a driver can be created, and
information from the manufacturer regarding bug workarounds. Many network
chip manufacturers stubbornly refuse to grant these requests, however. Theo
de Raadt told me in an email, "Our efforts to do more wireless involves a
few approaches. We reverse-engineer what we can. We borrow from other
people's reverse-engineering lessons where we can, for instance,
prism54.org<http://www.prism54.org/>is a Linux team, but their
reverse-engineer work has resulted in knowledge
which we can obviously use to write a BSD driver. And finally, we dialogue
directly with vendors to get more free access to documentation, early access
to hardware, or firmware distribution rights (sometimes there is just no
other way). Some vendors (in particular Ralink or Realtek) will even give us
hardware before it goes on sale. Some give us documentation, some give us
code. But largely a lot of American vendors are still stupidly resisting
giving anything. In any case, all these efforts together now mean that we
have more wireless support in OpenBSD than all the Linux distributions.
Maybe even combined!"

According to Jonathan Gray, the drivers that OpenBSD currently has that
require firmware that vendors won't let the OpenBSD Project distribute are:

  - *acx* (4) - TI ACX100/ACX111 IEEE 802.11a/b/g wireless network
  device
  - *an* (4) - Aironet Communications 4500/4800 IEEE 802.11FH/b
  - *atw* (4) - ADMtek ADM8211 IEEE 802.11b
  - *awi* (4) - BayStack 650 IEEE 802.11FH PCMCIA
  - *bcw* (4) - Broadcom IEEE 802.11b/g (this driver is still under
  development and does not currently work)
  - *ipw* (4) - Intel PRO/Wireless 2100 IEEE 802.11b wireless network
  device
  - *iwi* (4) - Intel PRO/Wireless 2200BG/2225BG/2915ABG IEEE 802.11a/b/g
  wireless network device
  - *wpi* (4) - Intel PRO/Wireless 3945ABG IEEE 802.11a/b/g
  - *malo* (4) - Marvell Libertas IEEE 802.11b/g
  - *pgt* (4) - Connexant/Intersil Prism GT Full-MAC IEEE 802.11a/b/g
  wireless network device
  - *uath* (4) - Atheros USB IEEE 802.11a/b/g
  - *wpi* (4) - Intel PRO/Wireless 3945ABG IEEE 802.11a/b/g wireless
  network device

And the wireless devices that either do not require firmware, or that have
runtime firmware that OpenBSD is allowed to distribute:

  - *ath* (4) - Atheros IEEE 802.11a/b/g
  - *atu* (4) - Atmel AT76C50x USB IEEE 802.11b wireless network device
  - *ral, ural* (4) - Ralink Technology IEEE 802.11a/b/g wireless
  network device (2nd gen 802.11 Ralink)
  - *rum* (4) - Ralink Technology USB IEEE 802.11a/b/g wireless network
  device
  - *ray* (4) - Raytheon Raylink/WebGear Aviator IEEE 802.11FH
  - *rtw* (4) - Realtek RTL8180L IEEE 802.11b
  - *wi* (4) - WaveLAN/IEEE, PRISM 2-3, and Spectrum24 IEEE 802.11b
  - *zyd* (4) - Zydas ZD1211 USB IEEE 802.11b/g wireless network device

Intel requires that people who use its Centrino wireless firmware submit to
a lengthy license
agreement<http://ipw2100.sourceforge.net/firmware.php?fid=4>before
downloading and using it. While the majority of end-users may just
scroll down and click the "I Agree" link to get past it, the process is not
quite so simple for free software operating systems, which would have to
provide the same license hurdle for every one of their users, and agree not
to modify any of the driver header code that Intel provides. Commercial
desktop GNU/Linux distributions like SUSE, Mandriva, and Linspire already do
this, or have independent distribution agreements with Intel.
When reverse-engineering works

Reyk Floeter and various other OpenBSD programmers managed to write a free
replacement for the proprietary Atheros hardware abstraction layer (HAL)
called ar5k <http://www.openbsd.org/cgi-bin/cvsweb/src/sys/dev/ic/ar5xxx.h>or
"OpenHAL." The HAL isn't firmware; it is a form of abstracted device
driver that actually loads into the operating system kernel through a small
amount of driver code. The big difference between a HAL and a firmware is
where the code resides; if it's on the device, it's firmware, and if it is
loaded into the operating system's kernel, it's HAL.

ar5k works with many Atheros-based wireless cards and has been examined and
recently given a clean bill of copyright health from the Software Freedom
Law Center <http://lkml.org/lkml/2006/11/14/357>. Mysteriously, it has
virtually no support from the Madwifi Project <http://madwifi.org/>, which
is the development team responsible for creating Atheros drivers for
Linux-based operating systems. Madwifi continues to primarily support the
proprietary Atheros HAL, though there is an old and uninformative page in
the Madwifi Wiki <http://madwifi.org/wiki/OpenHAL> about it. The email
addresses listed for the Madwifi developers either bounce or elicit no
response to requests for comments on why there is no apparent effort to use
the open source HAL in Madwifi. So if it exists, it's free-as-in-rights, and
it works, then why doesn't Madwifi use ar5k instead of the proprietary HAL?
It may very well be pressure from Atheros that keeps ar5k out of Madwifi,
but no one at Madwifi or Atheros would talk to me about it.
The manufacturers speak -- or don't

In order to find out why network chip manufacturers are so polarized in
their support of free software operating systems, I made contact with
company representatives at Atheros, Intel, Marvell, Atmel, Ralink, Texas
Instruments, Broadcom, and Realtek. Not surprisingly, the manufacturers who
shun operating system programmers also seem to be reluctant to talk to the
press.
Atheros

After weeks of repeated requests and pleas for any kind of response just to
verify that the PR email address works, an Atheros representative told me
that she was unable to find anyone at the company who was qualified and
willing to comment for this article.
Intel

Intel punted me to different people a few times, then after a short delay
and a bizarre inquisition into my professional background and "intentions"
in writing this article, told me that the company had nothing to say on the
matter of wireless firmware distribution rights and interface documentation.
Considering Intel's outstanding PR record and its general willingness to
provide hardware documentation for the PCI chipsets and drive controllers
that it makes, this behavior is unusual. One of the questions I asked Intel
was if it felt its uncooperativeness with free software developers was in
direct conflict with this
presentation<http://developer.osdl.org/dev/opendrivers/summit2006/james_ketrenos.pdf>about
the importance of participating in and supporting open source software
development that Intel engineer James Ketrenos gave last summer. In it,
Ketrenos says several things in favor of corporate cooperation with open
source software developers, including these points:


  - Enable the community to do as much as possible
  - Only keep internal those things that the community can not
  contribute to (Example: Certification testing)
  - If you need to keep IP closed source (for example some whiz-bang
  algorithm), document the hardware sufficiently that the community can
  provide their own.
  - Treat the community as if they were a member of your internal team

Broadcom

The Broadcom press relations person I spoke with very much wanted to help
me, but could not find anyone at Broadcom who felt qualified to comment.
Marvell

I have already written about the substandard responsiveness of Marvell's
outsourced PR agency
<http://www.thejemreport.com/mambo/content/view/286/>(see the bottom
of the linked article).
Texas Instruments

An internal Texas Instruments press relations person was very responsive to
my requests for comment at first, then punted me to an outsourced PR agency
which, in turn, ignored several emails asking for information. After more
than two weeks of repeated inquiries, a representative of the PR agency
informed me that she was unable to find anyone at Texas Instruments who was
qualified to comment on TI's policies on providing hardware documentation
and firmware redistribution agreements.
Ralink Technologies

I had no trouble getting through to Ralink, where I spoke with company
representative Lillian Chiu.

*Ralink has been responsive to requests for hardware documentation without
requiring an NDA. Why can Ralink do this when competing manufacturers such
as Intel and Marvell require non-disclosure agreements?*

*Lillian Chiu:* It's our philosophy to spread the technology without border,
along with high performance and low cost.

*Because of Ralink's cooperativeness with projects like OpenBSD and Linux
kernel developers, Ralink's products tend to be very well supported in
so-called "alternative" operating systems. Do you see this as a competitive
advantage? Does Ralink sell more network products as a result?*

*LC:* Our customers have often provided positive feedback for our
best-in-class device support. We see it as a win-win situation where
advanced users get more flexibility while we sell more products.

*If a programmer needs to access Ralink hardware documentation, whom at
Ralink should they contact?*

*LC:* Please visit Ralink Web site at www.ralinktech.com for details under
/support/forums.
Atmel Inc.

Richard Bisset, product marketing director of the multimedia and
communications group, offered several minutes of his time to answer some
questions about wireless drivers and firmware.

*Atmel has been responsive to requests for hardware documentation without
requiring an NDA. Why can Atmel do this when competing manufacturers such as
Intel and Marvell require non-disclosure agreements?*

*Richard Bisset:* For some things we do require non-disclosure agreements,
but we are generally able to provide the API documentation and the firmware
driver interface specifications for our hardware. As to why others may not
be able to do this... well, our software is developed in-house, but others
might out outsource their driver development to third-party companies, so
they may not even have the documentation that a programmer requests.

As to why Intel might be so secretive with this information if they do have
it, I can't say. Perhaps because more and more features are actually being
moved from the firmware to the driver, some companies are being more guarded
than others. Additionally, as the Centrino laptop processor includes
wireless, it may be they don't want to give up any potential IP that could
help their competition -- I don't really know for sure.

*Because of Atmel's cooperativeness with projects like OpenBSD and Linux
kernel developers, Atmel's products tend to be very well supported in
so-called "alternative" operating systems. Do you see this as a competitive
advantage? Does Atmel sell more network products as a result?*
*RB:* Well, if you look at the success of Intel, Marvell, Broadcom, etc. --
the big players in the wireless industry -- they are successful with the OEM
and peripheral markets. As the embedded markets generally require more
technical support, they probably don't care much about getting another
50,000 or 100,000 units sold and can be tight with their APIs. We were late
to market with our 802.11g products, so we missed the OEM opportunities, and
with wireless, you've only got three real chances for success: you can be
first to market with a technology, or you can have valuable and unique
features that no one else has and the market wants, or you can have the
lowest price. Atmel wasn't first, didn't have any new unique features, and
wasn't the cheapest, either. With the PC and OEM markets being somewhat
locked out, we repositioned to focus on the embedded space where the market
was experiencing and predicting large growth. In the embedded market, if you
don't get documentation to developers, then you both fail.

*What's the big secret with device firmware, then?*

*RB:* Firmware is generally running on the chip itself, as opposed to the
host. The code is written tightly coupled to the chip it is running on and
implements/enables functions that the hardware supports. Therefore, there
generally is a lot of IP within the firmware. Our firmware is loaded from an
external flash into internal SRAM, but more and more semiconductor
manufacturers are moving firmware functions to the driver.

We usually provide driver source code, and we try to put it under the GPL if
possible, so that's usually good enough if you want to write your own
driver. If you want to see more than that, we generally require an NDA, or
if you're an embedded customer, we provide reference platforms.

*So the firmware is so secret that you have to sign an NDA to see it, but
not secret enough that some stranger who works on an obscure operating
system can see it with merely a signature?*

*RB:* It is kind of a strange situation, yes, and sometimes we run into
people who are reluctant to sign NDAs. It really depends on the customer and
target application. We don't require an NDA to see hardware APIs, and I
think that's what you're talking about with regard to documentation.

*If a programmer needs to access Atmel hardware documentation, whom at Atmel
should they contact?*

*RB:* We have contact forms through our Web site
<http://www.atmel.com/>where you can make requests, and they are
directed to the proper person at
Atmel.
Realtek Inc.

After a few emails, I made contact with Tracy Ho, a Taiwan-based
representative for Realtek.

*Realtek has reportedly been responsive to requests for hardware
documentation without requiring a non-disclosure agreement (NDA). Why can
Realtek do this when competing manufacturers such as Intel and Marvell
require NDAs?*

*Tracy Ho:* For some years Realtek has been one of the largest NIC/LOM
solution providers. One of the major reasons for this is that Realtek takes
customer service as one of its top priorities. We aim to work with our
customers as partners in a mutually cooperative environment. Our product
sales and technical support teams are well-recognized by our customers and
we strive to provide them with the convenience of flexibility and real-time
support. One of the ways we do this is by releasing general hardware
documents without requiring a NDA.

*Because of Realtek's cooperativeness with projects like OpenBSD and Linux
kernel developers, Realtek's products tend to be very well supported in
so-called "alternative" operating systems. Do you see this as a competitive
advantage? Does Realtek sell more network products as a result?*

*TH:* Realtek has been working very closely with various operating system
providers pretty much since the company was first started, and we do
consider this to be one of our competitive advantages. Over the last decade,
the great expansion of networked devices throughout the home and work
environment, and the use of open-source operating systems in many such
devices, has definitely helped us expand market share.

*If a programmer needs to access Realtek hardware documentation, whom at
Realtek should they contact?*

*TH:* For most hardware documentation, programmers can access Realtek's
website <http://www.realtek.com.tw/>. Any other questions can be directed to
our technical support teams (contact information available from the "Contact
Us" link on our website).
All we need to do is make sure we keep talking

Some of the non-responsiveness of manufacturers may just be bad PR work, but
the same companies that wouldn't talk to me have also refused to reply to
free software programmers who have requested the same information. The
impression I got from most of these companies (excepting Intel) was that
they were not at all prepared to deal with the issues of firmware
redistribution rights and hardware API documentation requests. That they
have ignored free software programmers' requests is not necessarily a sign
of unwillingness to participate, but perhaps a general sense of confusion as
to *how* they are able to help. No one seems to know whom to talk to at the
company, and in some cases the proper documentation may not exist -- or it
may belong to yet another company that the hardware manufacturer outsourced
the firmware development to.

On the other hand, it was just as difficult to contact and get comments from
the Madwifi developers, and it seems that each individual wireless driver in
the Linux kernel has different people working on it. Even if one of the
above companies wanted to provide the appropriate materials to create free
drivers and firmware, how would they know whom to contact? The irony in this
story is that the heart of the problem lies in a lack of communication, but
not between operating systems and network devices -- between hardware
manufacturers and the software developers who are trying to support their
devices.

Discuss this article or get technical support on our
forum.<http://www.tjrforum.com/>

*Copyright 2006 Jem Matzan.*
[image: Creative Commons
License]<http://creativecommons.org/licenses/by-nc-nd/2.5/>

This work is licensed under a Creative Commons
Attribution-NonCommercial-NoDerivs 2.5
License<http://creativecommons.org/licenses/by-nc-nd/2.5/>


--
Free Software, Free Society <http://fsfs.hipatia.net>

Other related posts:

  • » [School-GNU-Linux] The battle for wireless network drivers