[opendtv] 'Backwards' approach leads to 1-pixel camera

  • From: "Manfredi, Albert E" <albert.e.manfredi@xxxxxxxxxx>
  • To: <opendtv@xxxxxxxxxxxxx>
  • Date: Sun, 22 Oct 2006 18:07:18 -0400

There must be a missing piece in this explanation.

If the light from the digital micromirror array is focused on a single sensor, 
then what is it that differentiates between the MPEG or MPEG-like coefficients 
that are supposed to result directly from this process? Somehow, you have to 
retain knowledge of location in the image, color, and intensity.

In contrast, the 1-bit audio idea seems straightforward, and the tradeoff very 
obvious. That single audio bit has to be transmitted at a much higher rate than 
any single bit of a multibit audio scheme. The sequence 101010 creates a flat 
line, and that line can be bent up or down by changing the proportion of 0s and 
1s in the sequence.


'Backwards' approach leads to 1-pixel camera

R. Colin Johnson
(10/16/2006 9:00 AM EDT)
URL: http://www.eetimes.com/showArticle.jhtml?articleID=193200273

Portland, Ore. -- Remember how digital converters for audio started out at 8 
bits, then went to 16 and 24 bits before resetting to 1 bit with oversampling? 
Engineers at Rice University will propose this week that we reset our megapixel 
cameras to 1 pixel and our video cameras to 1 voxel, both with oversampling.

They will make their case at the Optical Society of America's 90th annual 
meeting--Frontiers in Optics 2006--in Rochester, N.Y.

The 1-pixel camera takes tens of thousands of rapid-fire shots to capture the 
equivalent of 1 million pixels in an image. So instead of expensive megapixel 
sensors with separate detectors for red, green and blue, the Rice EEs' approach 
needs only a 1-pixel multispectral sensor, simplifying hardware resources while 
enabling images to be formed from spectra never before imaged. "There are all 
sorts of detectors used in the physics lab--now most of them can be used to 
make images too," said electrical and computer engineering professor Kevin 
Kelly. "Physics labs, for instance, can now make images from neutron detectors 
using our approach, and astronomers can make images from radio waves." He 
performed the work with Richard Baraniuk, the Victor E. Cameron professor of 
electrical and computer engineering.

The enabling chip for the 1-pixel camera is not the detector used to sense an 
application-specific spectrum--that could use any technology. Instead, Texas 
Instruments Inc.'s digital micromirror array is used to project light from the 
lens onto the sensor. The micromirror array is the same chip that's used in 
Digital Light Processor televisions. Here, the lens focuses light onto the 
1,024 x 768-pixel digital micromirror chip, which in turn projects all of its 
light into a single photodiode.

"What we are essentially doing is running a Digital Light Processor backwards 
and replacing the light source with a photodiode," said Baraniuk.

You probably thought a 1-pixel camera would work like a flatbed scanner, moving 
the single sensor over the area ordinarily occupied by the film. In fact, the 
only moving part--the micromirror array--is not taking the place of a scanner 
for two reasons. First, it would be too slow to measure the light from each 
pixel location separately, requiring 1 million cycles per megapixel. And 
second, you would need an ultrasensitive photodiode to detect the light from 
just one micromirror at a time.

In contrast, the Rice algorithm relies on tens or hundreds of thousands of 
measurements, not millions. In each clock cycle, it also focuses all the light 
from the micromirror array onto the photodiode, so it doesn't have to be 
ultrasensitive. Each mirror in the array either focuses on the photodiode and 
thus is "on" or away from it and thus "off." The secret is that the micromirror 
array takes on random configurations each clock cycle. The Rice algorithm then 
measures the sum of the light from that set of coefficients in a mathematical 
decomposition of the original image--essentially deducing its most obvious 
features from a set of random measurements.

"The math is pretty dense, but the critical point is that instead of making 
millions of measurements at millions of pixel locations, we just take tens of 
thousands of measurements using a single sensor," said Baraniuk. "We are using 
some very new image reconstruction techniques to form an image from a series of 
random projections from the micromirror array."

Just as the hardware setup is the same as a projector--only backwards--so, too, 
the algorithm that deduces what the original image must have been is backwards, 
making its calculations from the random configurations of the micromirror.

A normal megapixel camera uses a sensor at every pixel location, taking 
millions of highly accurate measurements simultaneously. Later, the raw data is 
compressed into a series of coefficients for a hypothetical filter bank that 
could reconstruct the original image from white noise as an input. The Rice 
algorithm runs that backwards by starting with random noise as its input, then 
directly measuring a series of coefficients that enable reconstruction of the 
original image. "Instead of sampling millions of pixels in the light field and 
then compressing all that information . . . we can go directly from the light 
field to the compressed data," said Baraniuk.

Cameras everywhere

Today it's possible to make images only with sensors that can be built in 
arrays, but many wavelengths are impractical or too expensive. For instance, 
terahertz sensors could enable cameras that see through clothing at airport 
checkpoints, but terahertz sensors are impractical to build today. Likewise, 
sensors for low light need detectors that could use avalanche photodiodes, but 
again those are too expensive to build in arrays.

"With our approach, you could use the most expensive processes there are," said 
Kelly. "For instance, you could use an avalanche diode for a low-light camera 
that would be ultrasensitive, but impractical to build in big arrays. Today, 
the best photodetectors are built with indium gallium arsenide that you can't 
make in large arrays, but with our camera you can use the most exotic sensor 
you can find. For instance, for unrivaled color discrimination on a 
visible-light camera you could afford to get Foveon [Inc. in Santa Clara, 
Calif.] to make you a multispectral sensor with eight, 16 or even 24 layers of 
color sensitivity instead of just red, green and blue."

Next, EEs will try their method with sensors at different wavelengths, 
including infrared, terahertz and multispectral sensors such as those from 

Professor Dave Brady at Duke University is also working on a camera that uses 
the same 1-pixel approach but that is completely planar, because it requires no 
optical lens. Other new designs, the researchers said, are constructing images 
from medical sensors such as computed tomography and magnetic resonance.

The theory behind the 1-pixel camera has been knocking around for about two 
decades, but crystallized in the past couple of years with the breakthrough 
work of cooperating scientists led by mathematician Emmanuel Candès, a 
professor at the California Institute of Technology, who received the National 
Science Foundation's Alan T. Waterman Award this year.

All material on this site Copyright 2006 CMP Media LLC. All rights reserved.
You can UNSUBSCRIBE from the OpenDTV list in two ways:

- Using the UNSUBSCRIBE command in your user configuration settings at 

- By sending a message to: opendtv-request@xxxxxxxxxxxxx with the word 
unsubscribe in the subject line.

Other related posts:

  • » [opendtv] 'Backwards' approach leads to 1-pixel camera