[frgeek-michiana] Re: Server update gone bad

  • From: Marc R Rockwell <marcrrockwell@xxxxxxxxx>
  • To: frgeek-michiana@xxxxxxxxxxxxx
  • Date: Sun, 8 Jan 2012 10:16:05 -0500

if you guys need any pci cards for this let me know i will lokk through my
stuff and see what i still have !!!

On Sat, Jan 7, 2012 at 9:56 AM, Imre <imreb321@xxxxxxxxx> wrote:

> Can the server boot from usb? If it's older 4-5 years old then probably
> not. But if you were able to boot from usb then you put a rescue cd on a
> flash drive. I use unetbootin available in ubuntu and debian.
>
> On Sat, 2012-01-07 at 14:07 -0500, Tom Brown wrote:
> > The PATA drives are in a raid configuration, probably a simple mirror.
> > Richard thinks the onboard IDE controllers have raid capability. It is
> > also possible the raid is software, not hardware, based. If I move the
> > PATA drives to the IDE expansion card (no raid), might I lose the raid
> > and possibly data? If data is not jeopardized, that would be a good
> > solution to preserving the data and booting the system with a rescue
> > disc. However, without assurance the data is safe, I’m hesitant to
> > disturb the raid.
> >
> >
> >
> > Another option is to connect the optical drive to the slave position
> > on one of the onboard IDE cables though I don’t know how that might
> > affect the raid configuration. Hopefully, not at all.
> >
> >
> >
> > Tom --
> >
> >
> >
> >
> >
> > ______________________________________________________________________
> > From: frgeek-michiana-bounce@xxxxxxxxxxxxx
> > [mailto:frgeek-michiana-bounce@xxxxxxxxxxxxx] On Behalf Of Chuq
> > Sent: Friday, January 06, 2012 08:34
> > To: frgeek-michiana@xxxxxxxxxxxxx
> > Subject: [frgeek-michiana] Re: Server update gone bad
> >
> >
> >
> >
> > Typically the bios will not see an optical drive that is on an IDE
> > expansion card. I would move the optical drive to the main board, move
> > the hdd's to the ide expansion card and re-reun grub (grub-probe) and
> > see the results of such.
> >
> >
> > ______________________________________________________________________
> >         Phil is right. The PATA drives are connected to the main board
> >         IDE
> >         channels.
> >
> >         There are two expansion cards in the PCI daughter board
> >         (riser): an
> >         Ethernet card and a two-channel IDE card. Channel 1 of the IDE
> >         expansion
> >         card is connected to the DVD drive.
> >
> >         Phil and I wanted to boot from the original DVD drive to
> >         inspect and
> >         possibly edit the Grub menu list. The BIOS was not reporting
> >         the
> >         original optical drive so we replaced it. The BIOS continues
> >         to report
> >         the FD and the two PATA HDs, both connected to the main board,
> >         but still
> >         does not report the optical drive.
> >
> >         Last night I reseated the PCI riser bus, the IDE expansion
> >         card and the
> >         data and power cables to the optical drive. The BIOS still
> >         wouldn't
> >         report it although the LED in the optical drive lights up
> >         during
> >         bootstrap. Oddly, the Linux kernel detects and reports the
> >         optical
> >         drive.  But, because the BIOS doesn't "see" the optical drive,
> >         it is not
> >         in the Boot Order menu, and the BIOS won't boot from it.
> >
> >         My options seem to be:
> >
> >         1. Replace the IDE/optical drive data cable.
> >         2. Move the IDE/optical drive to channel 2 on the PCI
> >         expansion card.
> >         3. Replace the DVD optical drive with a simpler CDROM drive.
> >         4. Connect the optical drive to the slave data connector on
> >         one of the
> >         onboard PATA HD cables.
> >         5. Replace the PCI IDE expansion card.
> >
> >         Tom --
> >
> >         Mike Cook wrote:
> >         > It sounds like the raid card isn't being used for raid which
> >         would seem to indicate that either no raid is being used for
> >         the hard drives or that the server is running software raid
> >         and using the raid card for additional ide channel(s). I have
> >         seen. some ide raid cards (Promise) that use the onboard ide
> >         channels  though. I would proceed carefully if we wish to
> >         salvage data.
> >         >
> >         >
> >         > -----Original Message-----
> >         >
> >         >> From: Imre <imreb321@xxxxxxxxx>
> >         >> Sent: Jan 5, 2012 12:06 PM
> >         >> To: frgeek-michiana@xxxxxxxxxxxxx
> >         >> Subject: [frgeek-michiana] Re: Server update gone bad
> >         >>
> >         >> Then it seems to me that's the problem. The motherboard
> >         bios is not
> >         >> seeing the optical drive, therefore it cannot boot from it.
> >         The optical
> >         >> is probably being recognized by the raid card bios only.
> >         But the system
> >         >> will not be able to boot from the optical. You have to
> >         connect the
> >         >> optical to the onboard ide.
> >         >>
> >         >> On Thu, 2012-01-05 at 16:47 -0500, Phil wrote:
> >         >>
> >         >>> The hard drives are connected to the motherboard directly,
> >         one each on the IDE channels. The optical is connected the
> >         RAID (PCI IDE) card.
> >         >>>
> >         >>> Sent via Android, G2
> >         >>>
> >         >>> Richard Zimmerman <ke4rit@xxxxxxxxx> wrote:
> >         >>>
> >         >>>
> >         >>>> I looked at the raid setup on the colo server
> >         (/proc/mdstat says raid
> >         >>>> arrays fine :) )
> >         >>>>
> >         >>>> here's what it reads out on cat /boot/grub/menu.lst
> >         >>>>
> >         >>>> <readout>
> >         >>>> ## ## End Default Options ##
> >         >>>>
> >         >>>> title           Debian GNU/Linux, kernel
> >         2.6.26-2-openvz-686
> >         >>>> root            (hd0,5)
> >         >>>> kernel          /boot/vmlinuz-2.6.26-2-openvz-686
> >         root=/dev/md0 ro vga=791
> >         >>>> initrd          /boot/initrd.img-2.6.26-2-openvz-686
> >         >>>>
> >         >>>> title           Debian GNU/Linux, kernel
> >         2.6.26-2-openvz-686 (single-user mode)
> >         >>>> root            (hd0,5)
> >         >>>> kernel          /boot/vmlinuz-2.6.26-2-openvz-686
> >         root=/dev/md0 ro single
> >         >>>> initrd          /boot/initrd.img-2.6.26-2-openvz-686
> >         >>>>
> >         >>>> title           Debian GNU/Linux, kernel 2.6.26-2-686
> >         >>>> root            (hd0,5)
> >         >>>> kernel          /boot/vmlinuz-2.6.26-2-686 root=/dev/md0
> >         ro vga=791
> >         >>>> initrd          /boot/initrd.img-2.6.26-2-686
> >         >>>>
> >         >>>> title           Debian GNU/Linux, kernel 2.6.26-2-686
> >         (single-user mode)
> >         >>>> root            (hd0,5)
> >         >>>> kernel          /boot/vmlinuz-2.6.26-2-686 root=/dev/md0
> >         ro single
> >         >>>> initrd          /boot/initrd.img-2.6.26-2-686
> >         >>>>
> >         >>>> title           Debian GNU/Linux, kernel
> >         2.6.26-1-openvz-686
> >         >>>> root            (hd0,5)
> >         >>>> kernel          /boot/vmlinuz-2.6.26-1-openvz-686
> >         root=/dev/md0 ro vga=791
> >         >>>> initrd          /boot/initrd.img-2.6.26-1-openvz-686
> >         >>>>
> >         >>>> title           Debian GNU/Linux, kernel
> >         2.6.26-1-openvz-686 (single-user mode)
> >         >>>> root            (hd0,5)
> >         >>>> kernel          /boot/vmlinuz-2.6.26-1-openvz-686
> >         root=/dev/md0 ro single
> >         >>>> initrd          /boot/initrd.img-2.6.26-1-openvz-686
> >         >>>>
> >         >>>> title           Debian GNU/Linux, kernel 2.6.26-1-686
> >         >>>> root            (hd0,5)
> >         >>>> kernel          /boot/vmlinuz-2.6.26-1-686 root=/dev/md0
> >         ro vga=791
> >         >>>> initrd          /boot/initrd.img-2.6.26-1-686
> >         >>>>
> >         >>>> title           Debian GNU/Linux, kernel 2.6.26-1-686
> >         (single-user mode)
> >         >>>> root            (hd0,5)
> >         >>>> kernel          /boot/vmlinuz-2.6.26-1-686 root=/dev/md0
> >         ro single
> >         >>>> initrd          /boot/initrd.img-2.6.26-1-686
> >         >>>>
> >         >>>> ### END DEBIAN AUTOMAGIC KERNELS LIST
> >         >>>> </readout>
> >         >>>>
> >         >>>> colo mounts root=/dev/md0
> >         >>>>
> >         >>>> The error "mount: mounting /dev/hda2 on /root failed:
> >         Device or
> >         >>>> resource busy" would make sense as it's attached to the
> >         array md0...
> >         >>>>
> >         >>>> If you can get onto the array.... mount /dev/md0 look at
> >         the menu.lst file...
> >         >>>>
> >         >>>> /dev/md1 should be the swap file IIRC...
> >         >>>>
> >         >>>> Richard
> >         >>>>
> >         >>>>
> >         >>>>
> >         >>>>
> >         >>>> On Thu, Jan 5, 2012 at 11:17 AM, Tom Brown
> >         <tbrown@xxxxxxxxxxxx> wrote:
> >         >>>>
> >         >>>>> I tried to boot the server off an Ultimate Boot CD
> >         (UBCD) Phil carries in
> >         >>>>> his kit. The goal was to find, copy and edit the GRUB
> >         conf file. btw, the
> >         >>>>> contents of the Grub conf file are displayed on the
> >         console for about 1-2
> >         >>>>> seconds, too fast for me to pause the console output.
> >         >>>>>
> >         >>>>> The server wouldn't boot off the optical drive even
> >         though we set it as the
> >         >>>>> first boot device. The BIOS continued to boot off the
> >         hard drive. Phil and I
> >         >>>>> disconnected and reconnected cables with no improvement.
> >         Assuming the BIOS
> >         >>>>> doesn't see the optical drive or the drive is broken, we
> >         installed a DVD ROM
> >         >>>>> from inventory. We must have disconnected the floppy
> >         drive because the
> >         >>>>> server won't boot and throws the type of error you'd
> >         expect from a bad or
> >         >>>>> disconnected FD. We ran out of time and will pick it up
> >         again tonight,
> >         >>>>> starting with reconnecting the FD.
> >         >>>>>
> >         >>>>> Here is what I transcribed from the end of the console
> >         on booting. It
> >         >>>>> appears the BIOS is not using UUID. The init script
> >         finds hda (hda1, hda2)
> >         >>>>> and hdc (hdc1, hdc2) so I’ll quote the init output from
> >         there.
> >         >>>>>
> >         >>>>> <boot log snippet>
> >         >>>>>
> >         >>>>> Begin: Mounting root files system ...
> >         >>>>> Begin: Running /scripts/local-top ...
> >         >>>>> Begin: Loading md: raid1 personality registered for
> >         level 1
> >         >>>>> Success: loaded module raid1.
> >         >>>>> done.
> >         >>>>> Begin: Assembling all MD arrays ...
> >         >>>>> md: md0 stopped.
> >         >>>>> md: bind<hdc2>
> >         >>>>> md: bind<hda2>
> >         >>>>> raid1: raid set md0 active with 2 out of 2 mirrors
> >         >>>>> mdadm: /dev/md0 has been started with 2 drives.
> >         >>>>> Success: assembled all arrays.
> >         >>>>> done.
> >         >>>>> Begin: waiting for udev to process events ... done.
> >         >>>>> done.
> >         >>>>> Begin: Running /scripts/local-premount ... kinit:
> >         name_to_dev_t(/dev/hda1) =
> >         >>>>> hda1(3,1)
> >         >>>>> kinit: trying to resume from /dev/hda1
> >         >>>>> PM: Starting manual resume from disk
> >         >>>>> kinit: No resume image, doing normal boot...
> >         >>>>> resume: libgcrypt version: 1.4.1
> >         >>>>> done.
> >         >>>>> mount: mounting /dev/hda2 on /root failed: Device or
> >         resource busy
> >         >>>>> Begin: Running /scripts/local-bottom ... done.
> >         >>>>> done.
> >         >>>>> Begin: Running /scripts/init-bottom ... mount:
> >         mounting /dev on /root/dev
> >         >>>>> failed: No such file or directory
> >         >>>>> done.
> >         >>>>> mount: mounting /sys on /root/sys failed: No such file
> >         or direcotry
> >         >>>>> mount: mounting /proc on /root/proc failed: No such file
> >         or directory
> >         >>>>> Target filesystem doesn’t have /sbin/init.
> >         >>>>> No init found. Try passing init= bootarg.
> >         >>>>>
> >         >>>>> BusyBox v1.10.2 (Debian 1:1.10.2-2) built-in shell
> >         (ash).
> >         >>>>> Enter 'help' for a list of commands.
> >         >>>>>
> >         >>>>> /bin/sh: can’t access tty; job control turned off
> >         >>>>> (initramfs) _
> >         >>>>>
> >         >>>>> </boot log snippet>
> >         >>>>>
> >         >>>>> Tom --
> >         >>>>>
> >         >>>>>
> >         >>>>> Imre B wrote:
> >         >>>>>
> >         >>>>>> # blkid
> >         >>>>>> will show disks with their UUID
> >         >>>>>>
> >         >>>>>> On Fri, Dec 30, 2011 at 1:45 PM, Tony Germano
> >         <tony_germano@xxxxxxxxxxx>
> >         >>>>>> wrote:
> >         >>>>>>
> >         >>>>>>
> >         >>>>>>> Is the UUID used anywhere besides fstab and maybe the
> >         boot loader? It
> >         >>>>>>> makes
> >         >>>>>>> sense to use it there when you have multiple drives
> >         because it guarantees
> >         >>>>>>> that your mounts will all work even if boot order
> >         changes for some
> >         >>>>>>> reason.
> >         >>>>>>>
> >         >>>>>>> Each file system is assigned a UUID upon creation. You
> >         can use the basic
> >         >>>>>>> tools for that file system to find out what the UUID
> >         is. It will never
> >         >>>>>>> change, even if you move the drive to a different
> >         server. It will also
> >         >>>>>>> not
> >         >>>>>>> change if you clone the partition.
> >         >>>>>>>
> >         >>>>>>> If you are able to boot from a cd, I think you can use
> >         tune2fs -l
> >         >>>>>>> /dev/<hdxx> to see the UUID for the file system on a
> >         given partition
> >         >>>>>>> (assuming it is ext2/3/4.)
> >         >>>>>>>
> >         >>>>>>> This page also lists some other ways (I haven't tried
> >         them) to get the
> >         >>>>>>> UUID
> >         >>>>>>>
> >
> http://liquidat.wordpress.com/2007/10/15/short-tip-get-uuid-of-hard-disks/
> >         >>>>>>> The ls -l /dev/disk/by-uuid way looks pretty cool. I
> >         didn't know that
> >         >>>>>>> directory was there.
> >         >>>>>>>
> >         >>>>>>> Tony
> >         >>>>>>>
> >         >>>>>>>
> >         >>>>>>>
> >         >>>>>>>
> >         >>>>>>>> Date: Thu, 29 Dec 2011 17:46:07 -0500
> >         >>>>>>>> From: tbrown@xxxxxxxxxxxx
> >         >>>>>>>> To: frgeek-michiana@xxxxxxxxxxxxx
> >         >>>>>>>> Subject: [frgeek-michiana] Re: Server update gone bad
> >         >>>>>>>>
> >         >>>>>>>> Sorry to reply so late. I've been very busy over the
> >         holiday and out of
> >         >>>>>>>> town for a few days.
> >         >>>>>>>>
> >         >>>>>>>> I ran 'apt-get update' using the default 'stable'
> >         sources, but that
> >         >>>>>>>> listed a lot of unverified packages which turned out
> >         to be Squeeze
> >         >>>>>>>> packages so I never started the upgrade. After some
> >         research I realized
> >         >>>>>>>> I needed to stay at Lenny by changing the sources
> >         from 'stable' to
> >         >>>>>>>> 'Lenny' as stable now means 'squeeze'. That seemed to
> >         work as the number
> >         >>>>>>>> of unverified packages were very few.
> >         >>>>>>>>
> >         >>>>>>>> I don't know UUID. I'll research it at the workshop
> >         tonight.
> >         >>>>>>>>
> >         >>>>>>>> afaik, there is no data backup. I'll try booting with
> >         a rescue disk and
> >         >>>>>>>> poke around. As a last resort, I will pull the drives
> >         and try to read
> >         >>>>>>>> them on another system. I don't know yet whether the
> >         drives are SCSI,
> >         >>>>>>>> SATA, PATA or some combination. I believe the drives
> >         were in some sort
> >         >>>>>>>> of RAID config.
> >         >>>>>>>>
> >         >>>>>>>> Tom --
> >         >>>>>>>>
> >         >>>>>>>> Richard Zimmerman wrote:
> >         >>>>>>>>
> >         >>>>>>>>
> >         >>>>>>>>> Reminds me since we are on the subject...
> >         >>>>>>>>>
> >         >>>>>>>>> Colo server IS LOCKED to "Lenny" and I have every
> >         intention of leaving
> >         >>>>>>>>> it there....
> >         >>>>>>>>>
> >         >>>>>>>>> If a new server admin takes over, his or her call
> >         then...lists
> >         >>>>>>>>>
> >         >>>>>>>>> BTW, also speaking from experience after I toasted
> >         my home server on a
> >         >>>>>>>>> blind 'dist-upgrade' option.... (from the Lenny to
> >         Squeeze upgrade)
> >         >>>>>>>>>
> >         >>>>>>>>> ***********************************
> >         >>>>>>>>> NEVER EVER USE IT AGAIN
> >         >>>>>>>>> ***********************************
> >         >>>>>>>>>
> >         >>>>>>>>> That BTDT thing I mentioned in the previous email
> >         also happened on my
> >         >>>>>>>>> home server but I got lucky and recovered it....
> >         >>>>>>>>>
> >         >>>>>>>>> Now days I **ONLY** use the 'safe-upgrade' option...
> >         >>>>>>>>>
> >         >>>>>>>>> Lessons learned the hard way about 8 months ago...
> >         >>>>>>>>>
> >         >>>>>>>>> Goose
> >         >>>>>>>>>
> >         >>>>>>>>> / Where is the old RedHat 5.2 when you want it? It
> >         was simple and 'just
> >         >>>>>>>>> worked'
> >         >>>>>>>>> // Debian Lenny is the same way. Functional and
> >         'just works'
> >         >>>>>>>>> /// Why keep screwing with it...???
> >         >>>>>>>>>
> >         >>>>>>>>>
> >         >>>>>>>>> On Sat, Dec 24, 2011 at 7:14 PM, Richard Zimmerman
> >         <ke4rit@xxxxxxxxx>
> >         >>>>>>>>> wrote:
> >         >>>>>>>>>
> >         >>>>>>>>>
> >         >>>>>>>>>
> >         >>>>>>>>>> If I'm reading it correctly. You pretty well
> >         screwed the server....
> >         >>>>>>>>>>
> >         >>>>>>>>>> The new Debian forces everything to that NOT SO
> >         wonderful UUID Crap
> >         >>>>>>>>>> for everything. If you then rolled back to Lenny,
> >         congrats, you own a
> >         >>>>>>>>>> toasted server....
> >         >>>>>>>>>>
> >         >>>>>>>>>> BTDT, you pretty screwed... That happened to me....
> >         Happen to have a
> >         >>>>>>>>>> backup of the server?
> >         >>>>>>>>>>
> >         >>>>>>>>>> Goose
> >         >>>>>>>>>>
> >         >>>>>>>>>> / Happened on a server at work. FORTUNATELY, I had
> >         grab some dead
> >         >>>>>>>>>> hardware out of the back room and had used it so it
> >         wasn't a
> >         >>>>>>>>>> production server I toasted...
> >         >>>>>>>>>>
> >         >>>>>>>>>> // Left all work servers @ Lenny and have been
> >         researching finding a
> >         >>>>>>>>>> distro that DOES NOT use UUID. That crap is for the
> >         birds....
> >         >>>>>>>>>>
> >         >>>>>>>>>> /// Maybe force it UUID in Debian for the win!
> >         >>>>>>>>>>
> >         >>>>>>>>>> //// If I can't find a distro that doesn't use
> >         UUID, guess I'll be
> >         >>>>>>>>>> retiring as a server admin....
> >         >>>>>>>>>>
> >         >>>>>>>>>>
> >         >>>>>>>>>>
> >         >>>>>>>>>>
> >         >>>>>>>>>> On Sat, Dec 24, 2011 at 12:41 PM, Tom Brown
> >         <tbrown@xxxxxxxxxxxx>
> >         >>>>>>>>>> wrote:
> >         >>>>>>>>>>
> >         >>>>>>>>>>
> >         >>>>>>>>>>
> >         >>>>>>>>>>> Mike started a hard drive wipe too late to
> >         complete by closing time
> >         >>>>>>>>>>> last
> >         >>>>>>>>>>> Thursday. I started a warehouse server update
> >         which also didn't
> >         >>>>>>>>>>> complete
> >         >>>>>>>>>>> before we departed so Mike and I left the systems
> >         running with the
> >         >>>>>>>>>>> intention
> >         >>>>>>>>>>> I would stop by the warehouse today and turn off
> >         the computers.
> >         >>>>>>>>>>>
> >         >>>>>>>>>>> Mike's DBAN wipe completed so I turned off the PC.
> >         The server update
> >         >>>>>>>>>>> completed so I rebooted to test the update.
> >         >>>>>>>>>>>
> >         >>>>>>>>>>> When the server first tried to boot it died trying
> >         to load drivers
> >         >>>>>>>>>>> for
> >         >>>>>>>>>>> hdc
> >         >>>>>>>>>>> or the next item after that; I couldn't tell
> >         which. I turned the
> >         >>>>>>>>>>> server off,
> >         >>>>>>>>>>> booted and got the following errors.
> >         >>>>>>>>>>>
> >         >>>>>>>>>>> ----------
> >         >>>>>>>>>>>
> >         >>>>>>>>>>> Running /scripts/inti-bottom ... mount:
> >         mounting /dev on /root/dev
> >         >>>>>>>>>>> failed:
> >         >>>>>>>>>>> No such file or directory
> >         >>>>>>>>>>> done
> >         >>>>>>>>>>> mount: mounting /sys on /root/sys failed: No such
> >         file or directory
> >         >>>>>>>>>>> mount: mounting /proc on /root/proc failed: No
> >         such file or directory
> >         >>>>>>>>>>> Target filesystem doesn't have /sbin/init.
> >         >>>>>>>>>>> No init found. Try passing init=bootarg.
> >         >>>>>>>>>>>
> >         >>>>>>>>>>> BusyBox v1.10.2 (Debian 1:1.10.2-2) built-in shell
> >         (ash).
> >         >>>>>>>>>>> Enter 'help' for a list of commands.
> >         >>>>>>>>>>>
> >         >>>>>>>>>>> /bin/sh can't access tty; job control turned off
> >         >>>>>>>>>>> (initramfs) __
> >         >>>>>>>>>>>
> >         >>>>>>>>>>> ----------
> >         >>>>>>>>>>>
> >         >>>>>>>>>>> A Google search didn't find anything in the first
> >         two pages of
> >         >>>>>>>>>>> results
> >         >>>>>>>>>>> which
> >         >>>>>>>>>>> exactly applies to the problem and update back
> >         story.
> >         >>>>>>>>>>>
> >         >>>>>>>>>>> The source lists for apt-get were set to "stable"
> >         repositories. Since
> >         >>>>>>>>>>> Debian advanced to v6.x 'Squeeze', stable was
> >         resulting in v6
> >         >>>>>>>>>>> packages
> >         >>>>>>>>>>> which
> >         >>>>>>>>>>> wouldn't verify. So I changed 'stable' to
> >         'Lenny' (Debian v5.x) in
> >         >>>>>>>>>>> the
> >         >>>>>>>>>>> sources list, and the update/upgrade sequence went
> >         much better (until
> >         >>>>>>>>>>> I
> >         >>>>>>>>>>> rebooted).
> >         >>>>>>>>>>>
> >         >>>>>>>>>>> I'm not sure if there was a kernel update or not.
> >         >>>>>>>>>>>
> >         >>>>>>>>>>> The server is running RAID and possibly LVM. I
> >         seldom use either so
> >         >>>>>>>>>>> I'm not
> >         >>>>>>>>>>> well versed on either. Also, Debian uses Grub
> >         (possibly several
> >         >>>>>>>>>>> versions of
> >         >>>>>>>>>>> Grub). Grub runs on my Ubuntu 10.04 LTS netbook
> >         install, but I never
> >         >>>>>>>>>>> touch
> >         >>>>>>>>>>> it.
> >         >>>>>>>>>>>
> >         >>>>>>>>>>> Any ideas?
> >         >>>>>>>>>>>
> >         >>>>>>>>>>> Tom --
> >         >>>>>>>>>>> ** This list is PUBLICLY archived. **
> >         >>>>>>>>>>> PLEASE don't post personal or sensitive
> >         information unless you wish
> >         >>>>>>>>>>> for it
> >         >>>>>>>>>>> to be in the public domain. Email addresses are
> >         removed from the
> >         >>>>>>>>>>> archive but
> >         >>>>>>>>>>> other items included in the body of the email will
> >         be shown.
> >         >>>>>>>>>>>
> >         >>>>>>>>>>> To visit the main website for Free Geek Michiana
> >         go to
> >         >>>>>>>>>>> http://www.freegeekmichiana.org
> >         >>>>>>>>>>>
> >         >>>>>>>>>>> To post to the list send email to
> >         frgeek-michiana@xxxxxxxxxxxxxx
> >         >>>>>>>>>>> The archive is available at
> >         >>>>>>>>>>> //www.freelists.org/archives/frgeek-michiana/
> >         >>>>>>>>>>>
> >         >>>>>>>>>>> You may unsubscribe or change your list settings
> >         by going to the list
> >         >>>>>>>>>>> website at
> >         //www.freelists.org/webpage/frgeek-michiana
> >         >>>>>>>>>>>
> >         >>>>>>>>>>>
> >         >>>>>>>>>>>
> >         >>>>>>>>>>>
> >         >>>>>>>>>> --
> >         >>>>>>>>>>
> >         >>>>>>>>>> --
> >         >>>>>>>>>> Richard "Goose" Zimmerman, ke4rit
> >         >>>>>>>>>> Mishawaka, IN
> >         >>>>>>>>>>
> >         >>>>>>>>>>
> >         >>>>>>>>>>
> >         >>>>>>>>>
> >         >>>>>>>>>
> >         >>>>>>>>>
> >         >>>>>>>> ** This list is PUBLICLY archived. **
> >         >>>>>>>> PLEASE don't post personal or sensitive information
> >         unless you wish for
> >         >>>>>>>> it
> >         >>>>>>>> to be in the public domain. Email addresses are
> >         removed from the archive
> >         >>>>>>>> but
> >         >>>>>>>> other items included in the body of the email will be
> >         shown.
> >         >>>>>>>>
> >         >>>>>>>> To visit the main website for Free Geek Michiana go
> >         to
> >         >>>>>>>> http://www.freegeekmichiana.org
> >         >>>>>>>>
> >         >>>>>>>> To post to the list send email to
> >         frgeek-michiana@xxxxxxxxxxxxxx
> >         >>>>>>>> The archive is available at
> >         >>>>>>>> //www.freelists.org/archives/frgeek-michiana/
> >         >>>>>>>>
> >         >>>>>>>> You may unsubscribe or change your list settings by
> >         going to the list
> >         >>>>>>>> website at
> >         //www.freelists.org/webpage/frgeek-michiana
> >         >>>>>>>>
> >         >>>>>>>>
> >         >>>>>>>>
> >         >>>>>> ** This list is PUBLICLY archived. **
> >         >>>>>> PLEASE don't post personal or sensitive information
> >         unless you wish for it
> >         >>>>>> to be in the public domain. Email addresses are removed
> >         from the archive but
> >         >>>>>> other items included in the body of the email will be
> >         shown.
> >         >>>>>>
> >         >>>>>> To visit the main website for Free Geek Michiana go to
> >         >>>>>> http://www.freegeekmichiana.org
> >         >>>>>>
> >         >>>>>> To post to the list send email to
> >         frgeek-michiana@xxxxxxxxxxxxxx
> >         >>>>>> The archive is available at
> >         >>>>>> //www.freelists.org/archives/frgeek-michiana/
> >         >>>>>>
> >         >>>>>> You may unsubscribe or change your list settings by
> >         going to the list
> >         >>>>>> website at
> >         //www.freelists.org/webpage/frgeek-michiana
> >         >>>>>>
> >         >>>>>>
> >         >>>>>>
> >         >>>>>>
> >         >>>>> ** This list is PUBLICLY archived. **
> >         >>>>> PLEASE don't post personal or sensitive information
> >         unless you wish for it
> >         >>>>> to be in the public domain. Email addresses are removed
> >         from the archive but
> >         >>>>> other items included in the body of the email will be
> >         shown.
> >         >>>>>
> >         >>>>> To visit the main website for Free Geek Michiana go to
> >         >>>>> http://www.freegeekmichiana.org
> >         >>>>>
> >         >>>>> To post to the list send email to
> >         frgeek-michiana@xxxxxxxxxxxxxx
> >         >>>>> The archive is available at
> >         >>>>> //www.freelists.org/archives/frgeek-michiana/
> >         >>>>>
> >         >>>>> You may unsubscribe or change your list settings by
> >         going to the list
> >         >>>>> website at
> >         //www.freelists.org/webpage/frgeek-michiana
> >         >>>>>
> >         >>>>>
> >         >>>>
> >         >>>> --
> >         >>>>
> >         >>>> --
> >         >>>> Richard "Goose" Zimmerman, ke4rit
> >         >>>> Mishawaka, IN
> >         >>>> ** This list is PUBLICLY archived. **
> >         >>>> PLEASE don't post personal or sensitive information
> >         unless you wish for it to be in the public domain. Email
> >         addresses are removed from the archive but other items
> >         included in the body of the email will be shown.
> >         >>>>
> >         >>>> To visit the main website for Free Geek Michiana go to
> >         http://www.freegeekmichiana.org
> >         >>>>
> >         >>>> To post to the list send email to
> >         frgeek-michiana@xxxxxxxxxxxxxx
> >         >>>> The archive is available at
> >         //www.freelists.org/archives/frgeek-michiana/
> >         >>>>
> >         >>>> You may unsubscribe or change your list settings by going
> >         to the list website at
> >         //www.freelists.org/webpage/frgeek-michiana
> >         >>>>
> >         >>>>
> >         >>> N+--ryv-vȭ+b{,ʋ_ފ{az'fIVvޚu^j!-鬊w%ם{ayʇj)pY[zht
> >         +"^ןkyzC"r(my&bjv見-azXǧu隊[h~zIryb(y܆+ފƯj)ZnWmyb(ڭb?~zIrآ칻
> >         &ޢ!jxʋ+-mx,o((x-^+-ךmyb(yZ߮h&
> >         >>>
> >         >> ** This list is PUBLICLY archived. **
> >         >> PLEASE don't post personal or sensitive information unless
> >         you wish for it to be in the public domain. Email addresses
> >         are removed from the archive but other items included in the
> >         body of the email will be shown.
> >         >>
> >         >> To visit the main website for Free Geek Michiana go to
> >         http://www.freegeekmichiana.org
> >         >>
> >         >> To post to the list send email to
> >         frgeek-michiana@xxxxxxxxxxxxxx
> >         >> The archive is available at
> >         //www.freelists.org/archives/frgeek-michiana/
> >         >>
> >         >> You may unsubscribe or change your list settings by going
> >         to the list website at
> >         //www.freelists.org/webpage/frgeek-michiana
> >         >>
> >         >>
> >         >
> >         > ** This list is PUBLICLY archived. **
> >         > PLEASE don't post personal or sensitive information unless
> >         you wish for it to be in the public domain. Email addresses
> >         are removed from the archive but other items included in the
> >         body of the email will be shown.
> >         >
> >         > To visit the main website for Free Geek Michiana go to
> >         http://www.freegeekmichiana.org
> >         >
> >         > To post to the list send email to
> >         frgeek-michiana@xxxxxxxxxxxxxx
> >         > The archive is available at
> >         //www.freelists.org/archives/frgeek-michiana/
> >         >
> >         > You may unsubscribe or change your list settings by going to
> >         the list website at
> >         //www.freelists.org/webpage/frgeek-michiana
> >         >
> >         >
> >         >
> >         ** This list is PUBLICLY archived. **
> >         PLEASE don't post personal or sensitive information unless you
> >         wish for it to be in the public domain. Email addresses are
> >         removed from the archive but other items included in the body
> >         of the email will be shown.
> >
> >         To visit the main website for Free Geek Michiana go to
> >         http://www.freegeekmichiana.org
> >
> >         To post to the list send email to
> >         frgeek-michiana@xxxxxxxxxxxxxx
> >         The archive is available at
> >         //www.freelists.org/archives/frgeek-michiana/
> >
> >         You may unsubscribe or change your list settings by going to
> >         the list website at
> >         //www.freelists.org/webpage/frgeek-michiana
> >
> >
> >
> >
> >
>
>
> ** This list is PUBLICLY archived. **
> PLEASE don't post personal or sensitive information unless you wish for it
> to be in the public domain. Email addresses are removed from the archive
> but other items included in the body of the email will be shown.
>
> To visit the main website for Free Geek Michiana go to
> http://www.freegeekmichiana.org
>
> To post to the list send email to frgeek-michiana@xxxxxxxxxxxxxx
> The archive is available at
> //www.freelists.org/archives/frgeek-michiana/
>
> You may unsubscribe or change your list settings by going to the list
> website at //www.freelists.org/webpage/frgeek-michiana
>
>

Other related posts: