[linux-cirrus] Re: bootmem_init_node failure with 64M TS-7250

On Thu, 2006-12-28 at 23:38 -0700, Jesse Off wrote:
> > On Thu, Dec 28, 2006 at 11:02:19PM -0500, Charles Moschel wrote:
> >
> >> Is it _worth_ fixing?
> >
> > It's probably not a lot of work, so I think it's worth fixing at some
> > point.
> >
> > But even if it does get fixed, IMHO sparsemem/discontigmem should be
> > disabled by default for ep93xx as it has a performance impact.

OK, I didn't realize that.
> >
> >
> I think the costs are approaching parity, but 1 64M chip has been
> significantly more costly that 2 32M chips for some time.  We do put on
> 64M chips from time to time for customers that need it as our 2.4 kernel
> modifications allow the chips to be pretty much dropped in.  Both our
> RedBoot bootloader and our Linux-based bootloader (Linux booting Linux)
> auto-detect the SDRAM chip sizes and pass appropriate ATAGs.

Well, RedBoot was a bit of a red herring for me, initially.  I found it
did pass ATAGs correctly, but the redboot banner still says 32M RAM was
available (0x02000000).  And when I tried Lennert's suggestion of
changing PHYS_OFFSET and loading the kernel at 0xe0008000, RedBoot
helpfully asked if I was sure, because the address appeared to be
outside of RAM (see boot log below).

While talking about reboot, would it be possible for TS to build RedBoot
to pass the correct machine ID?  Or would that break the 2.4 TS kernels?

> 
> >
> > (It also wouldn't have been a problem if TS had strapped the ep9302
> > to put the 0....... chip at c....... (phys) instead, because then all
> > RAM would have been in the same 1G range.)
> 
> When we laid out this board, I don't think we were aware that Linux 2.6
> had this bug, err I mean, "quirk", regarding physical memory layout.  :-) 
> I'm pretty sure we just picked the chip-select line that resulted in the
> most efficient hardware track to the SDRAM chip.  Our hardware design
> culture at TS really hates to compromise to work around software bugs, and
> I'd probably get heckled if I had to suggest/explain this.  Also, we had
> custom boards that use all 4 chip selects for memory so it wouldn't have
> saved us any time to avoid the 0xd and 0xe chip selects-- we had to get
> them to work anyway.

So any ep9301/02 board that uses 4 chip selects would have this quirk
and benefit from a fix, as well as any 2 chip boards (if the wrong two
CS happen to be chosen).  Right?

> >
> > (I remember trying switching async/sync boot mode on the fly on the
> > ep9302 a while ago and vaguely remember that that worked.  It might be
> > another option to try and put a bit of trampoline code at e....... to
> > toggle the sync/async boot flag (relocating the RAM at 0....... to
> > c.......) before decompressing the kernel to the c....... area.)
> >
> >
> >> I don't know the market, are there other boards / manufacturers that
> >> would benefit from a proper fix?
> >
> > No idea, really.  I have four ep93xx boards, and none of them have
> > this issue.
> 
> FWIW, other OS' have exhibited similar short-sightedness regarding
> assumptions like this about physical memory layout.  We talked to a QNX
> engineer about a port to the ep93xx and he expressed doubts that the QNX
> VM system would tolerate such fragmented memory without kernel rework
> ($$$).  About the only OS' that didn't mind the heavily fragmented memory
> were the BSD's.  Just need to call uvm_physload(physaddr, sz) for each
> fragment of memory there IIRC.
> 

> >
> >> > Another option _might_ be to set PHYS_OFFSET to e0000000, and tell
> >> > Redboot to load the kernel at 0xe0008000.  This will likely mess up
> >> ATAG
> >> > passing (so you'll have to comment out the boot_params line in
> >> ts7250.c,
> >> > hardcode the machine ID and pass mem= parameters for every memory
> >> block
> >> > by hand), but it'll cause physical RAM to be within a 1G range
> >> (e0000000
> >> > .. 1fffffff, (ab)using 4G address wrap) and might just work.
> >>
> >> Yes, I saw that you made the suggestion of setting PHYS_OFFSET to
> >> e0000000 earlier this year on the ts-7000 yahoo list [2].  I tried that
> >> as well, but naively, didn't realize the other changes needed to be
> >> made.
> >
> > If you end up trying this, I'd be curious to know whether it works or
> > not.

For the record, I tried this without success: didn't even make it to
"Uncompressing the kernel ...":

RedBoot> load -r -b 0xe0008000 -h 192.168.2.1 -m http /~ccm/zImage
Specified address (0xe0008000) is not believed to be in RAM - continue
(y/n)? y
Raw file loaded 0xe0008000-0xe0147413, assumed entry at 0xe0008000
RedBoot> exec -c "mem=8M@0xe0000000 mem=8M@0xe1000000 mem=8M@0xe4000000
mem=8M@0xe5000000 mem=8M@0x0 mem=8M@0x1000000 mem=8M@0x4000000
mem=8M@0x5000000 console=ttyAM0,115200 root=/dev/nfs
nfsroot=192.168.2.1:/home/ts7200 ip=192.168.2.30"
Using base address 0xe0008000 and length 0x0013f414
$T050f:00802100;0d:14a20400;#14   <--- Interesting! 5-40 sec delay here

I tried without mem= on the command line, with mem= high banks first,
and with mem= low banks first, all with similar results.  Not quite sure
why the boot_params had to go though.

> >
> >> Is it relatively simple for you to properly fix the problem once you
> >> have access to an afflicted board?
> >
> > I think so.  [ Maybe I can trade in my 32M ts7250 for a 64M model. :) ]
> 
> No problem.  Where would you like it sent?  We could probably give you a
> different board too if you wanted.  Since the TS-7250, we've released a
> 7260, 7300, and 7400 based on the same processor.
> 
> //Jesse Off

Great!  It would be great if TS could tell customers 'our boards work
out of the box with Linux 2.6', even if 2.6 is not officially supported
(yet!) by TS.  Lennert has done an outstanding job in getting ep93xx
support upstream, and most of the bits are in place already.  More 2.6
users on TS boards == more development on TS boards, as almost all
new/interesting development is done on 2.6.

Regards, ......... Charlie

Other related posts: