[haiku-development] Re: jemalloc has been ported to haiku. some discussion
- From: "Axel Dörfler" <axeld@xxxxxxxxxxxxxxxx>
- To: haiku-development@xxxxxxxxxxxxx
- Date: Sun, 18 May 2008 21:25:00 +0200 CEST
"JiSheng Zhang" <jszhang3@xxxxxxxxx> wrote:
> I have ported jemalloc(FreeBSD current malloc implementation,
> licensed
> under 2 clause BSD license) to haiku. Although some thing is not
> resolved, but it works. I haven't compare the jemalloc's performance
> with haiku's current malloc version.
>
> Haiku's malloc implementation is derived from hoard LGPL version.
> There are common issues during Hoard and jemalloc design, such as
> avoiding false cache line sharing, Scalability and low fragmentation
> etc. From 3.3.0 hoard is licensed under GPL which is not compatible
> with haiku's license. Also the implementation is changed a lot, it
> depends on "heaplayer" from then on. I found the code is complex and
> not that readable. So, Can we take jemalloc as haiku's default
> implementation?
I don't know enough about jemalloc to be able to make a really useful
comment. Do you have any pointers, overview, benchmarks? I just know
Mozilla is using jemalloc.
Before we switch allocators just for fun, I would like to really
investigate our options. A good generic allocator should be designed
for speed in a heavily multi-threaded environment. Of course, low
fragmentation and memory usage would be desirable goals as well :-)
I would like to see some benchmarks, too, especially SMP tests.
> shortage of haiku's current malloc: it seems that the memory
> allocated
> will not give back to system at all when the app does not exit. If a
> long live app such as firefox allocate a large memory, use it and
> then
> free it. the memory now is not needed, but it can not be given back
> to
> system at all. It will cause lack of memory, also a lot of swap then
> a
> lot of harddisk I/O when swap available.
I don't know how you tested that, but I'm pretty sure it's not correct.
Of course, Hoard does not give back the memory back immediately, but
caches it for future use, but that doesn't mean it won't be given back
ever.
> Some question about haiku's mm subsystem:
> 1.how can I get the amount of free memory? get_system_info (info)
> then
> info.used_pages? If so, a simplet test app(source is atached) implies
> that first, haiku's physical page allocation isn't the lazy one;
> second, current munmap does not actually free memory.
I don't know about munmap() (it could be implemented incorrectly in
this regard), but the used_pages count mirrors how many pages are
committed by user apps. That doesn't mean they are actually used - you
can't use it to deduce actual page allocation.
I always wanted to add a way to retrieve some VM statistics from
> 2.an area allocated by create_area, how can I free the leading part
> of
> the area? For the tailing part resize_area can be used.
There is no way to do that.
> there are 2 ways of getting memory from system in current jemalloc
> ported to haiku:
> 1.anonymous mmap, but the munmap seems never give memory back to
> system
It probably does, but it might forget to uncommit the memory, that is,
the usage counter might not be updated. It's relatively new, and hasn't
tested a whole lot yet.
> 2.resize_area, the same as current haiku's malloc. but it will race
> with threads that call create_area and resize_area on the same
> virtuall address range directly.
It doesn't, since the heap is using a reserved area to cover the space
after the heap area: this space will only ever be used when there is no
other space left. And even then, it's filled from bottom up.
> PS:IMHO, there is no need to implement the the sbrk system call.
> For
> one thing, it is excluded from the POSIX.1 standard; for another,
> modern software does not use it any more.
I'm afraid we have to provide it for compatibility. When adding our
current allocator, I originally left it out, too, but it turned out to
be needed by some software.
Bye,
Axel.
Other related posts: