[haiku-development] Re: jemalloc has been ported to haiku. some discussion
- From: "JiSheng Zhang" <jszhang3@xxxxxxxxx>
- To: haiku-development@xxxxxxxxxxxxx
- Date: Mon, 19 May 2008 10:30:57 +0800
2008/5/19 Axel Dörfler <axeld@xxxxxxxxxxxxxxxx>:
> "JiSheng Zhang" <jszhang3@xxxxxxxxx> wrote:
>> I have ported jemalloc(FreeBSD current malloc implementation,
>> licensed
>> under 2 clause BSD license) to haiku. Although some thing is not
>> resolved, but it works. I haven't compare the jemalloc's performance
>> with haiku's current malloc version.
>>
>> Haiku's malloc implementation is derived from hoard LGPL version.
>> There are common issues during Hoard and jemalloc design, such as
>> avoiding false cache line sharing, Scalability and low fragmentation
>> etc. From 3.3.0 hoard is licensed under GPL which is not compatible
>> with haiku's license. Also the implementation is changed a lot, it
>> depends on "heaplayer" from then on. I found the code is complex and
>> not that readable. So, Can we take jemalloc as haiku's default
>> implementation?
>
> I don't know enough about jemalloc to be able to make a really useful
> comment. Do you have any pointers, overview, benchmarks? I just know
> Mozilla is using jemalloc.
> Before we switch allocators just for fun, I would like to really
> investigate our options. A good generic allocator should be designed
> for speed in a heavily multi-threaded environment. Of course, low
> fragmentation and memory usage would be desirable goals as well :-)
> I would like to see some benchmarks, too, especially SMP tests.
The usable information now we can get is their document.
For hoard(http://www.cs.umass.edu/%7Eemery/hoard/asplos2000.pdf)
For jemalloc(http://people.freebsd.org/~jasone/jemalloc/bsdcan2006/jemalloc.pdf)
For SMP benchmarks, I can't do it myself for having no SMP box. But
IMHO, their performance is similar. I will send the patch tonight
>
>> shortage of haiku's current malloc: it seems that the memory
>> allocated
>> will not give back to system at all when the app does not exit. If a
>> long live app such as firefox allocate a large memory, use it and
>> then
>> free it. the memory now is not needed, but it can not be given back
>> to
>> system at all. It will cause lack of memory, also a lot of swap then
>> a
>> lot of harddisk I/O when swap available.
>
> I don't know how you tested that, but I'm pretty sure it's not correct.
> Of course, Hoard does not give back the memory back immediately, but
> caches it for future use, but that doesn't mean it won't be given back
> ever.
Hoard caches some blocks for future use. When the cache block is large
enough, it will give back to system. But haiku now only link the freed
memory in a global free list without giving back to system.
>
>> Some question about haiku's mm subsystem:
>> 1.how can I get the amount of free memory? get_system_info (info)
>> then
>> info.used_pages? If so, a simplet test app(source is atached) implies
>> that first, haiku's physical page allocation isn't the lazy one;
>> second, current munmap does not actually free memory.
>
> I don't know about munmap() (it could be implemented incorrectly in
> this regard), but the used_pages count mirrors how many pages are
> committed by user apps. That doesn't mean they are actually used - you
> can't use it to deduce actual page allocation.
Does it means there's no available method to test memory usage? IIRC,
the aboutsystem application get the amount of free memory in this way.
> I always wanted to add a way to retrieve some VM statistics from
>> 2.an area allocated by create_area, how can I free the leading part
>> of
>> the area? For the tailing part resize_area can be used.
>
> There is no way to do that.
>
>> there are 2 ways of getting memory from system in current jemalloc
>> ported to haiku:
>> 1.anonymous mmap, but the munmap seems never give memory back to
>> system
>
> It probably does, but it might forget to uncommit the memory, that is,
> the usage counter might not be updated. It's relatively new, and hasn't
> tested a whole lot yet.
>
>> 2.resize_area, the same as current haiku's malloc. but it will race
>> with threads that call create_area and resize_area on the same
>> virtuall address range directly.
>
> It doesn't, since the heap is using a reserved area to cover the space
> after the heap area: this space will only ever be used when there is no
> other space left. And even then, it's filled from bottom up.
>
>> PS:IMHO, there is no need to implement the the sbrk system call.
>> For
>> one thing, it is excluded from the POSIX.1 standard; for another,
>> modern software does not use it any more.
>
> I'm afraid we have to provide it for compatibility. When adding our
> current allocator, I originally left it out, too, but it turned out to
> be needed by some software.
>
> Bye,
> Axel.
>
>
>
Other related posts:
