Hi!
For a while I have been puzzled by an issue that sometimes pops up when
using the rust compiler (rustc). When compiling something large, errors
would pop up that it was impossible to join a thread. That's why I have
been maintaining a workaround in the Rust builds for Haiku [1].
I made it my summer project to see what the root cause of this issue is,
and I _think_ I found the issue.
When cargo (Rust's build tool) runs, it creates some administrative
threads, and a thread for each of the crates that it builds. The bootstrap
tool [2] for the current rust nightly is build in 59 crates, this equals
about 59 threads. The worker threads end when the individual crate is
built. The administrative thread runs just about to the end. When the
process is complete, the main thread wants to join() each of the threads,
but pthread_join() returns B_BAD_THREAD_ID for the first worker thread and
in the unpatched code this triggers a failing assert. [3] Through adding
extra debug output, and inspecting the kernel, I found that the threads
that cargo wants to join have indeed existed. So why is the kernel
complaining?
After some investigation in the thread code, I found that our kernel keeps
a list of 'dead' threads exactly for the purpose of being able to join them
later. This is where the ended worker threads end up. Unfortunately, the
list of dead threads is limited to MAX_DEAD_THREADS (set to 32). The header
file that defines the limit implies that it is a soft limit [4], but it
really is a hard limit because if the list grows larger the oldest entries
are thrown away [5]. If I make the limit higher, the assert does not
trigger anymore and all threads are joined.
The question I have is this: what would be the proper way to fix this?
Personally, I would propose to up the limit of MAX_DEAD_THREADS to the max
number of threads (currently set to 4096)[6], with the rationale that if
you can create 4096 threads, you should be able to join them after they're
dead. I know that you can still hit the limit if you create more threads
after some of them ended, but it sounds like a more sensible constraint.
Any thoughts?
Regards,
N>
[1]
https://github.com/nielx/rust/commit/0737ac6c161d297f1153cef835e165fd7acf4e9a
and https://github.com/nielx/rust/issues/3
[2] https://github.com/rust-lang/rust/tree/master/src/bootstrap
[3]
https://github.com/rust-lang/rust/blob/3e3c552605180d739eb6c23bc657121679caa456/library/std/src/sys/unix/thread.rs#L193
[4]
https://git.haiku-os.org/haiku/tree/headers/private/kernel/thread_types.h#n99
[5] https://git.haiku-os.org/haiku/tree/src/system/kernel/thread.cpp#n2147
[6] https://git.haiku-os.org/haiku/tree/src/system/kernel/thread.cpp#n82
