[haiku-gsoc] Golang port: summary of the second half-term
- From: "Zhuowei zhang" <dmarc-noreply@xxxxxxxxxxxxx> (Redacted sender "zhuoweizhang@xxxxxxxxx" for DMARC)
- To: haiku-gsoc <haiku-gsoc@xxxxxxxxxxxxx>
- Date: Fri, 22 Aug 2014 02:14:39 -0700
Hello,
The goal of my GSoC project was to port the reference compiler for the Go
programming language to Haiku. I believe that I have furfilled that promise:
the Golang port is mostly functional (the standard library compiles, most tests
pass, and the Golang tour works on the ported runtime).
The port can be built using the golang recipe in Haikuports: to build, setup
Haikuporter and Mercurial, then type
haikuporter --no-dependencies golang_x86
if you're on a GCC2 hybrid, and
haikuporter --no-dependencies golang
otherwise.
What has been fixed during the second term:
- spawning new processes
- checking for stack overflows
- handling signals
- getting system time
- pipes
- any part of the runtime that needs the Go tool
- Other architectures - the runtime is also working on x86_64
What still remains to be fixed:
- TLS storage of the M and G structures
- address space reservation for the Golang heap on 32-bit
- Some functions in the Net package
- the port lacks a network poller, which is used to speed up networking
- listing directories' content one item at a time (currently only listing all
items at the same time, as used by the Go tool, is supported)
- various other tests in the standard library are not passing
- Cgo is not working yet
- the port is based on Golang 1.3; a port for 1.4 is required for upstreaming
the port
- cleanup is needed for upstreaming: the port accumulated quite a bit of //
FIXME: haiku comments
During the second half-term, the focus was to get every package in the standard
library building. First, I kludged around the memory reservation issue by...
not reserving any memory at all. (Later, I switched to the approach of mmapping
the memory, and then immediately un-mmapping it again. That's still not working
properly, and prevents the runtime from using all available memory on 32-bit,
but for most workloads this should be fine) That at least allowed fork/exec to
run, and after a week, the Go tool was able to successfully build all the
packages in the standard library.
After building all the packages, I was able to create a list of constants and
structures from the host headers using the Cgo tool, which, as I mentioned in
the last blog post, is written in Golang itself and depends on the standard
library being functional. The list of constants and structures are used to call
into libroot, and thus they must match what libroot uses and returns. This
includes the list of errnos, the list of signals, and more. With these files
regenerated, most functions in the syscall and runtime packages started working
properly.
Afterwards, I tried enabling signal handling and timers -- for both tasks, my
initial attempts didn't work, and I only figured out the confusing causes of
the problems after careful review.
For timers, when I tried adapting the Solaris code for Haiku, I was greeted
with the error message "stack expansion during syscall". In Golang, most
methods have a segmented stack - the stack for each thread can grow and shrink
as needed. However, in the semaphore code that controls sleeping, expanding the
stack could create deadlocks, so the Solaris code was careful to avoid calling
methods that require stack expansion. So why did this message appear when no
method called required stack expansion? It turns out that the method used
64-bit division, and on 32-bit x86, this is implemented with a call to a method
using a segmented stack, unlike on 64-bit amd64, where there are dedicated
instructions for 64-bit division. Replacing the division with a call to a
non-segmented method allowed timers to function.
For signals, whenever a signal was received by the runtime, the program would
suffer a segmentation fault in some completely unrelated code. The signal
handler I put together from parts of the Solaris amd64 handler and the FreeBSD
386 handler seemed to work right: it saves all the registers needed, and seems
to save the state of both the M and the G structure, which are used by the
Golang scheduler. It turns out that I did not notice that the register for the
M structure was overwritten when I restored the G structure, so I ended up
restoring what should be written into the M structure into the G structure
instead, confusing the scheduler and causing the segfaults.
At this point, I tried running the Golang Tour on Haiku, and noticed another
"impossible" bug: a timer with an interval below 1 second gets rounded up to 1
second. After staring for a while, I realized that the time that the runtime
reports was only precise down to seconds: anything below seconds was reported
as zero. Since the timer sleeps until the time changes, this limited the
minimum sleep interval to 1 second. It turned out I had returned the
nanoseconds in the BX register; the code expected it to be in the DX register.
Sigh.
During the last part of the work period, I worked on porting the runtime to
Haiku x86_64 as well. This took much less time than the x86 port, partly
because the Solaris port, which the Haiku port is based on, was developed for
x86_64 only, so I was able to use much of its code unmodified in the x86_64
port.
Finally, I submitted a HaikuPort recipe for Golang; it was merged; the package
isn't being built automatically right now, but can be built with haikuporter
manually, as stated above.
In the future, I would like to update this port to Golang 1.4. I've looked at
Golang 1.4 back in July, and it seems that the only major change was that the M
and the G structures (which, as I mentioned, are used in the scheduler) no
longer takes up two TLS slots: instead, only the G structure is stored in TLS,
and points to the active M structure. That should be a minor change, mostly in
the linker, the initialization code, and the signal handler.
The repository for this port is at
https://bitbucket.org/zhuowei/go-1-3-haiku/commits/branch/porthaiku2 , under
the porthaiku2 bookmark.
Again, I would like to thank everyone on the haiku-gsoc mailing list,
especially my mentor, Bruno Albuquerque, for their encouragement and advice
during this work period.
Zhuowei Zhang
Other related posts:
- » [haiku-gsoc] Golang port: summary of the second half-term - Zhuowei zhang
