@ Laurent Deniau
That was a terrific hint, thank you very much! I suppose you had to study the
LuaJIT implementation for a long time to work that out.
I did an extensive series of test runs and was able to reproduce your findings.
When the Address Space Layout Randomisation (ASLR) is switched off, I see long
sequences of similar durations like you did. ASLR can be switched off on my
machine by setting /proc/sys/kernel/randomize_va_space to "0" (it is "2" by
default).
But it is by no means the case that the performance is permanently constant on
my machine when ASLR is switched off. There is still the sudden halving of
speed, just much less often. For example, I had three sequences with eight runs
each, which were OK, then the fourth sequence was halved on the second run. In
the fifth sequence the halving came at the fifth run. And then came five
sequences with eight runs each, where the performance was consistently good.
Each sequence has its characteristic durations. In the first sequence of the
given example, all durations were about 12.1 seconds; in the second sequence,
13.4 seconds was the predominat duration, and so on. The geometric mean of all
successful sequences is 12.96 seconds, which is about two seconds more than the
fast state in my original tests (see
http://software.rochus-keller.info/St80LjVirtualMachine_0.6_Performance_Report_2020-07-14.pdf).
It's also noteworthy that in a sequence where the speed halving occurs, the
first run is usually around 10 seconds, as significantly faster than all other
runs. And also the slow runs are again usually 21 seconds.
There were also strange effects when I moved the executable to a different
location in the filesystem. The frequency of sequences with speed halves
increased massively, and it was difficult to get back to a longer phase of
successful sequences. There is a transient phase in most cases when I switch on
or off ASLR. The observations do not yet allow any clear conclusions to be
drawn. However, I have the impression that you also have to recompile the
executable if you switch off ASLR to reduce the probability of bad sequences.
And every time I think I have found a consistent set of rules, there comes
another observation that throws everything off track. I'll keep experimenting
and get back to you.
Best
R.
_______________________________
From: Laurent Deniau
Sent on: Thu, 16 Jul 2020 16:18:41 +0200
To: <luajit@xxxxxxxxxxxxx>
Cc:
Subject: Re: Inexplicable behaviour of JIT (sudden speed-down ofthe
samecodethat was running fast at first) in a long-running Lua program,update
We experienced significant performance variations between runs too, but once
you disable the Address Space Layout Randomisation of your OS, the performance
are almost deterministic from run to run. Unfortunately this is only a
temporary solution for debugging and it does not explains degradation during a
single run (unless trace flush occurs).
For the same reason, any change in the code can lead to better or worst
performance. See section 4.1.3 in
https://github.com/MethodicalAcceleratorDesign/MADdocs/blob/master/luajit/luajit-doc.pdf
[disclamer: this is our understanding of the cause, not a proof].
I also recently posted that using the profiler (e.g. -jp=vl) stabilises the
problem and even increase significantly the performance. I don’t really know
why though as I would expect the opposite...
Hope this helps a bit.
Best regards,
Laurent.
