Yeap. GC64 is still experimental.
Try to reduce the issue down to a standalone small script.
It often helps to stress the code under suspicion by looping it
100-1000 times and/or reducing "hotloop" LuaJIT setting down to 3-5.
2017-05-18 8:14 GMT+03:00 Caswal Parker <caswal@xxxxxxxxxxxxxxxxxxxx>:
I have quite a confusing and very troubling issue that I am finding
impossible to solve.
I had an original issue of running into the 2GB memory limit with 2.0.4,
Peter Crawley advised me to give the new GC64 2.1 version a try, and it did
resolve my issue
(https://www.freelists.org/post/luajit/Memory-Allocator-Issues). But I have
a new weird sporadic corruption of calculations in my game. It is relatively
easy to replicate in my project, but I cannot produce a smaller test case
replicating the issue. LuaJIT is embedded into an Unreal Engine 4 project,
and this issue is most prevalent in Packaged/Cooked builds, rather than in
Editor. Cooked builds are much better at consuming homogenous blocks of
memory than the Editor, and will easily consume the first 2GB of memory
space before LuaJIT has initialised in my project.
The Lua scripts in question have been stable, using LuaJIT for the past 4
years, used on our previous 32bit game version. These calculations involve
calculating the power and torque of a car engine using a parametric model.
Doing repetitive recalculations it eventually will produce garbage values.
E.g. 0/0 NaNs, or large negative numbers.
The loop that produces the base torque power curves is the following:
local torque = 0
while(rpmIter <= 12000 and rpmIter <= (maxRPM)) do --If the engine runs
out of power, the loop with break out
self.EngineInfo.ModelInfo.Results.Restrictions[iter] = { }
power[iter] = { }
power[iter][1], torque = rpmIter, FinalCamCurve[rpmIter]
power[iter].RPM = rpmIter
--Adjust torque based on compression:
--Old linear power adjustments
local compressionFraction = (100 - (
-0.01717172*(self.EngineInfo.ModelInfo.Compression)^3+0.80075758*(self.EngineInfo.ModelInfo.Compression)^2-13.82189755
*(self.EngineInfo.ModelInfo.Compression)+85.09891775)) * 0.01
torque = torque * compressionFraction
--Work out how much is lost to friction
local loss = self:GetFrictionValues(rpmIter) * 0.75
--print("Loss at RPM: " .. rpmIter .. " is: " .. loss)
--Do some intake Resonance Stuff
local intakeResonance = self:DoIntakeResonance(rpmIter)
local exhaustResonance = self:DoExhaustResonance(rpmIter)
--Swap these lines to see the Torque lost to friction
torque = torque + torque * (intakeResonance) *
self:GetQualityForPart(self.EngineInfo.ModelInfo.FuelSystem) + torque *
(exhaustResonance) *
self:GetQualityForPart(self.EngineInfo.ModelInfo.Headers)
local valveFloat = self:GetValveFloat(rpmIter) --DO we have any
valveFloat?
self.EngineInfo.ModelInfo.Results.Restrictions[iter].ValveFloat =
valveFloat
torque = torque * valveFloat --DO we have any valveFloat?
--torque = torque * PowerDueToFuelTune --Has the tune affected our
powers?
--Filters steel torque!
torque = torque * (1 - stolenTorque)
--Process Power lost due to cat
torque = torque * PowerLostToCat
torque = torque - loss
power[iter][2] = torque
power[iter][3] = rpmIter * power[iter][2] / 9549
power[iter][5] = loss --Will need to record this for car's economy
if torque > 0 and foundIdle == false then --Have we found the idle rpms
self.EngineInfo.ModelInfo.Results.Idle = rpmIter
foundIdle = true
end
if torque <= 0 and foundIdle == true then break end
if torque > 0 then
if not self.EngineInfo.ModelInfo.Aspiration.IsTurbo then --we can't do
any of these calculations if the engine doesn't even have any power yet
self:CalculateNA(power, iter)
else
self:CalculateTurbo(rpmIter, turboCurves, power, iter)
--power[iter][2] = power[iter][2] * turboCurves[rpmIter].RealPR
--power[iter][3] = power[iter][3] * turboCurves[rpmIter].RealPR
--peakBoost = math.max(peakBoost, turboCurves[rpmIter].Boost - 1)
if turboCurves[rpmIter].Boost - 1 >
self.EngineInfo.ModelInfo.Results.PeakBoost then
self.EngineInfo.ModelInfo.Results.PeakBoost = turboCurves[rpmIter].Boost
- 1
self.EngineInfo.ModelInfo.Results.PeakBoostRPM = rpmIter
end
end
end
if torque <= 0 and foundIdle then
maxRPM = rpmIter
break
end
powerCut = power[iter][2]
rpmIter = rpmIter + 100
iter = iter + 1
end
It normally produces a table like this:
{
{
500,
106.77286621740724,
5.5907878425702817,
[5]=0.6468888280108217,
RPM=500,
},
{
600,
0.034794449408922359,
37.957581173369839,
[5]=0.68401462161839932,
RPM=600,
},
{
700,
115.95296264529469,
8.5000600954766234,
[5]=0.72789055951826387,
RPM=700,
},
------- .... An So on
{
5800,
147.93325983186156,
89.853692221677363,
[5]=11.916254723983693,
RPM=5800,
}
}
If the game makes repetitive requests to recalculate the engine data, after
a few attempts I will get a resulting table similar to this:
{
{
500,
108.73974885306927,
5.6937767752156914,
[5]=0.6468888280108217,
RPM=500,
},
{
600,
0.034794449408922352,
37.957581173369839,
[5]=0.68401462161839932,
RPM=600,
},
{
700,
-526859087318.39368,
-368801361122875.56,
[5]=109.19215743256311,
RPM=700,
}
--No more results, as loop stops as torque (index 2) is < 0
}
With values just being jibberish. I have done a few tests storing results of
the various modifiers, and on a certain loop iteration, they will be a
random value.
If I wrap the offending while loop inside a jit.off() jit.on() set, then the
code works as expected, never producing corrupted results. It also works
correctly with a jit.flush() as the first line in the loop. Running the
project with jit.opt.start(0) does not alleviate the issue either.
My attempted test case was to make a simple application that grabbed the
first 2GB of memory using the same NtAllocateVirtualMemory call as LuaJIT
2.0.4. Create a new state, and run a cut down version of the above code with
fewer dependencies, but it does not exhibit the same behaviour.
How can I go about producing some more information to help? Can I get a dump
of the current bytecode of each iteration of the loop?
I can cope with jit.flush() to ease the issue, but I am worried that could
be something subtly broken in some edge-case corner I have worked myself
into.
Any help massively appreciated on this issue, it is pretty critical for us
to get this fixed.
Cheers, Caswal
--
---------------------------------------------------------------------------------
Caswal Parker
Co-Owner/Lead Programmer - Camshaft Software
Skype name: caswal
http://camshaftsoftware.com
