LSL-Luau running out of memory from relatively small lists
complete
Frio Belmonte
This started happening sometime after yesterday's (Mar 11) Luau test region restarts.
Allocating lists that take up as little as 12 kB under Mono, in an otherwise empty script, will cause an out of memory error under LSL-Luau.
Repro script: works under LSL-Mono and uses ~16 kB memory (4 kB code + 12 kB from the lists), runs out of memory under LSL-Luau.
default
{
state_entry()
{
list t1; list t2; list t3; // simply for speed under mono, appending to bigger lists gets slower
integer i;
for(i = 0; i < 1024; ++i) {
t1 += "";
t2 += "";
t3 += "";
}
llSleep(0.01);
llOwnerSay("memory used: " + (string)llGetUsedMemory()); // would be wrong under Luau anyway
}
}
Log In
H
Harold Linden
complete
This should be fixed with the latest SLua alpha, please let us know if there's still any issues!
H
Harold Linden
inĀ progress
We've written some tooling to make SLua's garbage collector aware of the memory limits when deciding how fast to work, this will be included in the next SLua update.
H
Harold Linden
Note that this script will take 40k of memory in the new LSL VM, but that's just because you're only storing a constant string in a list.
If you use any string value that isn't from a string constant, or something that needs boxing like an
integer
then Mono will use 50kb minimum. This is basically because list
s in Mono are object[]
arrays under the hood, and each list entry only takes sizeof(void *)
(4 bytes for us) if it doesn't require any extra storage for the value. Changing the code to t1 += 0
, for example, will take 50kb under Mono because the value has to be boxed in an Int32
to be stored in an object[]
array.By comparison, Luau takes minimum 16 bytes for _any_ entry in a list rather than the 4 minimum for Mono, but in practice memory usage ends up being less in Luau for real scripts due to not needing to value boxing being "baked into" the memory cost, and string interning reducing the cost of repeated non-constant strings, which Mono will not do.
H
Harold Linden
FYI, we're reworking the Luau garbage collector's collection strategy to deal with this. Luau by default is intended to run in an environment where you want incremental GC spread out across the entire script's run, and the heap for a single VM is allowed to grow up to 6GB. Obviously that's not workable under SL's creation model where you have "untrusted" scripts that don't belong to the region owner running in the region.
We've implemented a heap size limit, but the garbage collector isn't currently aware of it, and tries to incrementally clear out small bits of garbage as the script runs, so as not to cause long GC pauses. In general, that's what you want, but the garbage collector isn't currently aware of the memory limits we've imposed, so it doesn't know it needs to be a little more urgent about its garbage collection as it approaches the memory limit.
Naturally, concatenating tons of lists over and over in a tight loop creates a ton of garbage that the garbage collector can't currently keep up with. The reworked garbage collection strategy will take memory limits into account to make sure you don't run out of memory for typical usage.
Bandor Beningborough
I confirmed this with a script which loads some data into LSD. After it has run, LSD memory usage was comparable to the Main GRID, but free script memory on the main grid was 48K and in LSL 2025 it was 16,000.
H
Harold Linden
Bandor Beningborough
llGetFreeMemory()
doesn't return meaningful results currently under SLua, we don't want people relying on it too much for script logic before we fix up the garbage collector.Signal Linden
planned
Thanks! The memory bookkeeping is somewhat fiddly right now. We'll have a look at what's going on here.