Land Leveling Tools - Causing Region Issues
tracked
Mark Legendary
There have been long-standing issues with land leveling tools being left out on regions for too long, which causes performance problems that often require a region restart to resolve. While a restart is only a short-term solution, this has been a known issue for quite a number of years now. I believe it is time for Linden Lab to look into this again and implement a proper fix for a permanent resolution. It would be very much appreciated.
Thank you,
Mark
Log In
Maestro Linden
updated the status to
tracked
Maestro Linden
Checking back at my test location, which has had both the "LAND_LEVEL every 100ms" script _and_ "Alternate LAND_RAISE / LAND_LOWER every 2 seconds" running constantly for the past 3 days of uptime, I'm not seeing any significant increase in physics memory usage beyond the initial growth from when I first added the objects. The physics memory usage I see right now is 80MB, or about 8% of the maximum allowed for a full region before object rez starts to fail.
It seems likely to me that something is missing from the recipe - perhaps certain other physical objects in the region? I think we need some other details, such as either:
- Names of regions where this issue is recurring
- Some more involved steps for how to make it happen on demand on any given region
Mark Legendary
Maestro Linden One thing that we know for certain is that when the issue arises on a region, action taken is:
- The region is restarted
- The land leveling tool is removed
After this has been done and with the land leveling kept off the region completely and without any other changes made to the region, there is no further issues that recur, even after days or weeks of observation. If a land leveling tool is placed back on the region, used and kept there again, this will create performance issues again anytime within a few days.
This strongly indicates to me that the land leveling tool is the root cause. However, it is possible that other region-specific variables or conditions must be present simultaneously for the tool to trigger these negative effects.
The most recent occurrence was on the Legendary Dynasty region. I also ask that you to please review ticket #2526451 and reach out to Rocky Linden, who has processed several similar tickets and may have some additional insight into this recurring issue.
I am also wondering if some of the following could be something that is happening or could contribute to the cause:
- If a tool is heavily used on a region and left out, could this trigger navmesh rebakes and cause high memory use as a result impacts the regions performance?
- Could these tools being used along a stand alone regions edge create a scenario where it would affect a regions performance?
- Could pathfinding being disabled on a region have a difference to a region with this setting enabled in the debug console?
Maestro Linden
Mark Legendary: Okay, thanks for that context. The freshdesk tickets don't offer much context beyond you reporting the issue and support acknowledging it as a known issue type, but talking to the support team I see the internal issue that we're tracking this as.
I think the fix will entail some sort of punitive measure against excessive llModifyLand use, since terraforming _is_ an expensive operation. Something like a long-term throttle or sending the owner a system chat message on each LSL function call could help.
Maestro Linden
Mark Legendary pointed me to the marketplace item that's in use in the affected region. The item consists of full perm objects and scripts, each written for a given brush size. The medium-size script looks like this:
default
{
state_entry()
{
llSetTimerEvent(0.1);
}
timer()
{
llModifyLand(LAND_LEVEL, LAND_MEDIUM_BRUSH);
}
}
Having a script call a terraform operation 10x a second does not seem like a great idea for viewer or simulator performance. Luckily, the actual step to rebuild the terrain's physics is deferred for a bit and does not run very frequently, and
llModifyLand()
has no effect unless the script owner is present in the region and also owns the parcel, so the griefing potential is pretty limited.I'm not immediately seeing massive memory growth with this script rezzed in my test region, though there are a lot of minor updates visible in the terrain shape, probably due to rounding error, each time the 'land level' operation triggers.
Maestro Linden
updated the status to
needs info
Mark Legendary I take that the server bug for this issue is that simulator performance can suffer when certain scripts call llModifyLand constantly to update the shape of the terrain.
Can you please point to some example scripts that reproduce this issue, and how you're measuring the performance drop?
Maestro Linden
I set up a scripted object on land that I own which constantly raises and lowers the terrain:
integer c;
default
{
state_entry()
{
llSetTimerEvent(2);
}
timer()
{
if(++c % 2) llModifyLand(LAND_RAISE, LAND_MEDIUM_BRUSH);
else llModifyLand(LAND_LOWER, LAND_MEDIUM_BRUSH);
}
}
With this script running in a sim that's otherwise quiet, I'm seeing no impact to performance - the 'spare' time metric is no different from before the script started to run.
Mark Legendary
Maestro Linden Thank you for your reasponse and attention into this issue.
It can take a number of days for the region to start experiencing issues when a land leveling tool is left out on the region, normally within 5 days.
I have sent you a message in-world with a ticket number that has some more information contained in a conversation I had before about this issue that I hope may be more helpful to you.
Maestro Linden
Mark Legendary: Reading the ticket (from 2024), I see that the issue you encountered was a "object that caused problems in this region cannot be created" error message after several days of region uptime. Are you still seeing those symptoms? If so, in which region(s) is this occurring in? The SLURL named in the freshdesk ticket no longer exists.
I know from experience that the "object that caused problems in this region cannot be created" message can occur for every object rez attempt if the simulator's physics memory usage exceeds a certain threshold. The physics memory usage can actually be viewed in the viewer via Statistics -> Simulator -> Physics Details -> Memory Allocated. For a full region, I would expect to start seeing those errors once the physics Memory Allocated stat gets above 900MB.
Since terraforming updates the physics shape of the terrain, it seems plausible that doing so continuously could cause memory allocations for in physics engine. Those operations shouldn't cause the growth to be unbounded, but perhaps that is the case for your region. I think it would be interesting to see if "Memory Allocated" grows significantly in your region while the land leveling tool is rezzed for a few hours, compared to a similar interval when the tool is not rezzed.
I have the continuous-terraform test script from my previous comment rezzed in a test region, and am attempting to see if "Memory Allocated" grows significantly. So far with 20 hours of uptime, it's sitting pretty level at the 49-52MB range. I'll check back in a few days.
Mark Legendary
Maestro Linden As acknowledged in the ticket (from 2024) it was an issue that Linden Labs had been aware off and had seen an increase in support cases with land leveling tools causing issues on a region. We are still seeing this issue arise from time to time as no fix has been implimented to prevent this from happening when one or more of these tools are left out on a region for too long.
The most recent case experienced was around one week ago on region Legendary Dynasty. Actions taken was restarting the region and then removing this tool from the region when it is not in use. This has prevented the issue from continuing for now, but it will happen again when someone that is unaware of this uses one of these tools on a region and leaves it on the region for too long.
I think that alot of residents are unaware that these tools can cause region issues by leaving them out for too long and without a fix being implimented to prevent these tools from impacting a regions performance is what causes this to continue to happen.
There will be no immediate impact that you will see on the region, it will take a few days but hopefully it will give you more specific data once it starts to impact the region you are testing this on.
In the meantime, I will gather more information for you that will hopefully help.
AlettaMondragon Resident
It's generally not a good idea to leave the lawn mower outside. Just put it in the shed.
Could you be more specific about which tools cause problems and what kind of performance issues you noticed while using them or leaving them rezzed for a long time?
JenniWindrider Resident
AlettaMondragon Resident AFAIK the issue is with tools that constantly spam llModifyLand() which prevents the region from re-baking.
Dhalgren Correia
In order to vote, please press the button with the small arrow up just near the title.