Add llGetNotecardLineFast
complete
Signal Linden
Add a new
llGetNotecardLineFast
function with the same signature as llGetNotecardLine
. This new *Fast version will not have a delay if the notecard being requested is already in the region asset cache.Rationale
llGetNotecardLine
is infamously slow, with a built-in delay of 0.1s per line. This adversely impacts all sorts of scripts in Second Life, notably when scripts need to load user-customizable configuration (Furniture, etc.)By introducing a new *Fast permutation of the function (Similar in style to
llSetLinkPrimitiveParamsFast
, etc.) we can allow any behavior that depends on the built-in delay to function as expected while giving creators the ability to opt-in to a faster method.The new function would feature a delay (0.1s or similar) if the notecard being requested needs to be downloaded into the current region's asset cache. This is necessary as notecards can be referenced by asset ID, and presents a more transparent and fair representation of lookup cost.
Log In
This post was marked as
complete
Woolfyy Resident
The real question is : will it be as fast as reading a list declared as constant parameters in a script file ?
Till now i manage thousands of parameters just with script files including a huge list in header ... because it has always been quicker than reading notecards or even parameters in prims.
Moreover is it tens of times simpler to manage and even easy to check as you usually instantly see problems on compile failure pointing out to the line in error.
primerib1 Resident
Woolfyy Resident: This is mostly for user-configurable scripts, so the script is no-mod, and user configure the settings by editing an included notecard.
Woolfyy Resident
primerib1 Resident: this i know perfectly, but my main concern is performance knowing that anyway due to the complexity / interaction mode of my scripts that are constantly being upgraded i don't care about them to be totally open as anyway the next month (week and sometimes day) is already a new version ...
primerib1 Resident
Woolfyy Resident: Well then in your case you can still do it as you're already doing it. Why do you need to change?
Woolfyy Resident
primerib1 Resident: It's not a need : as far as things get better my code moves too ... knowing that performance is my main motto.
Kadah Coba
Dumb question, but how would one cause a notecard to become cached in the first place so that these functions will work?
This description here and on the wiki conflict in this regard: "If the notecard has not been previously cached on the simulator it will return the NAK constant."
Also, please define what "NAK", preferably create the currently non-existing wiki page. :p
primerib1 Resident
Kadah Coba: Signal told me that you load the notecard into the cache using
llGetNotecardLine()
or llGetNumberOfNotecardLines()
Signal Linden
in progress
We've added
llGetNotecardLineSync
to an upcoming release. Marking this as In Progress.Frans Charming
Signal Linden: ++
Without Ordinary
Bugs Larnia
Signal Linden: I noted that the Wiki for llGetNotecardLineSync and other functions has been created, but it is not in the release yet.
Would it be an idea to update Wiki pages for these and future functions with the server version as of which they are expected to be implemented?
Kristy Aurelia
Notecards are great way for users to configure no-mod scripts. I am currently working on scripts that parse data on load, and configure a whole bunch of settings.
Currently I am working on an item that will likely take around 30s to load all the configuration data. It only does it once, and stores it in LinkSetData, but I'm all for, for beign able to do it faster.
Woolfyy Resident
Kristy Aurelia: in modern times, people are used to wait no more than 1 to 2 sec max ...
30 secs always makes internet users think that SL is oldish especially when you have a gaming computer with 32GB memory and the latest NVIDIA card... or simply internet on your phone where all comes instantly like by magic.
So the real question to be solved by LL = how to make creators able to process a setup note card in less than 2 seconds...
The best way to be convinced about it is to have your children's feed-back = that slow timing is unfortunately part of the arguments that make them say that SL is done for grannies and pappies... And I must admit that, anyway, their 1st argument is that there is (for now) no SL viewer though they spend most of their time on their phone. But then, what will you do you do when you Will have just a tiny mono task screen and 30 secs to wait?
Vincent Nacon
While I'm not against this, I can't help but wonder what's the point when we can't write into notecard with LSL. In the last 15 years on SL, I had never really found a good use for llGetNotecardLine(). It's kinda a moot point, since now we already have LinkSetData to play with. With the ability to write the notecard, it would give us a reason to use it as data asset that can be traded with others.
A really good use would be telling people to send me a copy of the Notecard from inside the product that had a bug or a crash. Basically using NC as a data-log to debug stuff.
Coffee Pancake
Vincent Nacon: It's used extensively as a ROM datastore, either as a list of values to be iterated over, and as a hot look up to keep big strings out of a script, etc. Even with link set data .. that data has to come from somewhere, and why waste space in a script when it can be in a notecard.
Chaser Zaks
Vincent Nacon: Last I heard, it is because LL wants to avoid having scripts create assets.
Currently, the way notecards work, is that even if you save a existing notecard, it creates a new asset.
If someone creates a notecard using LSL, it'd be like llSetNotecardLine("line", number);, and each call would create a new asset.
I could think of ways to implement this, such as "region only" assets, and if, and only if the script does llSaveNotecard(), it would save it to the asset server with a long delay to avoid creating millions of notecards.
Though, people create a bunch of junk notecards/scripts/etc all the time when saving/editing a bunch of times, so it shouldn't be too bad if the asset server at least keeps track of reference counts.
Vincent Nacon
Chaser Zaks: Yeah, then that need to change to make it happen. I'm sure there are better ways to do history backup if that's the reason they're doing that.
Andromeda Quonset
Vincent Nacon: I can't imagine, with the size my inventory has grown, of why writing notecards programmatically is such a huge issue when objects get created all the time and waste so much space in an inventory. Also, OpenSim allows for notecard writing, and has for a long long time, and I would think they have fewer resources than Linden Labs.
V
VriSeriphim Resident
Vincent Nacon: Also, notecards are a familiar and simple way for users to configure how script behave - even when the script has mod permission.
Granted, it does require users to carefully edit the configuration, it does avoid complex coding in scripts - the more complex, the more likely to have errors. It's not a bad trade off.
Bleuhazenfurfle Resident
Andromeda Quonset: I'd say it's because you can't count UUID's. Objects can be reference counted (rezzed objects especially), where Notecards (also Textures, Sounds, etc.) can not — how do you know how many references are floating around in scripts as strings? And users can only produce so much junk, a script can do it much faster, every single second of every day essentially forever, just churning out more and more stuff that can never be deleted because it can't be reference counted. A single script could very quickly out-do every notecard you've ever made, and then just keep on going… And thousands of scripts can be running on a region, and there's how many regions across a sim? They simply can not allow scripts to create uncountable numbers of an asset like that. OpenSim must do something different, but changing this for SL now would break so much.
Kadah Coba
LinkSetData is a viable alternative.
primerib1 Resident
Kadah Coba: Users can't edit LinksetData easily, though.
Vincent Nacon
primerib1 Resident: That may change soon with Firestorm viewer. One of the devs made a window to view and edit prim's data. It's still a work in progress though.
primerib1 Resident
Vincent Nacon: Is that even possible? I don't think LL is purposefully exposing the LSD API to public. Remember that scripts run server-side, and many functions are implemented using internal API calls from the scripting virtual machine to (micro)services providing, in this case, Linkset Data functionality.
Relying on an undocumented non-public API sounds like a recipe for future disaster.
Vincent Nacon
primerib1 Resident: Um... sure. I don't need the window though, LSL script is good enough for me anyway.
Gwyneth Llewelyn
Bleuhazenfurfle Resident & Andromeda Quonset — Strictly speaking from my experience, this is not the first time I thought that _sometimes_ Linden Lab should just take a peek into OpenSimulator's code to see what happens there under the hood.
Creating
a notecard is really nothing special: a new
UUID for the notecard is created, its contents are updated using a library function call (note: _that_ is the tricky bit...!), and the old UUID gets discarded, just as if llRemoveInventory()
had been called on it. Why exactly this should be a problem is beyond my understanding; the freshly created notecard is created exactly like any other notecard created by a user — just inside
the object — while deleting the old version from inventory, well, is a standard functionality available since time immemorial, so I can hardly understand what you mean with "impossible to do reference counting" in this scenario. llMakeNotecard()
would simply get throttled like any other function in SL that is able to rezz new objects; and I'm pretty sure that something _can_ count how many items are in inventory and place a safe limit to how many can be created.I understand that your dreadful scenario comes into play when people multiply prims containing scripts over and over again, possibly even with temp-on-rez, and each is creating new notecards like mad while it's still rezzed, thus allowing a denial-of-service situation. But LL has cleverly been dealing with that sort of thing all the time. Suppose, for instance, that
llMakeNotecard
wouldn't work on temp-on-rez objects (or you could just call it _once_ for the rarest cases when it might be useful for some weird reason). Or suppose that there is a memory limit regarding how many notecards can be created per unit of time. Whatever such limits are and how they are implemented is irrelevant; there are so many anyway, exactly to prevent this from happening. Also, a prim "gone rogue" due to a programming error which is creating notecards like crazy can easily be.... deleted. Or... when a certain limit is reached, the object is returned automatically to Lost & Found. There are lots and lots of ways to deal with those scenarios.There is, however,
one
issue which needs addressing, and this will be in a separate comment below :) — the writing function would need to be atomic and not
go through the insanely crazy dataserver event. That, indeed, was what the OpenSimulator developers used to read/write notecards efficiently...V
VriSeriphim Resident
primerib1 Resident wrote "Is that even possible?"
Yes. If nothing else, FireStorm has "lslbridge", which it can use to execute LSL functions in the server, including accessing Linkset Data.
(Linden Labs is aware of the ForeStorm lslbridge)
Coffee Pancake
Could the behavior of llGetNotecardLine just be changed, anyone using the baked in 0.1s per line delay for anything important likely got their tongue stuck to a lamp post over xmas and will never be seen in SL again anyway.
Seems like an easy way to make everything depending on notecards as a read-only data source universally better, over night, for everyone everywhere.
Vincent Nacon
Coffee Pancake: Nope, like with all other old delayed functions, some scripts are written with the expectation of those delays. Removing the delay can break something by going too fast.
Coffee Pancake
Vincent Nacon: I doubt that for a 0.1s delay .. no one is doing important stuff in that tiny window with how LSL works. It would actually be hard to do on purpose.
V
VriSeriphim Resident
Coffee Pancake: An example would be, knowing the script is going to read 10 lines, the author omits an explicit delay of 1s. Get rid of the delay, then that expectation is broken.
Signal Linden
planned
Gwyneth Llewelyn
Signal Linden while giving a "faster" alternative to read notecards is naturally a Good Thing, and should be encouraged by all means, I think you're still short of what would be considered
perfection
.And there is no need to reinvent the wheel. Just see how it's done under OpenSimulator. There is
one
function to retrieve the contents of the whole
notecard — GetNotecard — which returns the entire contents of a notecard as a string (which can then be parsed as usual). And there is one
function to write the whole
contents of a string (or a list) to a new notecard in inventory, MakeNotecard. Both are, from the perspective of the script programmer, atomic
operations (making notecards would not work otherwise). Obviously, "under the hood" a lot
is being done, namely, retrieving the notecard from asset servers if it's not yet on the current region and so forth. But why should the programmer
worry about any of that?Let's admit it — dataserver events were useful to a degree 20 years ago, when everything was slow and costly in terms of resources. Therefore, it made sense to be able to concurrently retrieve the contents of a notecard while the script was busy handling other things. But in 2024, well... notecards are not
that
huge in any case and transmitting them around AWS instances should take less time than copying & pasting the word GetNotecard
. There is little reason for using NAK
or any such mechanism just to make the script programmers' lives even more infernal than what they already are. Simply let them call the function and wait until it returns the string with the notecard's content. They can always put a script timer to wait for some sort of timeout.If you wish to go wilder... implement all the above using the HTTP retrieval mechanism. :-P Consider the following: notecards' content being able to be retrieved via Web (once the proper permissions are checked, of course) given its UUID. Then you can grab them using the existing
llHTTPRequest
functionality — asynchronously, which is even cooler. Assuming that the right capabilities were in place, you could even do that from the 'outside world' as well. And what about MakeNotecard
? Simple — use the same HTTP-based mechanism, just allowing, in this case, to use the PUT
and DELETE
methods; in the return body would be the UUID of the freshly created notecard (or some sort of error if the operation failed due to permissions, unexpected system failures, etc.).You'd also benefit from having the whole throttling mechanisms already figured out for HTTP-in and HTTP-out (and implemented!), so the denial-of-service scenarios alluded above in the comments would not apply. Or, rather, they would be mitigated by the built-in functionality that
already
deals with them on the Web functions.Ok, so you might argue, but retrieving notecards via a Web service is crazy and awkward, hard to track, and an exception to most of the other already-existing content retrieval functions that already exist. Well, I agree. But
now
that you have implemented not one, but two
key-value pair storage mechanisms (one for Experiences, one for Objects...), just let Notecards get retrieved in the same way. I mean, I hardly know how exactly you store Notecards inside your asset servers, but... I'm sure that it would be trivial to place them inside your KVP storage system as well. After all, you just need to retrieve an unstructured object (the notecard's body) with some metadata (permissions) — the rest of the magic for displaying the notecard with links, attached content, etc. happens on the viewer. All you would need to do is to allow the KVP functions to act on Notecard UUIDs inside the object — the script programmer would list inventory to grab the desired UUID and retrieve its contents using the KVP functions (conversely, creating
a notecard would be as easy as submitting a special command as key and the notecard's body as the value — and get the freshly created notecard's UUID or an error code in return).There you go. Next-to-nothing programming in terms of additional functionality, reusing existing code, just making it cleverer. You
might
put a wrapper around it (i.e. llGetNotecard
and llMakeNotecard
) but it would really be just some stub code (a C pre-processor macro style of thing) to the KVP interface. Not even the LSO/Mono compilers would require any significant changes; there would not even be necessary to add extra bytecode operators, nor function numbers, nothing. You already have everything
you need fully implemented and thoroughly tested! It's just a question of cleverly funneling some requests from one place to the other...And if your argument against using this mechanism is "but notecards are currently stored on a RDBMS on the asset servers, so we cannot do this!"... please. I'm sure you can devise a very efficient mechanism to get the KVP calls reach the KVP storage server, which either already has the notecard, or simply fetches it from the RDBMS. In other words — just use the KVP storage facilities (AWS also provides them, right?) as a
caching server
for the asset server. In fact, if you're not already doing that, you're missing some crucial optimisations at some point — since almost everything in SL requires knowing the UUID in advance, and it's hardly possible to retrieve any sort of asset without knowing its UUID beforehand, it logically follows that the best
way to cache assets is to use some sort of KVP storage — possibly distributed and locally available on each region. And working under that assumption, retrieving (or even writing) notecards should not
be harder (or different!) than reading/writing to the already-existing KVP storage facilities available to LSL programmers...— Slightly edited the Markdown, which is a bit odd in some cases
Cheshyr Pontchartrain
Gwyneth Llewelyn Reading an entire notecard at once seems problematic. Notecards have a maximum length of 64Kb, while strings are limited to 32Kb and essentially much smaller unless your script does nothing else.
Gwyneth Llewelyn
Cheshyr Pontchartrain point taken — if you just have 64Kb of memory per script, and strings just able to store 32Kb, but routinely use notecards with 64Kb of data to be processed, well, then obviously my solution won't work.
On the other hand, I'm amazed that there
are
people reading 64Kb-long notecards, line by line, to process data. I can
understand the need of storing a massive amount of data, sure, but then store it externally. As mentioned, even today, unless you hit a limit and get throttled, it's faster to read from an external web server, especially if we are talking about reading line per line for further in-world processing. And if it's stored off-world, well... it can be as big as you want.