[ZC 2.54 ] setting Link->Action to LA_DROWNING makes link turn invisable

[DarkDragon]

@ZoriaRPG Errors in the documentation should be corrected. If you have fixed versions of the .txts, you should submit a pull request.

As for writing to Action specifically, exposing this as a raw variable was a huge mistake, and as a writable variable, even more so. Many of Link's actions simply cannot ever be made safe to set or cancel at arbitrary times, without rewriting the Link code completely (at which point, the existing Actions may be obsolete anyway).

What will actually happen is that future code will detect when a script tries to read or write to Action, and use an emulation layer to preserve existing behavior. This is a heap of work and an instant migraine even under the best of circumstances, which is why it is critical that the current version is extremely specific about what behavior is supported, and prevents quest authors from accidentally (or intentionally) using features that could be removed or changed at any time, due to being unsupported but "just happening to work" in the current version.

This DROWNING issue perfect illustrates the problem.

If quest authors have been ignoring the documentation and using unsupported features, with a great big sigh, I will have to somehow ensure that the current (undocumented) behavior is preserved. But I'm extremely keen to firewall the damage, as thoroughly and as quickly as possible. If you have a list of Actions that people are currently trying to write to, that will help a lot.

[ZoriaRPG]

@ZoriaRPG Errors in the documentation should be corrected. If you have fixed versions of the .txts, you should submit a pull request.

As for writing to Action specifically, exposing this as a raw variable was a huge mistake, and as a writable variable, even more so. Many of Link's actions simply cannot ever be made safe to set or cancel at arbitrary times, without rewriting the Link code completely (at which point, the existing Actions may be obsolete anyway).

What will actually happen is that future code will detect when a script tries to read or write to Action, and use an emulation layer to preserve existing behavior. This is a heap of work and an instant migraine even under the best of circumstances, which is why it is critical that the current version is extremely specific about what behavior is supported, and prevents quest authors from accidentally (or intentionally) using features that could be removed or changed at any time, due to being unsupported but "just happening to work" in the current version.

This DROWNING issue perfect illustrates the problem.

If quest authors have been ignoring the documentation and using unsupported features, with a great big sigh, I will have to somehow ensure that the current (undocumented) behavior is preserved. But I'm extremely keen to firewall the damage, as thoroughly and as quickly as possible. If you have a list of Actions that people are currently trying to write to, that will help a lot.

Err... Developers have used these features, too, I believe. :/

A bigger problem, is that people left so much unfinished in the first place. In reality, at least 30% of the script engine syntax was left unresolved, many planned features were partially implemented, and no-one was interested in completing the task after L and jman left. That's the job that I signed up to do, with 2.future; where the philosophy from the onset was 'Complete the original ZScript plans'. I've been steadily working on that with rather good results, and what I'm doing may be a stopgap measure between q2.5 and 3.x, but that is all it is intended to be. People keep taking about a rewrite, which is honestly where this kind of emulation belongs.

Look, if you want my advice, instead of adding in countless layers of absurd abstraction, consider migrating behaviours to the script engine, and have the game engine read and act from those. That way, the script engine and game engine can always do the same things, and you don't need insane emulation layers. I do not frankly see a problem with setting class variables. I do have issues with the engine writing to raw variables, instead of using some kind of accessors. That alone has made integrating Link hitbox manipulation in the script engine many times harder than it need be.

In my eyes, the script engine should be able to do anything that the game engine can do natively. Otherwise, it;s being artificially crippled, and adding in multiple layers of abstraction in something like this kills performance. I would see this as valuable, for a v3 or something, if you want to change out more than half of the core, but for the present, we need to keep this thing alive. Worrying about these class variables if getting too far ahead, especially given that every class acts this way.

Are you going to do the same thing with npc vars, item vars, and the like; because users do the same things with those. You pretty much want to write an abstraction layer for 50% of the game engine and the script engine. that's just not the right way to go about it. If we slowly migrate internal stuff over to the script engine, it will be much easier to control in the future without this sort of thing.

Then again, I learnt my computer science back in the late 1970s through early 1990s. We had very different views then, about how to handle code optimisation: Hit the hardware, and use it for as much as possible. I can't even mentally encompass the kind of change that you suggest here, and killing any existing command is just a bad idea. Writing emulation for them, we can;t do without many more participants.

Nothing stops us from doing that in the future, when more people are involved;l which isn't going to happen if we take an eternity just to pick a direction. My bottom line, leave these things be, for the present, worry about updating the libs, making a stable release, making at least some of the userbase happy, so that in a year,or two, there are still people using the programme. If you ultimately want to emulate ZC's core features, we'd be better off taking Solarus, and making a ZC emulation layer on top of it.

For now, I'm happy to fix these things, so that they do work as intended, but I'm not going to go disabling things because someone might want to change or replace them in the future. Disabling the irrational ones is fine, unless we find a way to implement them that makes sense, but the user does need a way to set Link;s actions, animations, sprites, and such, that isn't just a complete hack, which for Link's graphics, is precisely what users have had to do. These are precisely the kinds of impediments that has caused so many users to shift to other software, and as a result, why the userbase today can be counted on me fingers.

As to what features scripters use, if it is in ZScript.txt, you can wager money that someone is using it. Most of the stuff in there does work, save a few things that say they do, and do not; which hasn't stopped users from utilising them for self-defined behaviour. That aspect, can't be helped, and the only viable solution is to read the quest version with some new features, and do not execute any new code tied to them if the version is < current.

My solution for Link->Action is to define the actions. If this causes any quest compatibility concerns, add a routine to prevent he newly defined behaviour from exec'ing. In addition to the list above, LA_SPINNING, and LA_CHARGING have been used. I can;t say with authority what others have been used, as scripts are not openly published, and the vast majority of the userbase that created them is now gone, moving to Gamemaker, or RPG Maker, or something.

[DarkDragon]

I've explained this several times before and won't rehash the argument here in full.

Giving the script engine raw access to implementation details about how ZC is implemented deep under the hood calcifies poor decisions, bugs, and unmaintainanle spaghetti code that infests the core of the ZC logic.

The hardest, and most underrated, part of being a good developer of a long term project is knowing when to listen to what users are asking for, and when to realize that they are succumbing to short-term and ill-informed thinking, saying "no," and standing your ground. Giving people access to engine internals satisfies the users screaming "gimme gimme gimme now now now," at the price of signing ZC's death warrant.

I refuse to make that bargain.

What you say about how to implement actions is absolutely the right way to go, though. Instead of mucking directly with the Link class internals, scripts should have access to a restricted set of high-level actions, which are then passed on to the game engine, which tries to act on the script command as best as possible. So if a script tries to tell Link to charge, while he's already riding a raft, the game engine does something reasonable (like ignoring the charge command) rather than entering an unpredictable, inconsistent and corrupt state, which is what currently happens when scripts blindly trample willy nilly over internal variables.

[ZoriaRPG]

I've explained this several times before and won't rehash the argument here in full.

Giving the script engine raw access to implementation details about how ZC is implemented deep under the hood calcifies poor decisions, bugs, and unmaintainanle spaghetti code that infests the core of the ZC logic.

The hardest, and most underrated, part of being a good developer of a long term project is knowing when to listen to what users are asking for, and when to realize that they are succumbing to short-term and ill-informed thinking, saying "no," and standing your ground. Giving people access to engine internals satisfies the users screaming "gimme gimme gimme now now now," at the price of signing ZC's death warrant.

I refuse to make that bargain.

What you say about how to implement actions is absolutely the right way to go. Instead of mucking directly with the Link class internals, scripts should have access to a restricted set of high-level actions, which are then passed on to the game engine, which tries to act on the script command as best as possible. So if a script tries to tell Link to charge, while he's already riding a raft, the game engine does something reasonable (like ignoring the charge command) rather than entering an unpredictable, inconsistent and corrupt state, which is what currently happens when scripts blindly trample willy nilly over internal variables.

Right, but what I am also saying, is that the engine shouldn't be directly mucking with them either. I would rather that the engine point to the script engine, and let the script engine have priority when deciding how to handle them. This way, the script engine and the game engine do not diverge. Where the values are stored becomes immaterial, as you can always relocate them.

This is also why I'm not rushing to make pull requests, or to try to force anything into main. There exists at present no implementation that works this way, short of the ffrules thing that I've been working on. I tend to see this entire thing as a slow migration though. If you want to move things into a form of abstraction, you need to do it a bit at a time, unless you have sixty hours a week to spend rewriting the engine.

I do not see anything that prevents reimplementing these things in this manner, and still being safe. Keeping the variables in the classes preserved compatibility, as well. Where would you store the values? Do you want duplicate values in the script engine, that are setting the class values? What is the ultimate advantage over doing it at a function level? I've done some of that, and frankly, I would prefer the variables in the script engine, but I don't have the time to completely rewrite ZC, which is what this amounts to needing as far as I can see.

The other concern, is always, how much compatibility are you willing to lose when doing this? The answer thus far, for most of us, has been none. That means that everything need be reimplemented, and as long as you are going to do that, you may as well finish the stuff that was left out, in the process.

One thing I can say, is that if you slave the script engine to the game engine, you preclude being able to do things later down the road, that you might want to do. Making the game engine call tot he script engine, gives you far more options. It also maintains uniformity between the two. The notion that a scripted behaviour does A, but the internal behaviour does B, is a huge fault in the engine, as it stands.

[DarkDragon]

Right, but what I am also saying, is that the engine shouldn't be directly mucking with them either. I would rather that the engine point to the script engine, and let the script engine have priority when deciding how to handle them. This way, the script engine and the game engine do not diverge. Where the values are stored becomes immaterial, as you can always relocate them.

Yes, this is one way to go. It's not a bad way. Although I'm not aware of the full history, I well imagine that the push to rewrite game internals in AngelScript was one foray in this direction.

If we were starting from scratch, I'd be in favor of the approach you suggest. But if you want a fully-scripted engine, those options exist, and I don't see much point in reinventing the wheel with a Zelda sticker slapped on top (especially since removing copyrighted assets from the core engine is one of our near-term goals!) The unique strength of ZC is its large database of quests that can be played and quest assets that can be reused in the future. Preserving this advantage, while also slowly improving the quality of the ZC code and the power of its features, is a very delicate operation. I don't think ripping out the guts and trying to replace them with something purely scripted is likely to work out.

One thing I can say, is that if you slave the script engine to the game engine, you preclude being able to do things later down the road, that you might want to do. Making the game engine call tot he script engine, gives you far more options. It also maintains uniformity between the two. The notion that a scripted behaviour does A, but the internal behaviour does B, is a huge fault in the engine, as it stands.

Again, I disagree.

The responsibility of the game engine is to maintain a consistent game state. This (heavy!) responsibility shouldn't get foisted onto the scripters or quest authors. They should be able to write code that tells Link to attack, say, without the fear that if they send the command at just the wrong frame, when Link is already doing just the wrong existing action, he gets stuck with infinite invincibility, or starts moving around wildly, or crashes ZC, etc.

That means there must necessarily exist some separation between the interface exposed to scripts, and the internals of how the game actually operates. A blood-brain barrier, if you will. I don't know if you played Megazeux back in the day, but that to me is an example of scripting done right, and an ideal for ZC to try to aspire to. MZX games were extremely scriptable using a versatile language, but the set of commands was restricted and the game environment sandboxed.

[DarkDragon]

I do not see anything that prevents reimplementing these things in this manner, and still being safe. Keeping the variables in the classes preserved compatibility, as well. Where would you store the values? Do you want duplicate values in the script engine, that are setting the class values? What is the ultimate advantage over doing it at a function level?

It's not a matter of variables vs functions; that's surface-level syntax and not very important in the grand scheme of things.

The issue here is internal implementation vs. externally-visible interface. If you look at LinkClass::setAction(), you can see there is already a bunch of code there that tries to prevent scripts from issuing actions that break the consistency of the game state. That's the way that actions (and anything else scriptable) should be handled, not by tweaking the internal state and damn the consequences.

I don't know if you're a computer science student, but a key concept in computer science is that of invariants: relations that you can prove must be true about an algorithm as it's running, and that you can use to reason that an algorithm is correct. For example, in a for loop, you know that the index variable must be between the start and end of the for loop range, because if it weren't, the loop would have terminated on the previous iteration. This lets you write code like

for(int i=0; i<size; i++)
  array[i] = 0;

without worrying that you might segfault due to writing past the end of the array.

If some external agent is allowed to muck with the internals of your code at any time (for example, by setting a breakpoint, and writing new values to memory) it is extremely easy to break your code, because you can no longer trust your invariants. I can set i to be size+1 at some point in the middle of executing the loop, and bam, your program crashes. To prevent this, any changes that you make to the program's internal state need to be aware of, and go out of their way to preserve, the invariants. Instead of letting the user write random integer values directly into the memory address of i, you provide a callback which reads the user's new requested value, and sanitizes it, before continuing with the loop.

It is the same with the ZC engine, except much more complicated, because the invariants are unclear and undocumented, and require great effort to tease out in a lot of cases.

[ZoriaRPG]

Yes, this is one way to go. It's not a bad way. Although I'm not aware of the full history, I well imagine that the push to rewrite game internals in AngelScript was one foray in this direction.

If we were starting from scratch, I'd be in favor of the approach you suggest. But if you want a fully-scripted engine, those options exist, and I don't see much point in reinventing the wheel with a Zelda sticker slapped on top (especially since removing copyrighted assets from the core engine is one of our near-term goals!) The unique strength of ZC is its large database of quests that can be played and quest assets that can be reused in the future. Preserving this advantage, while also slowly improving the quality of the ZC code and the power of its features, is a very delicate operation. I don't think ripping out the guts and trying to replace them with something purely scripted is likely to work out.



Again, I disagree.

The responsibility of the game engine is to maintain a consistent game state. This (heavy!) responsibility shouldn't get foisted onto the scripters or quest authors. They should be able to write code that tells Link to attack, say, without the fear that if they send the command at just the wrong frame, when Link is already doing just the wrong existing action, he gets stuck with infinite invincibility, or starts moving around wildly, or crashes ZC, etc.

That means there must necessarily exist some separation between the interface exposed to scripts, and the internals of how the game actually operates. A blood-brain barrier, if you will. I don't know if you played Megazeux back in the day, but that to me is an example of scripting done right, and an ideal for ZC to try to aspire to. MZX games were extremely scriptable using a versatile language, but the set of commands was restricted and the game environment sandboxed.

I suppose it depends on if Gleeok, and Saffith still plan to implement AngelScript in the future. Hell, Grayswandir is doing things with the parser, specifically to support easier ZScript to AS conversion later down the road, and this is why I advised running the game engine off the script engine, as if you are going to eventually replace the script engine with AS, and run almost all the stuff off of scripts, which is how most modern game engines work, them you may as well start working on making the engine calls portable.

I'm just not all that interested in writing either side of that at present.

I also wonder just how much of the extant ZC library is in vogue, at all. Pretty much everything that gets attention at all these days, is a script-heavy quest. That means that there si a significant demand place don the script engine, and that if it was at the forefront, you might have a more popular product. I honestly do not know. What is the reason that RPG Maker, and Game Maker are such popular choices?

Any time that someone mentions them, they praise Lua (I do not care for Lua), and how flexible everything is; how the engine is linked to the script engine, and so forth. Solarus is near entirely tied to the script engine. Even graphics are effectively set by script. Most of these things today run in real time, without compiling. It;s actually rather fascinating, and also, requires far more processing power, and RAM, than i happen to like. That;s the problem with abstraction layers, and JIT compilers.

I also wonder how many ZC users have anything approaching modern hardware, given that most of the questmakers are using systems that are seven to ten years old.

Anyway, I think it;s pretty much your show at this point for core ZC, and you will do what you want. We're still going to produce the 2.future fork, and we'll see what plan evolves. If you create some abstraction layer that works, and does most of what we do in 2.future, we can make a port-over utility. I certainly have no problem contributing to the core, but I effectively know that you are going to take a very small number of additions, and you will not like doing it. Look at itemdata. All of those are tied to class variables. If you don't want to include anhything tied to class variables, that is new, then I hinestly have no idea what you would include from the 2.future basecode.

That's why for the core, I'd rather work on the libs and such, because it just sounds as if you do not want any of our stuff, and neither Gleeok, nor Saffith are particularly active in what you are doing, and despite the stuff you made to date being forks, no-one is going to deny you the lead position in this; I certainly won't. That said, dissuading us from making our own thing here isn't happening either.

If there is anything specific that you have seen in the 2.future base that you think would be welcome, I think it'd be better to work with it in that regard. Clearly, you won't be happy with the majority of our work product, plans, or motivations.

It's not a matter of variables vs functions; that's surface-level syntax and not very important in the grand scheme of things.

The issue here is internal implementation vs. externally-visible interface. If you look at LinkClass::setAction(), you can see there is already a bunch of code there that tries to prevent scripts from issuing actions that break the consistency of the game state. That's the way that actions (and anything else scriptable) should be handled, not by tweaking the internal state and damn the consequences.

I don't know if you're a computer science student, but a key concept in computer science is that of invariants: relations that you can prove must be true about an algorithm as it's running, and that you can use to reason that an algorithm is correct. For example, in a for loop, you know that the index variable must be between the start and end of the for loop range, because if it weren't, the loop would have terminated on the previous iteration. This lets you write code like

for(int i=0; i<size; i++)
  array[i] = 0;

without worrying that you might segfault due to writing past the end of the array.

If some external agent is allowed to muck with the internals of your code at any time (for example, by setting a breakpoint, and writing new values to memory) it is extremely easy to break your code, because you can no longer trust your invariants. I can set i to be size+1 at some point in the middle of executing the loop, and bam, your program crashes. To prevent this, any changes that you make to the program's internal state need to be aware of, and go out of their way to preserve, the invariants. Instead of letting the user write random integer values directly into the memory address of i, you provide a callback which reads the user's new requested value, and sanitizes it, before continuing with the loop.

It is the same with the ZC engine, except much more complicated, because the invariants are unclear and undocumented, and require great effort to tease out in a lot of cases.

At what point did I advise directly writing to the class variables with absolutely no handling? There's a clear difference between sane handling and interpretation, and writing emulation, and onion layers of obfustication.

Computer science student... I've been a student of all arts, and sciences, throughout the nearly sixty years of my life.

I should note that when I went in to try and define LA_CASTING, I specifically used the setAction() function as an entry point. I have slowly been plotting out moving all the direct variable manipulation out of ZScript, so that handlers on the class side interface with it. (I prefer to keep them with the class, when that is rational, but there are things that I would like to move into the script engine.)

My present project involves finding all the engine points that read Link.x, LinkX(), and Link.getX(), and so forth, and setting up Link.AbstractedX() so that hitbox modification isn't as bloody wretched. That is simple separation-based abstraction, and if that is what you want, that's agreeable. If you want ten layers of separation, I'm just not interested in doing it. I can't even place odds on being alive by the time that is done, at the present rate of progress.

There are also some facets where abstraction other than by passing values through a function, seems either superfluous, or for which I see no method.

How you you set invincibility frames, that are normally handled by a class variable? Note that I vbounded it between the legal values, but beyond that, there aren;t any system definitions where it would be illegal, or unsafe. The engine is extremely open-ended about most of those things, from my experience.

[DarkDragon]

That said, dissuading us from making our own thing here isn't happening either.

I do think your work will have more impact if it is incorporated into the main ZC release, but of course, ZC is open source and you are welcome to do with it whatever you wish (that is compatible with the ZC license). If I ever suggested otherwise, I was wrong.

If there is anything specific that you have seen in the 2.future base that you think would be welcome, I think it'd be better to work with it in that regard.

Sure. I believe in the 2.future ZScript thread I highlighted quite a few additions I would love to see submitted?

[Gleeok]

There are certain problems with working on old, large software such as ZC. One major problem I have is the amount of state under the hood is incomprehensible (in a practical sense) when you realize it has to support every quirk in every quest ever made. In the past if I ever had to look into bugs or features that change even one single bit of state, I have to, at minimum, find every single place in the code where this could directly or indirectly effect logic, then, if so, repeat the same process for each new thing that pops up. If this process also includes realizing that every script also has complete control of this then that's all folks; it's either rewrite the whole subsystem in question to an overly-complicated-don't-break-compatability specification, or just give up altogether.

A few years ago I suggested using a new scripting language. This would allow new systems to be created that can simply piggy-back off the old ones but also be tightly coupled with scripts that can actually rewrite built-in behaviors without having major issues or losing compatability (or creating bugs with existing quests). The new npc scripts and supporting of various global callbacks I envisioned would be a good test of that experiment before doing anything else with that. ...It's not been worked on really, but it should be. .. :/

I think I'm going to make putting in the compiler/script processor/script engine my next priority besides the few other things I have here. Even if it's not used immediately, if I can get a ZScript->angelscript conversion utility working you might take it more seriously.

[DarkDragon]

Was that the main reason you took back out the AngelScript stuff (concern about porting existing ZScripts)?