A New Beta Philosophy?

[jman2050]

Right now, we tend to release betas sort of as milestones. Different representations of the developmental progress of the program. How about we try something different.

Instead of releasing milestone betas like we have been with tons of new features, why don't we release betas incrementally? Heck, I'll go as far as releasing builds directly derived from our SVN builds. Each new build will be released in short periods of time, instead of having to wait long periods between betas. Furthermore, it'll allow everyone to quickly verify the destruction of quaranteed bugs, and to catch new ones when they occur to avoid them being too tightly integrated into the program.

What do you all think of this? Should we release betas in this fashion from now on? It'll make quest building more difficult, and intensifies the risk of quest or save file corruption, but I think it'll result in much faster progress.

[blue_knight]

I'd recommend having major builds, which are deemed to be pretty safe and compatible with previous major builds. Then you have incremental builds which are basically the completely unsafe builds. So you still have your milestones which are important for organizing progress and having goals to meet but then you have the incremental builds for pure beta-testing and "feature preview". Example major builds - 16c0, 16d0; Incremental builds between would then be 16c1, 16c2, 16c3, etc. Does this make sense to you?

[Petoe]

I'd recommend having major builds, which are deemed to be pretty safe and compatible with previous major builds. Then you have incremental builds which are basically the completely unsafe builds. So you still have your milestones which are important for organizing progress and having goals to meet but then you have the incremental builds for pure beta-testing and "feature preview". Example major builds - 16c0, 16d0; Incremental builds between would then be 16c1, 16c2, 16c3, etc. Does this make sense to you?

Agreed.

[jman2050]

It would mostly be semantics at this point. The issue isn't with what they're called, but how often they're released ;)

[blue_knight]

The difference would be having major builds where extra effort was taken to ensure stability, compatibility and completeness (relative to the features in that release and not 2.5 of course). On the other hand incremental builds can be "experimental" and may break saves or compatibility or maybe be less tested for stability. If you think of a game company, its the difference between an external milestone (that a publisher would see) and an internal milestone - most likely a buggy development snapshot but it still must be internally tested and new and/or experimental or partially complete features are introduced and tested. Do you know what I mean?

[Nicholas Steel]

if they release only relatively safe builds then there may be bugs that are very hard to fix due to them being buried and/or tied to multiple parts in the code... thus early detection is needed... so i say release them more frequently.

[Freedom]

History as I see it. ;)
The b10b was a pretty solid build
b11 not as good
b12 A, B, C, and D were all trash and as a result I closed ZC and didn't touch it for two months
After that Christmas was coming up, so I thought a stable release was planned, so when I started again it was with the b15, and again the build wasn't as good as 10b, it had a lot more stuff, sure, but almost none of it seemed to work right.

b16 even worse

b16b slightly better but still a lot of bugs with these "new" features, and subscreen crashes from hell.

b16c, can't even walk one screen in the player without crashing and in that short distance noticed Links walk is all screwed up as well.

So.... I guess it doesn't matter if we get 1 beta a month or 10 since the likelyhood of one ever being stable seems pretty remote at this time.

I've decided tonight on a LONG vacation from Zelda Classic, because to be honest all it ever does is piss me off anymore.
Every time a new beta comes out I have to go back and fix dozens of things in my quest that I thought I was done with, and I'm just spinning my wheels.

I'm turning agnostic.... SHOW me a solid release, and then I'll start using it again. ;)

It's been fun, and thanks for your efforts guys, but it's become nothing but frustating to me.

[blue_knight]

Franpa, maybe I wasn't clear enough but I was actually agreeing with having builds more often, in the form of "incremental" builds. What I was saying is that there would still be relatively safe, stable and complete builds at regular intervals. Think of it as "big milestones", kind of like what we have now and then "mini-milestones" in-between. Like some software companies release "daily builds" internally so content creators can start using testing/learning new features and get quick-fixes but still have "milestones" in which some portion of the program is complete, stable and bug-free as possible. I believe that you still need big milestones to not only mark progress but to have specific overall program goals. Is this more clear?

[Nicholas Steel]

yep.

[DarkFlameWolf]

major stable beta milestones with smaller unstable releases inbetween. Sounds good to me.

[Revfan9]

Sounds good to me as well.

[ShadowTiger]

This means that fewer people can actually develop quests in these due to their intentions of being tested, rather than used as building tools.

This is both good and bad.

That means that people can no longer complain that you can't build a quest in it. Nobody was ever really SUPPOSED to be building a full fledged official quest in them, but they did it anyway, and complained all to high hell over and over again because of it. However, on the other side of the coin, you've got people who were actually able to report bug after bug because they WERE trying to build a major quest in them, and in doing so, were actually able to find all these bugs through using the program the way it was intended.

I'm starting to think we should really get moving on making that official, ever-changing "test quest" that tests out literally 110% of ZC's features, one after another after another after another. No plot, no combat, ... just pitting feature after feature to the player and seeing what happens. Test out combinations of items onto combos, combos onto combos, and items onto flags, and flags with other flags. Everything we can think of.


At the moment, I'm not really sure what to think. I'm still not really sure what you meant by "Incremental," really. The imagery comes to mind of a building not with a shoddy fourth wall, but with no fourth wall at all.

[Freedom]

Nobody was ever really SUPPOSED to be building a full fledged official quest in them, but they did it anyway, and complained all to high hell over and over again because of it.

uh..... is there another program to use?
everything up to 2.10 was a beta, and everything after, and since 2.10 itself has crippling bugs then that leaves 190 huh?

What good is a mousetrap that can't be used to catch mice?

[The_Amaster]

I agree with the increment/milestone idea. And ShadowTiger brings up some good points. I know that my building of my quest has found me a couple bugs.
Freedom, the best way to build a quest now is in the mostly stable 2.10, or else, if you really want, the totally stable 1.90. Mind you, I'm still making my quest in a beta, but all I do is map-work and a few other things, which is enough to occupy me until 2.5(My quest is huge). I leave things that are likely to be buggy(such as cut-scenes) to 2.5

[*b*]

I really don't care either way, but as long as it gets out a stable version quicker, I'm for it. It might be nice to get to see some of the new features you tease us about (jman) that much quicker

[beefster09]

I like having betas often, but I don't like main betas coming out like hotcakes so that you have to change all of your settings often to accommodate. A new main beta every 3 weeks is good, and I've associated that betas with even numbers are more stable than odd numbered ones. So odds should be the "experimental betas" and evens should be stable betas.

[Shoelace]

I say make beta 17 or 16d as stable as you can to make everyone happy for the time being. Then do the smaller unstable releases in between. That way, people that want to continue working on their quests can do so on the beta 17 or beta 16d. And the unstable ones sounds good because I will do my part to crack down on the bugs. This sounds good, because a lot of people want to see a stable release and with these more releases, they would find bugs quicker. I just hope that the beta bug forum can get up to date with the bugs that are fixed and not fixed.

[Radien ZC]

When beta 16 first came out, I looked around and couldn't see the forest for all the trees. There were so many bugs staring me in the face that I really wondered why you needed our help "refining" it, since it most of the new features didn't work. It's pretty hard figuring out some features without a help file when we didn't help program them, so it's 10 times more frustrating when the features are unpredictably buggy to boot.

I know that beta testing means unstable versions, but I'm afraid that if they become even more unstable, casual beta testers aren't going to want to bother with anything until a stable release.

ShadowTiger made some really good points. You may lose a few useful testers if you make the beta versions even less capable of working with actual quest material. Like it or not, many of the people who work with quests and scripts in between stable releases actually take a risk in order to do creative projects, which are a more accurate rendition of what ZC should ultimately be capable of. I daresay we might not have an amazing script that plays Pong in ZC if it had become impossible in a beta released later that week.

[koopa]

I'd suggest we give out incremental betas regularly (once a week or more) so those interested in bugtesting can do it to full effect, and report new bugs/fixed bugs as fast as possible. Then we could give out more stable betas, say, once a month for those interested in playing/questmaking.

This is of course all a bit academic until we actually have a stable beta to work from, but after that I'd be all for incremental betas.

[Limzo]

I don't agree with this. I'm treading in shallow water as it stands by downloading new versions of ZC every month, but every week and my parents would hate me as we'd get a virus or something (we have a bad history of downloading things)

So please for the love of keeping Limzo with you (I know you all love me on the inside *gets shot* *gets shot* *gets shot*) do not do this!

[Shoelace]

Limzo did you read what they said? They said that they are releasing minor releases so people that want to can test out the bugs and then they will release a beta. The betas are supposed to be the ones that you would want to download. So just don't download these ones.

As they said, they don't want anyone making games on these minor betas as it is only to insure quicker bug findings and stabler betas.

[Limzo]

Oh. D'oh!
:poke: I'm the one on the right, Shoelace is on the left, BTW.

Thanks for pointing that out to my Shoelace.

[Freedom]

Limzo,
It isn't the number of times you download that puts you at risk of a virus, it's WHAT you download. ;)

[Limzo]

Well yeah, I knew that, but it's just my family are all overprotective about computer viruses. I meant like a "family virus" and I wouldn't be allowed ZC anymore

[Revfan9]

Limzo: I know how that feels... I love having my own laptop to use. Whenever I try to use my family computer, it has so many viruses that you can hardly do anything without it crashing.
Here's an idea: Teach them how to surf the internet properly.

[Limzo]

Here's an idea: Teach them how to surf the internet properly.
Lolz! Yeah, my mum freaked out when I started using Firefox, and it took me forever to pursuade them to let me use Beta versions of ZC in the first place.

[Lotus_Eater]

I think the devs should do what they think is right, if you are afraid of a new version being unstable, don't download it and your problem is solved.

jman, do as you will, I think your judgment is better than most, you know exactly what you are working with, we don't. If it helps you make a better program than you should do it.

[Freedom]

Limzo,
We have a thrift shop here in town that is full of old computers for dirt cheap.
You might look around in your town.
They are all older slower computers, but all seem to me running 98 and xp
usually they are 64 mb ram, which will run ZC and the internet just fine.
They sell for $10 to $60 bucks, the $60 ones are the occasional HP's with Pentium 4's that go through, which last about 5 minutes on their shelves.

You should look for a thrift shop in your town, then mom and dad can't say too much about clogging theirs up.

[Radien ZC]

I'd suggest we give out incremental betas regularly (once a week or more) so those interested in bugtesting can do it to full effect, and report new bugs/fixed bugs as fast as possible. Then we could give out more stable betas, say, once a month for those interested in playing/questmaking.

This is of course all a bit academic until we actually have a stable beta to work from, but after that I'd be all for incremental betas.
I like Koopa's suggestion. It's sort of a compromise. Keep the semi-stable releases coming, but separate them from the "sold as-is" weekly betas.


Lolz! Yeah, my mum freaked out when I started using Firefox, and it took me forever to pursuade them to let me use Beta versions of ZC in the first place.
Yeah, my parents were exactly like that when we got our first internet-capable computer, and that was 10+ years ago. http://www.purezc.com/forums/style_emoticons/default/icon_rolleyes.gif Maybe in 20 years there will be hardly any parents left who didn't grow up with a computer accessible to them, one way or another, but for now we'll have to be patient with the adults who haven't had a chance to learn.

[cbailey78]

Right now, we tend to release betas sort of as milestones. Different representations of the developmental progress of the program. How about we try something different.

Instead of releasing milestone betas like we have been with tons of new features, why don't we release betas incrementally? Heck, I'll go as far as releasing builds directly derived from our SVN builds. Each new build will be released in short periods of time, instead of having to wait long periods between betas. Furthermore, it'll allow everyone to quickly verify the destruction of quaranteed bugs, and to catch new ones when they occur to avoid them being too tightly integrated into the program.

What do you all think of this? Should we release betas in this fashion from now on? It'll make quest building more difficult, and intensifies the risk of quest or save file corruption, but I think it'll result in much faster progress.
That's a GREAT idea, Jman!