Discussion: Testing standards & practices

[Dart Zaidyer]

I've been too busy to finish up a proper submission for this contest, but you might be interested in the standards we came up with at Talking Time for testing and peer review. Many of these are based on experiences of playing the 3rd and 4th Quests, especially from an outsider's perspective. Die-hard ZC authors might scoff at some of this, but we were interested in giving the average player a balanced and polished experience.
Some of these items also address pitfalls the 3rd and 4th Quests fell into that make them feel unpolished or like bad romhacks in some places.

• All four cheat codes should be enabled during initial testing, if only to save time. That way it will be easier to do targeted testing of different situations and play styles for each Dungeon.
• Be ready to swallow your pride and seriously consider any problems players might be having with your quest design. It's possible to be too clever or too skilled for most players to match.
• Except for Boss/Miniboss rooms, the player should not be able to get stuck in a situation where the only escape is death. They could have a Clock at the time, forcing them to use the Continue screen as a last resort.
• It should not be possible for the player to be unable to return to the Dungeon entrance if they've run out of keys or bombs.
• "your life or your money" Guys should always give the player a chance to pay rupees, even if it means leaving the room and coming back later.
• Dungeon difficulty should rely on being challenging but not impossible for a player who has minimal items and equipment at that point in the game. (Meaning they only collected required items and Heart Containers from previous dungeons or shops and haven't gotten any of the optional stuff from the Overworld.) Note that Potions and the Blue Ring can mitigate this somewhat, but they probably shouldn't be considered necessary for a seasoned NES Legend of Zelda player until Level-6.
• Never, ever put Darknuts and Wizzrobes of any kind together in the same room. Even a seasoned NES Legend of Zelda player could (and often will) die over and over again upon meeting this challenge.
• If it's a boss or a miniboss, it should stay dead when you kill it.
• Hidden Dungeon entrances should always have a clue somewhere in the game. The player should not be expected to burn trees or blast rock faces with no indication of where to look.

These guidelines should produce a more fun and playable quest, even for the toughest challenge yet. Please consider them, at least.

[CJC]

• All four cheat codes should be enabled during initial testing, if only to save time. That way it will be easier to do targeted testing of different situations and play styles for each Dungeon.

It might be better if the quest was simply unpassworded. It will be a publicly available quest if it wins anyway, and being unpassworded allows testers to:

• Enable cheats if they are needed
• See dungeon paths and tile warps if they are confused
• Locate secrets by their flags if needed
• Fix game-breaking bugs (and document them) without having to wait for the creator to update the quest

Definitely agree that other quests should have cheats enabled during testing, but for this contest being unpassworded would be much more beneficial.

• Never, ever put Darknuts and Wizzrobes of any kind together in the same room. Even a seasoned NES Legend of Zelda player could (and often will) die over and over again upon meeting this challenge.

Depends how many enemies are in the room, and isn't restricted to just that combination. With battle-incompatible enemies, the room should never have more than four aggressors. One of my least favorite combos ever encountered was Pols Voices and Like Likes.

• If it's a boss or a miniboss, it should stay dead when you kill it.

I have a bug with one of my Digdoggers... it splits into a regular enemy type (A blue floating wizzrobe, the rapid fire type as known in the 1st quest), and even though the Digdogger parent is set not to return, it still does because its children respawn. Do you have any recommendations? Short of making a 'no return blue wizzrobe' copy, I don't know what to do.


Thank you for assembling this list, it's a very good reference.

[zcfan27330]

Do we have to have 4 cheats or can we just have a level 4 cheat?

Do you want us to announce what it is in the quest thread?

[Chris Miller]

I like to think I mostly met those. There are clues for just about everything, even some heart containers. Of course, some things are in the cryptic Engrish original Zelda 1 style in keeping with authenticity. And, there is an L4 cheat code. If anybody wants it, they can ask me and I'll be glad to give it to them. If they really get stuck, I'll be glad to help.

[Imzogelmo]

I think I met all of them but the last one. The original (2nd quest specifically) didn't really tell you where to look for dungeons. I tried to keep with that theme, as searching for stuff aimlessly is part of the authenticity. :) I also agree with Miller, making it sound like a simplified English translation is also part of authenticity.

However, for really obscure secrets or new enemies, I put some hints. That is the sort of thing that really would be found in the game booklet (or other extraneous material) but since we don't really have that, I made the old guys talk about it. If I get a lot of the same question, or something hard to figure out that I just overlooked, I will add some hints to smooth it out.

My quest is currently passworded, but I'll put out one with no password if (and who am I kidding... "when") a new bug or difficulty fix is released. Also I'll give the cheat code if someone cares to need it.

[CJC]

Do we have to have 4 cheats or can we just have a level 4 cheat?

Do you want us to announce what it is in the quest thread?

None of this is official (yet), but the next time you supply an update you might want to add a L4 cheat (that's all you need since it unlocks all the cheat levels) or you might want to remove the password (so that players can add their own cheat if they need it or look at your dungeon layouts if they are stuck). At this point, all of these suggestions are optional.

If you do apply an L4 cheat you don't have to report it in your thread, but be willing to PM it to play-testers who request it.

[Dart Zaidyer]

If you don't mind my saying so, your first statement:

I think I met all of them but the last one. The original (2nd quest specifically) didn't really tell you where to look for dungeons. I tried to keep with that theme, as searching for stuff aimlessly is part of the authenticity. :)

Is at odds with your second:

However, for really obscure secrets or new enemies, I put some hints. That is the sort of thing that really would be found in the game booklet (or other extraneous material) but since we don't really have that, I made the old guys talk about it. If I get a lot of the same question, or something hard to figure out that I just overlooked, I will add some hints to smooth it out.

We preferred to lean towards the second statement. The idea is that if the player has to duck OUT of the game at any point to solve a problem IN the game (usually by saying "screw this, I'm loading a walkthrough"), the game is at fault. Although we wanted to write our clues in mangled English and provide suspicious visual cues to make the player think of the solution on their own, we wanted to at least give them that nudge, something they think they can shoot for. Unclued dungeon entrances were a mistake in the 80s, and they're a mistake now. They're one of the few yet certainly the biggest barrier to finishing the game in a reasonable time frame. And if a player feels they can't do that without a walkthrough, they'll give up on your quest, no matter how faithful and loving an homage it is to the original.

We'll see how this stuff shakes out during testing. Since our guidelines seem agreeable to you at least, I don't expect there to be many problems. But this is certainly one of the bigger pitfalls.

[Nightmare]

I actually think it is a BAD idea to include cheat codes with your quest, especially if it's like a finihsed product. It gives people an easy way out, rather than enjoying your quest and testing to see how it really plays. I personally take all cheat codes out of my quests as soon as I humanly can, as I have been embarassed by people finding them randomly. If they can't play it normally then they have no idea of how it plays to any type of player.

-James

[Imzogelmo]

I see what you're saying there, but I guess I distinguish between 2 kinds of hints:

1. Strongly hinting (or outright telling) the player where their next objective is, and
2. Telling them some enemy weakness or strategy that may not be important right away, but will ultimately be important.


As a player, I find the first kind rather condescending and it takes away from my enjoyment. If that's the kind of game I wanted, I would make it a straight line, no branches, so that you have no choice but to go forward. My quest is rather non-linear; you can skip dungeons if you are so inclined (not in all cases, but for a great many it is possible). A good exploration game should not lead you, it should get out of your way. That being said, if "search the whole overworld" is too vague, I may add in some guys to give hints (but not explict directions) to some of the dungeons, as I'm open to change if something is too well hidden. Of course finding those guys is just moving the problem from one spot to locate to 2...

The second kind, I am completely in favor of. If some new gameplay element could not be known from experience (as this is a 5th quest, knowledge of the previous can be assumed at least minimally), then it should be given, even if cryptically. You have no way of knowing that Pols Voice are weak to arrows until some guy tells you. If they were a newly introduced enemy, that would be a potentially crucial thing to know. With that in mind, I included hints for the bosses, even though they are not new to the series, as well as some other new enemies that are mixed in with my quest (as that is a mandatory inclusion).

The two kinds of hints, to me, are as different as telling a child how to work a math problem and telling them the answers. One retains the interest and ultimately helps them (albeit with no immediate gratification), the other gets it over with quickly, sure, but guarantees that they will run into the same issue again later. The quest, in its early phase, has to teach them how to play, so that later on they will not get stuck. If they always have a post-it note about what to do, they never learn the lesson of "navigate thyself."

[Dart Zaidyer]

I see what you're saying there, but I guess I distinguish between 2 kinds of hints:

1. Strongly hinting (or outright telling) the player where their next objective is, and
2. Telling them some enemy weakness or strategy that may not be important right away, but will ultimately be important.


As a player, I find the first kind rather condescending and it takes away from my enjoyment. If that's the kind of game I wanted, I would make it a straight line, no branches, so that you have no choice but to go forward. My quest is rather non-linear; you can skip dungeons if you are so inclined (not in all cases, but for a great many it is possible). A good exploration game should not lead you, it should get out of your way. That being said, if "search the whole overworld" is too vague, I may add in some guys to give hints (but not explict directions) to some of the dungeons, as I'm open to change if something is too well hidden. Of course finding those guys is just moving the problem from one spot to locate to 2...

The second kind, I am completely in favor of. If some new gameplay element could not be known from experience (as this is a 5th quest, knowledge of the previous can be assumed at least minimally), then it should be given, even if cryptically. You have no way of knowing that Pols Voice are weak to arrows until some guy tells you. If they were a newly introduced enemy, that would be a potentially crucial thing to know. With that in mind, I included hints for the bosses, even though they are not new to the series, as well as some other new enemies that are mixed in with my quest (as that is a mandatory inclusion).

The two kinds of hints, to me, are as different as telling a child how to work a math problem and telling them the answers. One retains the interest and ultimately helps them (albeit with no immediate gratification), the other gets it over with quickly, sure, but guarantees that they will run into the same issue again later. The quest, in its early phase, has to teach them how to play, so that later on they will not get stuck. If they always have a post-it note about what to do, they never learn the lesson of "navigate thyself."

Yeah, you've got it right! This is what we were going for at Talking Time.

Oh and Nightmare, having cheats/no password is strictly meant for testing. You can find bugs and problems that way which could have taken hours to discover otherwise. The final, finished quest shouldn't have cheats enabled.