David Leary, one of the designers at Robot Entertainment looks back at his work on Halo Wars, and specifically the playtesting that went in to it:
One of the great Ensemble traditions that we’ve carried forward to Robot is universal company playtesting. We try to get our games up and running as soon as possible, get an early version into playtest, and iterate based on feedback. Everyone is expected to help test and offer constructive criticism, find bugs, and be familiar with the high-level concepts of the game.
I was a late addition to the Halo Wars team, joining up for the last six months. Halo Wars had been a long project, and needed an “all hands on deck” approach to get the game to the level of quality we wanted before it went out the door. As a designer, I wasn’t as important an addition as the extra programmers and artists that were rolling onto the game, so my job was to pick up secondary tasks from other designers who were overloaded. I ended up with a grab-bag of responsibilities, such as writing a bunch of the battle chatter for the Covenant units and organizing the game’s many text assets for voice recording sessions.
The 15th Floor Playtest Lab.
In addition, along with Kevin Holme and a few other folks, I ended up running a bunch of playtest sessions. This was something I’d done a lot of back on Age 3 and Warchiefs, though I was out of practice. Justin “Bear” Rouse had been handling most of the playtest sessions for Halo Wars, but he was busy fixing bugs on the multiplayer maps. The end of the project required us to keep our fourteen playtest seats busy all day long, so running playtest was a full-time job in itself.
Managing a Halo Wars playtest session was a lot like herding cats. Some of the playtests were scheduled, but late in the project, a lot of sessions would get called on an as-needed basis. I’d page for testers over the intercom, but since everyone had other responsibilities, I’d often only get four or five people on the first call. Usually a second page would fill the rest of the seats. If that didn’t work, I’d start walking around the office – it was hard for my co-workers to turn down a personal request, especially if I groveled.
Often we needed to test specific game features. An artist would come down, excited to take on their office mate in a 1v1 multiplayer match - only to be told that I needed him to run through scenario six on easy difficulty level. A group of testers might get asked to keep playing the same map over and over again with one particular Covenant leader, trying to find an elusive out-of-sync. For traditional QA folks, this was familiar ground - their primary job was to find bugs and play the same content over and over again. But for the rest of the team, the playtest process often felt like a grueling slog.
The 16th Floor Playtest Lab.
Toward the end of the project, Ensemble faced the closure of the studio. We wanted to make Halo Wars a great game and go out with a bang, but we were also under enormous pressure. The studio’s imminent demise definitely made for a few grumpy playtest sessions, but everyone remained totally professional throughout the difficult process. The feedback we got from those final sessions proved to be invaluable in helping us make the last few tweaks to the game.
Though we’re slowly developing our own identity at Robot and re-evaluating our old ways of doing business, we always assumed we’d continue to playtest our games early and often. Our new test lab may be eight seats instead of fourteen, and it may not have mood lighting and fancy automatic window shades (or, in fact, windows at all), but it’s still where the development magic happens.
The new Robot Entertainment Playtest Lab.
But before we could turn over the test lab to Robot’s new games, we had some Halo Wars business to attend to. About a month after we settled into our new office, the playtest kits were humming and the first cut of the Halo Wars DLC was ready to go. One afternoon, I was in the middle of working on a design problem when I saw Justin heading in my direction.
That day, it was his turn to herd the cats. I’ve been in his shoes. I immediately knew what he was going to ask me. I knew what exactly what I had to say to make his afternoon just a little bit easier and help make the Halo Wars DLC just a little bit better.
“Sure - I’ll come playtest!”