Game Testing: How To Make People Love Your Titles?

Experts from Braze, a mobile engagement automation company, define QA as a ‘key brand differentiator’. Quality improvements influence user experience and engagement, which are the key factors of game popularity.

Appboy professionals release studies about app users retention stating that less than 25% of the first-day users got usually engaged and decided to play the game further. So, they conclude everybody involved in the game development process contributes to user happiness, but QA is still the stage, which can not be omitted. Moreover, they say only 29% of tech companies perform exploratory testing (testers check the product while it’s being developed, not after full completion).

Both developer-based testing and QA are able to deal with commercial release bottle-necks, downtime on production, and decrease the number of significant customer defects. If anybody still doubts QA really provides more people’s love to your titles, here are some examples of the failed games, where not enough testing were performed:

• Dull and disappointing release a couple of years ago, Ni no Kuni II, with lack of voiced dialogues;
• Vampyr game disorder in a scenario, which was criticized by the users due to game complex battles;
• Seven Deadly Sins are much spoken about because game characters do not match the words they pronounce, which produces poor gameplay effects.

A lot of features are critical for a fun product: music, sounds, dialogues. If you don’t want your game to lose real fun, user engagement, and glory, QA must become one of the mandatory stages in your product software development lifecycle. And you are welcome to read this article to learn more about quality assurance in game design, its place in the software development lifecycle and get more insights about QA types, how to test different games as well as advice on how to find game bugs.

What is game testing and its place in the game development lifecycle?

Idea or a concept is a real start for any product. And to make it implemented, a range of stages should be followed. Cliff Bleszinski, a designer for Epic Games known for his Gears of Wars series, says about the game development lifecycle: ‘Nobody in-game industries can predict what their activities are, they just have a gut assumption’. But in spite of this humorous saying, let’s specify what game testing is and which place it takes in the entire game development process.

Game testing is a stage in software development applied for quality management and finding bugs. Testing is demanded during the development cycle to be ready at least for the Alpha launch which is an early version with minimum testing done and with high potential to be unstable.

While testing, the QA engineers must find the answers to the following questions:

• whether the character can or can’t complete the activities foreseen by the script;
• if the character is unable to overcome obstacles at some point;
• what dialogues are boring or not clear enough;
• whether the rules or levels are too complicated (or easy and not engaging);
• is there a ‘fun factor’ (whether it is satisfying enough when completed fully).

Answering these questions is vital because the game has all chances to fail if we do not know whether its mechanic works well if rules and levels are engaging enough to prompt the users to play again and again, what features need to be polished more before Alpha and Beta release. That is why QA is a part and parcel of the game development lifecycle alongside with:

• Planning or discovery stage: GDD (game design document) is created. In this document production team outlines game mechanics and rules as well as future design;
• Production is a time-, efforts-, and resources-consuming stage. It’s aimed to elaborate thoroughly every component of the game (scenes, characters, levels, scenarios) to render it on the screen. During this stage continuous iterations and exploratory testing are performed;
• Testing for quality control targets identifying bugs, checking smooth running of the game, and the rules (or levels completion) clarity. Sometimes a bug sheet is delivered to the devs as an outcome of this game development stage;
• Pre-launch in game development is an important milestone as it is both the testing and marketing stage to make the users get acquainted with the product before the Beta version is released. It’s a good chance to learn players feedback as well as to present the game at famous international industry events: E3 (The Electronic Entertainment Expo — annual trade fair) or PAX (Penny Arcade Expo — series of game cultural festivals);
• Launch (release) takes place when the game code is clean, and QA engineers complete the bug sheet;
• Post-launch is an exciting period of getting outcomes, but it’s high time to update the released game as well. And testing is still here: post-launch support is getting users’ bug reports continuously. Then, DLCs are issued (which is game downloadable additional content) and game patches are delivered by the development team.

Game development process takes a lot of time and demands a number of iterations, much testing, and launching Alpha and Beta versions, which leads finally to the engaging playable game. Every stage involves testing: exploratory one throughout the whole development process; clean room procedure to assure code reliability; users’ post-production reports as a kind of testing as well. Once testing is so important, we propose to dive deeper into stages, techniques, processes, and approaches to be aware of what is game testing inside.

Game Testing: The Complete Overview of Stages, Techniques, Processes, and Approaches

Marissa Mayer, the Yahoo CEO, noticed in her Twitter: ‘Geeks love tech issues so much, that all the details are of crucial importance’. This quote is relevant to our topic as well. Assuring game quality we exclude irritating bugs and errors spoiling nice game concepts. So, what stages should be followed to create the perfect product?

What are the stages of game testing?

We propose to follow every stage to learn about objectives, responsible specialists, and the sense of the procedure.

Stage: closed Alpha testing takes place on the preliminary level
Objective: identify bugs, which were not obvious in the earlier development stages
Who performs: a group of users reflecting the real user environment and playing as if it happens in real life after the game release.

Stage: closed Beta. It’s quite similar to closed Alpha but more extensive
Objective: figure out the potential obstacles to the natural smooth playing flow
Who performs: volunteers from the users, not development team members

Stage: open Beta is a stage, during which all users are welcome to play the game and report any poor functionalities
Objective: improve user experience and define any broken code in a production environment
Who performs: every user willing to play

Stage: Fun factor, balance, learnability, or reliability tests
Objective: make live user test process successful
Who performs: real players or focus groups

All the stages involve game testing processes delivered both by QA engineers and real players. We propose to follow every stage for a better understanding of the whole procedure.

What does the process of game testing consist of?

Game testing processes align with a basic typical SDLC (software development lifecycle): the game is tested on such aspects as endurance, performance, engaging user experience. In a plane way we can imagine game testing as that consisting of the following processes:

• Requirement elicitation, which stands for total game understanding (scenario, characters, game mechanics, levels completion concept and etc.) to create the most efficient game testing strategy;
• Game testing strategy determines the following: preparation of a certain document specifying testing roadmap, how many checking cycles are planned, what types of testing should be performed, and what processes must be presented (bugs, code defects, level errors, or game scenario irregularities);
• Test cases design. Each test case consists of different test inputs, conditions of their implementation, and outcomes available to be in focus;
• Game test cases execution gives the possibility to find more bugs and errors;
• Test results record: presenting all the bugs found in some document;
• Defect log maintenance: categorizing all the defects found to ease the further work on the title for the dev team.

Once we’ve answered the questions ‘What do we do?’, ‘Which tiny processes and operations are included in the activity’, it’s reasonable to answer ‘How do we perform it?’ Let’s consider what the basic game testing techniques are known.

What are the types and techniques of game testing?

Before commercial product release engineers work on finding bugs and errors. Quality control can be as following:

• If we want to know whether the product meets the specification requirements, the team performs functionality QA. It’s testing user interface (UI), making sure the game is user-friendly and all the mechanics work smoothly;
• Sometimes it’s necessary to check as many game situations as possible, in this case, combinatorial testing is applied (specialists generate more and more test cases);
• Spontaneous random testing with no plan or case list is called Ad Hoc one. It’s implemented as Monkey (random inputs to test possible game failures) or Pair (two engineers work on one keyboard to test the app robustly) testing;
• To assure whether the game works well on any device with any OS compatibility check is used;
• Cleanroom and tree testing is about code clarity checking;
• If after fixing the bus, it’s worth tracking whether errors occur or not in the same situations, regression testing is performed to run the past test cases;
• When the team tests an overall game, it is performance testing.

QA is applied practically on every stage of SDLC being composed with a number of processes and performed in a variety of manners. It could be done either manually or in an automated way. What’s better?

Manual or Automation: What’s best for game testing?

Sometimes the game is tested manually, sometimes the tech team writes special scripts to automate quality control. Both methods have pros and cons. We propose the chart below for better awareness of the differences considering a variety of the parameters.

Manual testing is convenient because it provides:

• Accurate visual feedback;
• No need to spend budget for automation tools design;
• Human experience and intuition always contribute to the testing process;
• For testing small change one should write the whole new test, and it’s time-consuming, while manually it takes a little time and effort.

At the same time automated QA:

• Is more productive, able to find more errors in the same period;
• Recorded thoroughly, and it’s possible to perform the same testing again and again;
• Supports a variety of applications;
• Has wider coverage as automated tools can check the tiniest units.

For getting the best of human creativity and automation precise efficiency sometimes there’s a need to combine. Exacol, a database developer, counts manual testing as 100 000 times slower than an automated one. They think it makes sense to automate repetitive tasks anywhere in development, but especially in testing. So, consider the pros and cons of both methods, and estimate which proportion of each testing you need on your project just like we do for our clients at Innovecs.

How long does it take to test a game?

If you want to be aware of time limits, it’s worth imagining that while testing every character must face every other character in every possible game situation on every possible level. It sounds quite time-consuming. It’s a lot of repetitions, which are not only about playing games but about repeating the same game fragment trying to break it. Time should be spent for finding code irregularities, art glitches definition, level bugs outline, and logic game errors excluding.

• Your time scale of testing process includes:
• Identifying incorrect software operation;
• Reporting the bug in the defect tracking system;
• Checking: artist, programmer, level designer research the defined malfunction;
• Verifying: after the developer spends time fixing the bug, the tester should prove an error doesn’t occur any longer.

To be aware of deadlines, testing plans are helpful with detailed objectives, resources, and processes. Of course, it’s possible to evaluate the timeframe for testing considering experience with the same games, your knowledge about testing processes, and awareness of how much time manual or automated testing requires. And we guess the more facts you know about QA, the easier it is for you to imagine the process and plan to outsource game testing. Hence, here are more smart details about bug types, dealing with which is helpful in improving user experience.

Common Types of Bugs That Will Help You in Game Testing

The fact that such a giant of the game industry as Sony announced a global Bug Bounty Program proves to us that it’s worth paying a lot of attention to testing if you want to get a profitable product. So, companies seeking opportunities to provide an engaging experience for their users’ community will test the game more and more times to ensure perfect quality. Let’s follow some glimpses concerning which usual bugs are:

• Graphics glitches, which are wrong image display, looking like this one from Sims 4 on the screenshot (a comedic one);

• Gameplay defects: certain function is not working like it was planned inside the game;
• Freezing, hanging, or crashing;
• In-game purchase problems: the absence of the necessary payment options or too complicated buying process;
• Audio defects are important to fix as the right audio and music create the very mood of the game;
• Text problems: wrong, not aligned, or missed text options;
• Game save glitch, which is one of the most upsetting for the players, because they invest time and money to play the game further, not from the beginning, which can occur in case of wrong level saving options.

Considering all these issues assists in avoiding bothering, annoying and game-breaking experiences. The testing focus should be placed on all of them so as not to miss the tiniest feature able to stress the player and influence on your ROI in the future.

Conclusion

To sum up, in this article we have outlined and described the following:

• QA is a substantial and critical part of SDLC, missing which leads to game quality losses;
• Testing comprises substituent processes aimed at bugs identifying, reporting, analyzing, and verifying;
• Quality control can be performed in different manners depending on the objectives (retesting the same case, comparative analyses, or breaking game in pair);
• Manual testing is more about expertise, human intuition, and precise checking while automated one is about wide coverage and time efficiency;
• To estimate testing time consumption it’s better to deal with a testing plan.

For your success in launching a bug-free solution, Innovecs provides smart QA engineers who can protect your game from failure risks, execute a variety of test cases to deal with the bottlenecks as well as prevent financial losses due to poor game code. We are a really trustworthy partner with QA expertise applying the latest and greatest new tech solutions.

In the end, we’d like to quote Andrew Moore, the Head of Google Cloud AI, who notes smartly on his Twitter that he’s extremely glad to live in the world, where it’s routine to perform multiple complicated processes ‘in a jiff’. And we could but agree that new technologies (in testing as well) contribute greatly to make outstanding titles efficiently.