GDC 2017: Epic Games' Ben Lewis-Evans on Player Psychology and Rewards

Game developers and publicists have a habit of throwing around scientific terms and concepts such as Skinner boxes and dopamine levels to succinctly explain larger gameplay ideas to consumers. According to Ben Lewis-Evans, a user experience researcher at Epic Games, reducing such ideas to marketing shorthand results in widespread misuse. More importantly, a studio's failure to grasp the nuances of workings such as dopamine could cause them to do their games a disservice.

"Where most people talk about dopamine in literature nowadays is in wanting, or needing, and compulsion," he explained. "The idea is that dopamine has something to do with a magnetic draw or need for something." Lewis-Evans, who earned his doctor of philosophy from University of Groningen and has been involved in user research since 2013, cited meth addiction as an example. Although the drug induces pleasure at first, taking too much diffuses that euphoria. All that remains is a base compulsion: addicts need their fix, but they no longer want it.

"So if you're saying your game gives people a shot of dopamine, you're not necessarily saying it's fun or anything worthwhile to them, just that it's compelling them to play it, which may not be a nice thing to say," he continued.

Sugar Water

By Lewis Evans' own admission, even that analogy is a gross oversimplification--good for getting across the gist of dopamine, but failing to cement the more salient points. It's part of a growing trend in popular culture to slap a "neuro" label on everything: neuro food, neuro exercise, neuro training.

"The reaction in the neuroscience community has been to declare them neuro trash," he said. "To say, 'The brain is more complicated than this, and all this stuff is preying on cognitive biases and people with money.'"

Dopamine is a chemical compound found throughout the body, and is a precursor to adrenaline. Over time, developers and marketers came to conflate and reduce that sequence into a simple behavior-reward loop: perform a certain action every so often or a set number of times, and receive a reward. Hence, dopamine shots.

The belief that dopamine played an integral role in learning--and subsequently triggered pleasure --gained ground within mainstream culture. All the while, neuroscientists found themselves abandoning it.

"This was based on the idea that in the brain, just before a reward is delivered, there's an increase in dopamine," Lewis-Evans said. "You're anticipating something happening. That's interesting, but there's some problems in that dopamine-deficient mice we've observed learn where rewards are in a maze. Rats that don't produce dopamine have been properly conditioned to learn new things. So, maybe dopamine has something to do with learning, but obviously it doesn't seem to be necessary."

Lewis-Evans made the same observation about patients afflicted with Alzheimer's. Dopamine facilitates movement and conducts information throughout various areas of the brain, but the disease stifles dopamine and cuts into cognitive processes. Nevertheless, Alzheimer's patients still like or dislike things even factoring the loss of their ability to learn and retain information.

What Lewis-Evans aims to do is get the industry to set aside its misuse of neuroscience and focus on implementing schedules and rewards in effective ways. "This is what we care about, right? If we're making games, we care that what we do rewards people on a variable schedule and what the behavioral outcome is: strongly motivated players."

Inside and Out

Developers tend to classify player motivation as intrinsic and extrinsic. Intrinsic motivation stems from personal gratification, while extrinsic motivation is derived from a game's environment. However, Lewis-Evans' goal as a researcher is to consider as many reward models as possible, find overlaps, and spread that knowledge to make more developers aware of them.

"Like the brain not being simple, human behavior isn't simple, so try to not be so dogmatic about what we're talking about," he said.

One reward model is classic conditioning, most famously explained with the example of Pavlov's bell: Pavlov rang a bell, and his dog came running, knowing that food would be served. The videogame equivalent is Mario's coin. Players see coins hanging in the air, often twirling and shimmering, and collect them, triggering a telltale chime and bringing them one step closer to an extra life. "They're linked together in a performative way while playing the game, and it gives you progress, so they become associated as a reward," Lewis-Evans said of the sight and sound of coins in Super Mario titles.

Earning currency is a reward in and of itself, and currency comes in many forms. The most effective forms are those that players understand right away: gold coins, gems, dollar bills.  Games that introduce a made-up currency have to go to the trouble of establishing it, prolonging rewards.

Operant conditioning involves asking the player to perform a task and rewarding them upon its completion. "Fixed ratio is when you have to do a certain number of responses to get a reward, such as turns until a unit or building is ready in a turn-based game is a classic example of this; you just have to wait and hit that 'next turn' button," Lewis-Evans said, defining one type of conditioning.

The downside of fixed ratios is that players are prone to stop performing tasks if they have to wait too long before receiving another reward. The trick, Lewis-Evans explained, is to structure a game so that players can have lots fixed ratios ticking away at once, such as multiple buildings in various stages of construction. When one building finishes, another may be only another turn or three away.

Fixed intervals are another example of operant conditioning. Epic's Paragon spawns minions at fixed intervals. The precise timing of the intervals encourages players to keep coming back for the chance to kill enemies and scavenge them for loot. Mobile games that feature daily login bonuses represent another form of fixed-interval rewards.

Unexpected rewards throw players for a loop, but in a good way. "An example of that would be in a game like Portal 2, you unexpectedly work out a way to solve a puzzle," Lewis-Evans said. "It's very satisfying. It's not something you necessarily saw coming, and it feels good." He noted that a high frequency of unexpected rewards dilutes the excitement players feel upon receiving them. They come to expect the unexpected.

Perhaps the most notorious form of conditioning, variable ratios derived from gambling. "You have a variable number of responses until a reward shows up. So it's not always 10; it might be 10 to 50, something within that range. Loot drops like the loot-crate system in Paragon, and critical hits are examples, where you hit a certain number of times and then do more damage."

Finally, variable intervals are most prominently on display as public events in games like Destiny. These events are erratic, rarely occurring on the same date and time. Consequently, players flock to an area where they know an event will take place and hang around until it transpires.

"In real games, you'll be mixing all these schedules together and they'll be running over top of each other, but it's interesting to pull them out one by one and look at their processes," Lewis-Evans said of the many types of reward models.

Good feedback is its own reward, and the most important type is game feel--the simple yet pivotal act of pressing buttons or keys, and moving control sticks or mice across pads. As an example, Gears of War's active reload system, where players press buttons at precisely the right moment to for a faster-than-usual reload, transformed a mundane action into a mini-game. Active reloads also trigger a flash and a sound of excitement from the player-character--visual and aural feedback that, like the telltale sparkle and chime of coins in Super Mario Bros., reinforce the player's success.

"If you take nothing else away from this talk, please take this," Lewis-Evans stressed. "If players don't know they got a reward, they can't get it again. If players don't know why they got a rewards, they can't do it again. If players don't know the value of a reward, they may not care about it. You can give them all the stuff in the world, and they [won't care]. Rewards are feedback, fundamentally, and a lot of feedback can be a reward."

Making Progress

Many games begin by setting up a long-term goal. Unfortunately, humans tend to falter when asked to hold out for long periods of time. By leaving a breadcrumb trail of short-term rewards, players are less likely to become distracted or dispirited on the road toward long-term rewards.

"Progression bars or experience bars are classic examples: little bursts of feedback and rewards let you know you're making progress toward a bigger [goal]," Lewis-Evans said.

Paragon sprinkles weekly quests, leveling up, learning new abilities, and victory chests along the player's path. Each shows players that their actions are being rewarded, that their characters are growing stronger, and that they are becoming more adept as players. Besides mandating that players perform every action in order to move on, Epic's Unreal Tournament reboot offer another layer of progression early on in the game. "The interesting thing about the UT tutorial is that it has optional skill tokens," Lewis-Evans said. "As the game teaches you, there will be skill tokens hidden around the tutorial that kind of build on what you've already learned. If you've learned jumping and wall running, maybe in the next room off in the corner, there's a token spinning that requires you to jump and wall run. You don't have to do it, but it's a way for you to put together what you've learned and show your progression."

Progression can factor heavily into story-based games, or not at all. In The Witcher 2, certain choices made by the player cause them to skip huge blocks of content later on. Telltale's The Walking Dead stands in contrast to Geralt's adventures. "It was praised for its feeling of choices, even though if you go back and make different decisions, you find out that the choices don't strongly matter to how the game plays out," Lewis-Evans remembered.

However, how momentous a choice feels in the moment is arguably more important than how its consequences play out. "There will be different people who fill in at different times for people you let die or whatever might happen, but because you feel like you were making a big decision at the time, it feels important."

Activities such as side missions and escort quests influence progression for better or worse. If players feel like they cannot progress unless they grind out items or experience, they're likely to grow frustrated and give up. The same rule applies when the character they're supposed to be protecting attracts unwanted attention during stealth sequences or gets killed too often during battles.

Escort quests are widely panned in large part because the unpredictability or unreliability of an AI-controlled companion takes success or failure out of the player's hands. "Modern escort quests like The Last of Us solve this problem by making the person you're escorting [exempt] from those behaviors: they can't break cover or can't die," Lewis-Evans said. In The Last of Us, Ellie has a tendency to scuttle around in plain sight. She's invulnerable, and enemies don't react to her; the trade-off is that seeing Ellie dance around right in front of hostiles breaks immersion, but not having to worry about her behavior mitigates that break in suspension of disbelief.

Control Freaks

Players want to be in control at all times. If they're not, they tend to act out. Players forced to grind in order to progress may stop playing. Even so, savvy developers can harness their players' desire for control to create interesting gameplay scenarios by encouraging players to rebel.

"A classic example of this is The Stanley Parable, which is a game all about a narrator trying to control you, and your attempts to break the control," explained Lewis-Evans. "It feels very satisfying to do so, even though you ultimately fail."

Establishing control right at the outset goes a long way in determining whether players push on. Too often, developers err on the side of too little information--believing that smart players prefer to figure things out for themselves--or bombarde them with information all at once.

"One of the really big ways to feel out of control is to feel confused," said Lewis-Evans. "[An advantage to] having a little bit of hand holding, a little bit of guidance in tutorials, is that it lets players get to a point where they can feel in control. You tell them what to do, then they have control and show that control. That's one of the nice things about optional skill tokens in Unreal Tournament: they're optional things players can collect to show their mastery."

There are passive ways to give players control. Noncontingent awards are known colloquially as participation awards--the trophies that Little League players get just for showing up. Despite the disdain for them that radiates from certain pockets of society, noncontingent awards have proven effective in games on two fronts. They're not controlling because most players have no idea they're meeting criteria to receive them; and because players are unaware that they are putting in work toward them, noncontingent goals are generally welcomed when they are received.

"Of course, because they're not based on performance, they're not giving you feedback on anything," Lewis-Evans added.

Noncontingent rewards work well in casual and "core" games alike. In Mario Kart, players who fall behind have a greater chance of receiving powerful items like the blue-shell homing missile that streaks toward the player in first place. In Titanfall 2, players steadily fill up the meter needed to summon their Titan mech regardless of how well they're performing in a match.

"That's what these non-contingent rewards can be quite useful for: balance, and to make sure people experience more of your game," said Lewis-Evans.

Anti-Social

Social data can be used as another means of tracking player progress. The drawback is that comparing players against their peers opens them up to abuse or ostracization. Players who notice that one person did poorly might write them off as a terrible player and either berate them or steer clear of matches that involve them.

When used haphazardly, social data can have the opposite effect of that which the developer intended. DOTA 2 gives players combat summaries that show how often other players have reported them for perceived negative behavior. The idea is that players flooded with poor report cards will change their ways. A risk, though, is that players might notice that someone with an abundance of negative reports still gets away with their crimes, encouraging well-behaved players to adopt a if-you-can't-beat-em-join-em mindset.

"Certainly in research where they've shown people their energy use, and you can see, hey, your neighbor uses more energy than you, congratulations--the people being told they don't use much start using more. They're like, well, everyone else is doing it. So you've got to be careful when presenting that information," Lewis-Evans said.

As a means of skirting social abuse, games such as Overwatch focus on comparing each player against his or her own performance rather than their partners and opponents. "So even if you lost a match, maybe you got better at something," Lewis-Evans said. "Maybe it was your best game for healing. That's some kind of reward that lets you know that you're progressing in some ways, and isn't a negative comparison against other people."

Another way of getting around the negative aspects of social data is by including competitive modes. "Nearly every top game you can think of nowadays has some kind of cooperation element," explained Lewis-Evans. "According to research, cooperation motivates people a lot more than competition. Competition has been shown to do things like suppress creativity, cause conflict, all those kinds of things."

Game of War: Fire Age contains one of the strongest implementations of cooperation: reciprocity, or the act of one player performing an action that benefits everyone. "When anyone in your guild spends, it benefits everyone in the guild," Lewis-Evans said. "That's kind of the idea of positive externalities. That's an economics term that says by spending in an economy, you don't just benefit yourself; you benefit everyone around you. That can be a powerful motivating force. It says that by contributing to this game, you're part of something bigger. You're helping friends. Also, if you've already spent as you want to help yourself, you can continue to spend to help other people and feel like a good person."

Biased

Regardless of how players mix and match reward models and methods of progression, Evans-Lewis warned of a minefield of cognitive biases. Players assume, quite naturally, that putting more time and effort into a game results in comparable rewards. Developers should respond in kind, although games that lean on procedural content, such as roguelikes, may need to take extra steps.

For instance, it's not uncommon in a procedurally generated dungeon to find a large chamber teeming with enemies. Players fight their way to freedom, expecting a grand prize in the next room--only to find nothing, or very little. "That can be taken into account with reward systems being tied to the proceduralization of those games: if you go over a certain amount of difficulty, you should get a better reward," said Lewis-Evans.

Striking a balance between time put in and rewards doled out is critical in games where players pay real or in-game currency for materials. "You need to make sure your economy is balanced to support the people who spend more in your game. It's important to go back to the idea of feedback and make sure everything is very clear: the value of buying things is clear, and the value of things you can purchase with real money is clear before you make the purchase," Lewis-Evans said.

It's okay for games to even the score behind the proverbial wizard's curtain. Lewis-Evans revealed that some games increase the odds of fortuitous events occurring--more and better item drops, higher chances of landing critical hits--without notifying players of the change. Consciously or otherwise, players realize that spending money gives them more than one edge against the game, and are more likely to spend again.

Along the same lines, a string of bad luck should be mitigated with a payoff. "This is what keeps people gambling, but it's something we can use in games to reinforce this expectation and make people feel like your game is fair and in line with what they want," Lewis-Evans said.

The UX researcher held up Destiny as a shining example of payoffs. When running strikes, the player's Vanguard bonus helps to determine the randomized loot they get. Running more strikes ups their Vanguard bonus, rewarding players who put in time and effort.

Bungie has had to course correct more than once over Destiny's lifecycle. As an example, engrams are color-coded items that players receive by killing enemies, running strikes, playing Crucible matches, or by purchasing them from certain NPCs. An engram's color communicates its value. Early on, most engrams were given the highest possible color. However, the game didn't clarify that the color represented the highest possible quality of the item. Understandably players assumed they were getting something extra special only to be disappointed.

"My understanding is that Bungie didn't change the loot or underlying math," Lewis-Evans said, "they just colored engrams at the lowest possible [reward]. So now, you see a blue engram, and you know you'll get at least a blue item, but maybe you'll get a surprise and receive an exotic. It's important to be aware of expectations you're setting."

Loss aversion is one of the trickiest cognitive biases to reconcile. In effect, this bias states that players will work harder to hold on to what they have than they will to earn something they might get. This leads to players abstaining from using consumable items like potions even if it means their character dies and they have to start over from a checkpoint.

"You need to make sure the base drop rate is good enough that maybe people will use it, or you demonstrate the value of doing so," Lewis-Evans urged. "Paying to continue is a classic example: you're paying to keep the progression you have."

Lewis-Evans brought his presentation to a close by doubling down on what he viewed as his most important findings. "I really hope you can be skeptical of simple explanations, whether they be neuro or not. Reward is feedback. Please do provide feedback and rewards as close to the behavior of the reward as possible."

David Craddock

Long Reads Editor

David L. Craddock writes fiction, nonfiction, and grocery lists. He is the author of the Stay Awhile and Listen series, and the Gairden Chronicles series of fantasy novels for young adults. Outside of writing, he enjoys playing Mario, Zelda, and Dark Souls games, and will be happy to discuss at length the myriad reasons why Dark Souls 2 is the best in the series. Follow him online at davidlcraddock.com and @davidlcraddock.