Happy Accident: The Making of Pokemon Go

In most Pokemon games, catching a Snorlax doesn’t elicit much excitement. Unlike rare pocket monsters such as Mewtwo, the rotund pocket monster can be found snoozing in front of specific paths, blocking players from advancing until they toot a flute and awaken it to do battle.

The sudden appearance of a Snorlax in Niantic Labs’ Pokemon Go generates significantly more fuss. In August 2016, a mob of thousands of players brought traffic in the Taiwanese capital of Taipei to a halt when the sleepy monster popped up in Beitou park. Eyes glued to their smartphones, the throng poured around cars and busses like water breaking around rocks, elbowing, shoving, and stampeding forward to collect the rare specimen.

Pokemon Go incited innumerable flash mobs during the summer of 2016. Just after 11:00 p.m. in mid-July, a Vaporeon popped up near Central Park. “Vaporeon!” someone shouted. Like sailors drawn by a siren’s call, hundreds of heads swiveled toward the voice. Players rushed through trees and dashed across streets, paying little mind to oncoming vehicles. Not that they had reason to worry. Cars sat gridlocked in every direction, brake lights dotting the night like the glowing red eyes of a Hoothoot. Many players clambered out of rides, leaving doors ajar as they sprinted to join the crush of trainers closing in on the water-type Pokemon’s location.

“When Pokemon Go launched, the conversation was all about augmented reality,” said Dennis Hwang, director of visual and interaction design at Niantic Labs, in front of a packed crowd at this year’s Game Developers Conference. “The capture mechanic that we designed had utilized the smartphone's camera feature. For us, it was a happy accident.”

Augmenting Reality

Hwang might have been too modest. After all, Niantic Labs and its AR-centric games had been on the tips of tongues since 2013 when its breakout hit Ingress changed the way players and developers viewed mobile games. Causing that very type of paradigm shift had been on the mind of founder John Hanke, the tech entrepreneur responsible for the rise of Google Maps and Google Earth before he received a generous stream of resources to found Niantic Labs as an internal division.

“We were founded in 2011 within Google,” Hwang said. “We're called what's an autonomous unit, so we're kind of a startup that was incubated within Google. We spun out in 2015.”

Out on their own, Hanke and his small but scrappy team allocated their time and energy toward changing the perception of smartphones. “You go to a beautiful beach or hiking trail, and inevitably you see a family, and the children are begging for attention while the parents are reading reviews or checking their email,” Hwang explained.

While Niantic’s developers recognized the potential of smartphones, they saw room for improvement. Not only did people concentrate more on what was happening on their screens than on the people around them, even the most profitable mobile games tended to be simplistic. That baffled Hanke, who thought of Android and iPhone platforms as veritable computers able to be held in one hand and fit in a pocket. Each unit boasted all sorts of sensors, high-res touchscreens, fast processors, and frills like GPS. Surely game developers could do more with all those accoutrements than churn out endless runners and ports of old PC and console games.

The upstart’s answer was Ingress, a sci-fi game riddled with conspiracy theories and tech talk. Ingress was an augmented reality title, a seamless blend of game elements such as polygonal characters and special effects overlaid atop real-world locations. To most pundits, AR was a pipe dream that would only be made possible by donning headsets like Microsoft’s embryonic HoloLens. To John Hanke, the means to paint in-game graphics over reality were already widely available in the pockets of most consumers.

“Part of it was the navigation experience, like Google Maps,” said Hwang. “If you think about it, a map application is giving you guidance in the real world. It's taking you from point A to point B in a timely, concurrent [manner] through all of its computing capabilities and sensors.”

More than a compelling sci-fi adventure, Ingress was an expression of Niantic’s collective view on the potential of augmented reality. Players interacted with the game by walking to real-world locations, completing missions such as erecting or destroying barriers belonging to a rival faction, and sharing data about the spots they visited with Niantic. In turn, the developer fed that information into a database of points of interest brimming with cultural or historical merit.

Requiring that players ambulate to points of interest was no happy accident. “Our goals were pretty modest initially with Ingress,” Hwang explained. “All we wanted, really, was to change someone's habit. Take your typical office worker: If we could change that person's habit during their commute where they walked an extra few blocks [to play], we'd be happy. We knew how hard it was to motivate someone to get out of a chair and move to use an app or a game.”

Niantic encouraged players to get up and move by implementing time-tested game mechanics. A game loop centered on exploration and collecting objects keeps players engaged during their walkabout. Every in-game action consumed energy, and the only way to stock up on energy was by walking.

Invariably, Ingress players bumped into each other. Public parks, shopping malls, at the bottom of a driveway—virtually any location could be a hotspot in the game. “You might battle it out all day long and then go out for a drink afterwards, get to know each other better,” Hwang said. “We started seeing Ingress marriages and Ingress babies, which was pretty awesome.”

In no time at all, Ingress swelled into a global phenomenon. Players went from organizing meetups with friends to massive get-togethers. “All around the world, every once in a while, thousands and thousands of players come together and duke it out all day in what's called an Ingress Anomaly,” he continued.

While players fixated on leveling up in the game, many underwent changes outside of Ingress. Players got tattoos to declare their loyalty to the Enlightened or the Resistance, the two factions warring over territory and control. “One of the designers and I looked at each other and said, ‘Oh, no. We can't ever change this logo again.’ We were still in beta, so we were always saying, ‘Oh, yeah, we'll go back and polish this or that part later,’” Hwang remembered.

One player belonging to the Enlightened was so intent on thwarting a Resistance scheme that she chartered a plane to fly her to a remote Alaskan town. Her goal was to connect to a point of interest registered there and claim it for her team. Upon arriving, a storm prevented her phone from making a connection. A real-world problem surfaced: Snow and ice coated the runway. The pilot urged her to get back in the plane so they could depart before conditions worsened. She held steady until her GPS registered a ping. The connection was made. She and the pilot could go home.

Walking and narrow escapes inspired a healthy side effect. “People were also losing weight playing our game,” he said, smiling. “Because our game requires you to walk, people were collecting tens or even hundreds of kilometers, just from playing this mobile app.”

Even as players branded their flesh and risked life and frostbite in Ingress, Niantic was laying groundwork for an even bigger augmented adventure.

Dissonance

Shortly after Niantic Labs struck out on its own in 2015, Google, Nintendo, and The Pokemon Company invested tens of millions to support the company’s ventures. Ingress marked an impressive step on its road to success. Pokemon Go, an AR game that challenged players to assume the role of Pokemon Trainer and collect the world’s most popular monsters, represented a leap.

"We wanted a more game compared to Ingress,” Hwang said. “We wanted to appeal to a wider demographic, a wider age group across genders. We made a lot of concept designs and tried to move forward in building a greater sense of immersion, but we ran into a lot of challenges."

One of the first challenges was landing on an art direction. Most Pokemon games skew toward younger audiences. Hence the player assuming the role of an adolescent trainer embarking on a quixotic journey to wander the world collecting Pokemon. In order for Pokemon Go to appeal to a wider demographic, Niantic needed to age the franchise’s archetypical protagonist.

After experimenting with an anime approach to visuals, the game’s artists reined in their influences and settled on a slightly more realistic depiction. "Toward lessening cognitive dissonance, we didn't want super stylized, big-headed-but-small-bodies characters running around on a map when you're [an older player],” said Hwang. “We wanted to match the virtual with the physical as much as possible."

To design Pikachu, Squirtle, and the hundreds of other pocket critters that roam Pokemon titles, Niantic received character data from DS Pokemon games and a surprising amount of leeway from Nintendo and The Pokemon Company. "The tech artists were really able to massage the data,” Hwang said, “not just the vertex data, but the texture data came over pretty [well]. But it was still very subjective for the look and art. There were a lot of directions we could have gone in terms of shading the characters. Our concept artists really had to go model by model and tune layers, or get the colors right."

Pikachu still had to look like everybody’s favorite electric mouse. Even so, the AR developer had the freedom to tweak the look of both Pokemon and the player-character to best fit its team’s designs.

"We went with a little more conventional approach with the avatar,” said Hwang. “Even though our team was small, we invested a lot of effort into avatar customization so players can customize their look."

Cognitive dissonance was arguably the most formidable obstacle Niantic faced in draping Pikachu and friends over the real world. As players moved about in Ingress, their phones depicted a map in lurid blues and greens, a color scheme and interface straight out of The Matrix and Neuromancer. The game’s visuals fit its theme, and served a second purpose. Niantic engineers worried that phones available in 2012 would be unable to juggle real-world visuals augmented by game trappings. They got around the issue by letting users submit data such as photos that pertained to points of interest like the Eiffel Tower, and showed the photo in the game.

For Pokemon Go, Niantic wanted to push the envelope of smartphones. At the same time, they were aware of limitations that might break players’ suspension of disbelief.

“Any time your brain is struggling to make sense of these two things that are clashing, that's cognitive dissonance” explained Hwang. “For us, what that means is if you're out in the real world, and let's say your screen shows this beautiful snowy mountain but you're on a beach on a sunny day, that separation between what you're being shown and where you're at physically is the cognitive dissonance that Niantic was struggling to solve.”

One early attempt at a solution was to incorporate photospheres, a 360-degree view of real locations most commonly associated with apps such as Google Maps. Although they showed more of an environment than a single photograph, photospheres were still static. Unmoving.

"If you just plop down a model on a static texture-mapped sphere, and if that placement doesn't have any awareness of where it is in the real [world], depending on its height, Pikachu could look like a giant monster, Godzilla sized,” said Hwang. “Or he could look really tiny.”

Another problem was that data used in photospheres tended to originate from cars. Driving around and recording footage made it easy to capture, but Niantic wanted Pokemon Go to center on walking to and from destinations, like Ingress. The final solution came from a Niantic engineer who whipped up a simulation of catching a Pokemon by battling it and throwing a ball.

"He's sitting on the floor to kind of get himself a good canvas to work with,” Hwang said. “This is sleight of hand. It's not true awareness of the environment, so not true augmented reality in that sense. But the experience when we tested it out in prototypes was just so compelling. The sense of immersion connected to the location you're standing in was really special."

One-Handed Pokedex

Developers of AR games have a harder time creating verisimilitude and immersion than those who create virtual reality experience. Users who slip on a VR headset are ready and able to accept any setting since environments are created from scratch. AR studios like Niantic walk a finer line.

"The map has this conflicting goal,” Hwang explained. “On one hand, it needs to function like a map application, a navigation application. It needs to be like Google Maps, but on the other hand, for Pokemon Go, it needs to function as part of a game element. Fun visuals tend to be a little bit harder to see. Effects can skew functional information."

At the start, Pokemon Go’s map was ultra realistic. The interface was clean and utilitarian. Anyone who looked at it would have struggled to point out differences between Pokemon’s vibrant world and a real environment viewed on Google Maps. Soon enough, the team voiced reservations. Although the map excelled at communicating information, it was too sterile. Too far removed from Pokemon’s lush topography.

Niantic traded one extreme for the other. In their next prototype, environments were verdant and colorful, just like in the Pokemon games for Nintendo’s handhelds. That, too, presented an issue. "When you start putting in trees, grass, and birds that aren't really there,” Hwang continued, “it starts to feel a little too much like the cognitive dissonance I was talking about. If you're at the middle of a [busy] intersection and you're staring at this lush vegetation, it's not quite the balance we wanted to achieve."

Niantic arrived at middle ground, crafting a colorful map that conveyed the info players needed to find and capture Pokemon near their location. Meanwhile, engineers and artists devised a user interface that would make it convenient and easy for players to interact with the game as they walked around with their eyes glued to their phones.

"In Ingress, on our portal screen, we had a lot of shortcomings,” Hwang admitted. “It was a very precision-, tap-based UI. Often the controls were in the wrong place, so you had to reach to the upper-right corner of the screen to get to the main game-action menu. The taps required for main game actions felt like a webpage navigation screen. I used to be a web guy, so I was applying past experience in the wrong spots. This was very movement-unfriendly, I would admit. With Pokemon Go, we wanted to change that for the better."

Pokemon Go’s UI improved on that of Niantic’s previous game by centralizing the most important controls at the bottom of the screen. Users can reach those critical controls, such as swiping up on a pokeball to fling it at a weakened Pokemon, with either thumb.

"Personally, I think swipe gestures on phones feel so satisfying because while you're swiping, you have a pretty good grip on the phone,” said Hwang. “So even while you're walking and playing Niantic games, that grip on the phone is quite important. You don't want to be doing precision tapping. In Pokemon Go, you can swipe to catch the [Pokemon]; you can swipe down to close panels; you can open panels using the swipe gesture. We tried to use the swipe quite liberally."

During testing, Niantic found that holding a phone in both hands curbed their momentum. It was much easier and more fun to hold a phone in one hand and swipe with a thumb to rotate the camera and equip pokeballs. "Most game actions you can get away with doing by using one hand,” continued Hwang. “If you live in a really cold country, the last thing you want is to take both hands [out of your pockets] in cold weather."

Poke Pros and Cons

Over 20 straight days of scavenging far and wide for pocket monsters, 32-year-old Sam Clark walked over 141 miles. When he downloaded the game, the father of four weighed approximately 308 pounds. After just over two weeks of play, he claimed he dropped to 264. In an article published on CNN, health experts pointed out that although Clark’s rapid weight loss seemed exaggerated, walking was an excellent way to burn calories and shed pounds. If a video game inspired players to get off the couch, all the better.

Clark wasn’t the only one hoofing it far and wide to catch Pokemon. Part of Pokemon Go’s game loop involves rewarding players for walking certain distances. For players with a penchant for adding notches to their achievement belts, being praised for walking is as worthy a goal to chase as capturing a Snorlax.

"With Pokemon Go, people have walked billions of kilometers,” Hwang said. “The game's dynamic elements that we're constantly thinking of, starting with egg hatching and rewards for walking, that's something we'll keep doing."

Given the success of Ingress, Niantic expected side effects from Pokemon Go such as players losing weight and communities hosting massive Pokemon crawls. What no one anticipated was the scale and magnitude of the game’s success almost immediately upon release on July 9, 2016. Internally, team leads composed a chart describing expected traffic to the game’s servers. An orange line represented their educated guess based on data gathered from Ingress. A red line was a worst-case scenario, a volume so huge their engineers would need to be standing by to keep servers up and running.

The reality, colored in days after launch, was shown by a green line that dwarfed the wild expectations of the red. "You can imagine our poor server engineers who were working around the clock, trying to keep the service alive,” Hwang said. “We were still just amazed and grateful by the world's response."

Other benefits proved just as surprising. C.S. Mott Children’s Hospital in Ann Arbor, Michigan, used Pokemon Go as a way to get sick children up out of bed and exercising. Doctors and nurses found that letting the kids play a game mitigated some of the fear patients experienced as a result of being sick or injured and cooped up in a strange place.

In other cases, the game has provided a way for children with special needs to make new friends. "We were getting letters from parents who had autistic children, and they were just thanking us, saying that for the first time in a long time, their children were able to interact with others in a very healthy way,” said Hwang. “When they were out in the park, for example, everybody was welcoming them because they were all out there playing Pokemon Go together. We've got a lot of touching letters from all around the world along those lines."

Not every side effect of Pokemon Go’s success was cause for celebration. In the game, PokeStops correspond to real locations where players can go to stock up on pokeballs, potions to heal their pocket monsters, and gyms where players duke it out to control the area. Due to the game’s dependency on data uploaded by players, urban regions tended to have more gyms, PokeStops, and Pokemon than rural areas, where landmarks could be few and far between.

"In Ingress, we have an online tool where players can submit and rate each other's location, photos, and description about that place, such as how interesting is it culturally,” Hwang explained. “I'm optimistic that, over time, we'll be able to grow that data so that rural areas can also have a rich gameplay experience."

A more serious drawback to Pokemon Go’s widespread popularity can’t be laid squarely at Niantic’s doorstep. More focused on their mobile devices instead of their surroundings, players have caused auto accidents, wandered into oncoming vehicles, and even walked off cliffs. "We didn't quite anticipate the explosion of player activity that Pokemon Go triggered. We were making adjustments,” Hwang confessed. “You saw the game added speed limit detection to warn players to pay attention to their surroundings. Those are really important changes, and we're going to continue to remind players about safety."

Another consequence, albeit one not as ripe for disaster, had to do with famous attractions and private residences matching up with in-game landmarks. Arlington Cemetery and the Auschwitz concentration camp in Poland lodged complaints about Pokemon Go players disrespecting the sanctity of their premises by passing through to hunt for creatures. At the Holocaust Museum in Washington, D.C., a Koffing Pokemon that emits poisonous gas materialized in an auditorium where visitors watched films about survivors of gas chambers. The pocket monster’s appearance was total coincidence, but still distracting to guests.

Delisting a real-world location as an in-game marker can only be done by submitting a request to Niantic, which holds the right to refuse or comply. The company acted quickly to remove or relocate gyms and other landmarks from locations considered inappropriate for gameplay. On some occasions, the studio has worked with representatives of a business, city, or country to host Pokemon Go events in neutral locations—places where players can enjoy the game without disrupting monuments.

"For example, in Japan, the local government has coordinated with us. They'll do these spawning events to move people throughout their cities where they wanted more tourism,” Hwang said.

Niantic weighs decisions such as delisting areas carefully, aware that every move it makes will make for watercooler talk around the globe due to Pokemon Go’s overnight success. "We put in a lot of time and effort on the graphic design specifically,” Hwang said, “but generally, the game got so much attention that anything we put out was scrutinized and analyzed. It was very natural to feel like once we put out something, it's anchored and we can't change it ever again. But I think it's very important, especially with mobile games, for devs to resist that impulse. It is this malleable thing where if something's not working well, you have to be able to go in and do the surgery to change it."

Pokemon Going Forward

The success of Pokemon Go is nothing short of staggering. Within months of release, the game eclipsed downloads and sales of previous top sellers such as Candy Crush to hold the number-one spot on the iTunes App Store. It was the first app to reach 650 million downloads in record time, and although it’s no longer as popular as it was during the summer of 2016, Niantic intends to mete out content and give players reasons to return.

Daily bonuses and a buddy system are just some of the updates Niantic has made to Pokemon Go. More updates, such as a player-versus-player mode that sees Pokemon square off in battles modeled after combat in the main games on Nintendo’s handhelds, are slated for release in the future.

"The technology is only going to get better,” Hwang said at the conclusion of his talk. “We're pretty excited about what's next."

David Craddock

Long Reads Editor

David L. Craddock writes fiction, nonfiction, and grocery lists. He is the author of the Stay Awhile and Listen series, and the Gairden Chronicles series of fantasy novels for young adults. Outside of writing, he enjoys playing Mario, Zelda, and Dark Souls games, and will be happy to discuss at length the myriad reasons why Dark Souls 2 is the best in the series. Follow him online at davidlcraddock.com and @davidlcraddock.