Russell Carroll met Lisa in a parking lot where they were waiting to pick up their kids from an activity. Unbeknownst to Lisa, she was one of Carroll's biggest fans.
They struck up a conversation to pass the time and, as American adults are prone to do, turned the topic to work. Carroll told his new friend that he made video games. Lisa's expression changed from a friendly smile to one of confusion and not a little fright. "Well, I don't play games," she told him. His next words made her change her tune.
"Then she found out I made Solitaire, and she was ecstatic," recalled Carroll, an executive producer at MobilityWare, during his talk at this year's Game Developers Conference. "She got so excited. She's like, 'I love Solitaire!' She went on and on about how much she enjoys it, specifically a feature called daily challenges."
MobilityWare's Solitaire launched in 2008. Daily challenges are a recent addition designed to give skilled players more ways to play. Players who complete a day's challenge cross that date off their calendars. Meeting a certain number of goals earns them a trophy, and conquering every challenge in a month gets them an even bigger trophy.
After gushing about daily challenges, Lisa gave Carroll a stern look. "Don't change a thing," she told him.
"This is an irony, because if we hadn't changed anything, the feature she loved wouldn't have existed. We would have never made it to begin with," Carroll said.
Testing, Testing
Solitaire is no mere card game. It's an institution. Every day tens of millions of players log approximately 270 years' worth of play time across dozens of versions. MobilityWare claims the lion's share of those users. The company's suite of card games--FreeCell, Pyramid, Spider, Tri Peaks, and Klondike, also known simply as Solitaire--have been downloaded over 200 million times across Android and iOS platforms. At one point, Solitaire was the second-most downloaded game on iOS, just behind Candy Crush. When Pokemon Go mania swept the world during the summer of 2016, Solitaire slipped only one spot.
"So these games may not be quite as flashy as others you've heard about, but we have massive numbers of people playing our game, which is a great opportunity, and also presents some challenges," Carroll said.
On mobile phones and tablets, on desktops and notebooks, in office buildings and homes and apartments, solitaire players are just like Lisa: they know the game, they love the game, and they don't want anyone tampering with its finely tuned formula.
Neither did MobilityWare, at least at first. "Solitaire's already been developed. It's been around for a long time. When I was getting hired, I was told what I would be doing, and that was to add features to solitaire, add users to solitaire, and don't mess up the revenue. Yeah. A little bit of fear, there," Carroll said, chuckling.
Carroll joined MobilityWare as an executive producer in July 2016. His purview extends across several development teams spread over three production environments. All totaled, the studio owns 16 versions of solitaire.
"We had 40 updates across the suite last year, we're going to have more than that this year. Some of those updates are small, but I try really hard to have a feature in every single update," he said.
One of MobilityWare's challenges is figuring out how to float updates to their users. Carroll shepherds his team's ideas to fruition by shielding users from widespread changes. As an example, the studio rolled out a number of progression-based systems within FreeCell during 2016. Progression encompasses experience points earned by knocking off daily challenges, and completing daily goals ranked easy, medium, or hard. As players earn XP, they level up. With levels come titles good for bragging rights and jockeying on leaderboards.
To get a feel for what their users thought of progression, MobilityWare asked them to complete an in-game Likert survey--the type that makes statements rather than asks questions, and ask users to choose between five responses such as Strongly Disagree, Neither Disagree or Disagree, and Agree. MobilityWare ran the survey for two weeks and got thousands of responses.
"When we get a survey response back, they're very honest," Carroll admitted. "They're brutally honest. They'll tell us exactly what they think and how much they hate us. This is good. We get more information--in fact, six times more information. When we ran this particular survey, we got six times as many people who responded this survey as who wrote in reviews."
Running AB tests, where users evaluate two or more executions of an idea, is another way MobilityWare looks out for its customers. When managers observed a downward trend in game installs, managers asked for feedback. The reason was that the font of cards was too difficult to read.
"Changing cards in solitaire is super hard," Carroll explained. "When I joined MobilityWare and we talked about changing cards, I had people start shaking their heads and saying, 'You cannot change cards in solitaire.' Not that we shouldn't; we couldn't. It could not be done."
The studio conducted an AB test with two groups of players. One group was given cards crafted using the previous font, while the second group's cards were rendered in the newer, harder-to-read style. Only new players participated, and on Android platforms, which has fewer users than Apple's iOS systems. Those two parameters enabled MobilityWare to protect their users even further by containing the reach of the AB test.
Results were recorded over a four-week period. During that time, Carroll observed that users playing on phones liked the new font--it made card faces easier to read on smaller screens--but players on tablet devices found it distracting.
"What we ended up doing was acting on what we felt really good about, which was that we knew on phones that this font was making players stick around longer," he said. "It was retaining users better, so we made it the default on phones and made it available to all our existing players as well. We let them know it was there so they could choose to use it."
Succeeding By Failing
Within minutes of joining the company, Carroll received advice from all corners of MobilityWare. Their feedback was the same: be careful when proposing changes to Solitaire to management. Besides looking out for their players, company brass expected their producers to protect their revenue stream.
"Those types of stories end up becoming culture," he explained. "They communicate to new employees how you interact with your games, and what you can do at the company. In this case, what we were communicating was: making the smallest, most minute change was not a good idea. Don't make changes."
Carroll's solution was to prioritize developers and players. If company culture encouraged developers to experiment with opportunities to grow products, and if players could be introduced to those changes slowly and their feedback taken into account--such as by conducting surveys and AB tests--revenue was bound to increase.
"I mentioned we have five games that are solitaire games," Carroll said. "We can test a new feature in a smaller title before we roll it into a bigger one. We're not unique in this. King [Candy Crush] does this, Supercell [Clash of Clans] does this. It's a very common thing to do."
As an example, Carroll pointed to a crash bug that popped up shortly after he joined. The team was tired and looking forward to MobilityWare's upcoming trip to Hawaii. As luck would have it, one of the games kept crashing. The team hesitated. They had already submitted a build of the game to Apple. Apple would reject it, but that was fine; they could deal with it after they got back from a much-needed vacation.
The more the developers poked and prodded, however, the more they came to the conclusion that the real problem was that one of their new features simply had not been implemented well. To solve the crash bug, they would need to tear the feature out by the roots.
Carroll described the way the team dealt with that problem as a defining moment in their culture. It would have been easy for developers to point fingers and blame peers. Instead, they met and decided to succeed or fail as a team. "I believe that as we try new things, we should expect that we won't get them right all the time," he explained. "We should have confidence in our developers. We should have confidence in each other. Instead of having people fearful of making mistakes--which, by the way, leads to more mistakes--we should have people who are focused on getting it right, trying new things, and making things awesome."
On a separate occasion, while introducing progression systems, Carroll's team changed the high-score button from a plain star to a long, ovular button wrapped around a star with a number beside it. The number stood for the player's ranking, but the change in button appearance confused and frustrated players used to the old implementation. The team was inundated with complaints and made the call to pivot.
"This was an easy change. We changed it to be a big star button and put an overlay to say, 'Hey, here's the button,'" Carroll said.
Recent feedback on progression has been more positive. Like Lisa, the brunt of users praised daily challenges in particular and expressed a desire for more--especially helpful feedback, as it paved a clear path for MobilityWare to follow when brainstorming future updates. One reviewer admitted that it took time for them to get used to the changes in Free Cell, but once they did, they felt "ok" about it.
Although hardly a glowing review, Carroll was fine with that lukewarm feedback. "The main reason why is because as I look at it, we got to 90 percent of customers liking what we were doing. Ninety percent liking plus 10 percent being 'ok' with what we're doing--that's 100 percent good."
In tense moments, such as champing at the bit to vacate the office and hop a plane to Hawaii, the team could have taken the easy way out and let sleeping cards lie instead of attempting to alter even the smallest strand of the game's DNA. So far, their gumption has paid off.
As Carroll pointed out, there's no place for milquetoast developers or same-old products, especially in the mobile market. "In our industry, hundreds of games came out on mobile today. There are thousands of games coming out every month. You cannot afford to be mediocre. You cannot afford to play it safe and be just like all the other titles. There are thousands of failing, mediocre, safe titles. They're failing every day. You've got to stand out. You've got to be willing to make risks that will help you stand out."
-
Shacknews
replyDavid Craddock posted a new article, GDC 2017: MobilityWare's Russell Carroll on the Challenges of Evolving Solitaire
