This weekend I served as a judge for Come Out and Play, an experimental urban games festival held in a different location around New York every year. During the course of the festival, I had an experience that I think might be worth describing for developers thinking about the native apps vs. web apps debate. It was an experience that confirmed everything that’s lead me to want to learn to build iPhone apps rather than sticking with web apps for mobile development.
One of the games, called Pathfindr, was billed as “The Amazing Race” meets “Foursquare.” They said in advance that it required an iPhone or Android phone to play. They’d built a web app that used javascript APIs to access the phone’s location, then they used that to check if you were close enough to the series of checkpoints and questions that made up the game terrain. Your typical treasure hunt.
The experience of trying to play this game pointed out everything that is inferior about the web app experience vs. the native experience. And not as minor technical annoyances, but in a way that completely ruined the experience of the game.
(Note: none of what follows is meant as a criticism of Pathfindr’s developers. I think they are diligent individuals who’s only mistake was naively believing all of the hype around web apps as a solution for mobile development. In fact the game works so well as a case study because they seem to have done everything exactly how the current conventional wisdom would have them do it.)
The app consisted of a series of menus, styled to resemble native iPhone UI, through which you signed up for the site with an email address and password and chose which game iteration you wanted to join or resume (they’d run Pathfindr once before in Minneapolis as a tour of notable locations in the life of F. Scott Fitzgerald). Once you reached a game, the play took place on a satellite map where a set of markers appeared noting three checkpoints that you had to visit in turn and a series of questions you had to answer along the way about things in the local environment. In order to eliminate a checkpoint or answer one of the questions, you had to get within a certain distance of the marker on the map and then tap that marker. The map also displayed the blue iPhone-style “you are here” indicator. The game took us an hour and a half to complete and stretched from the Brooklyn Lyceum, where the festival was based, all the way to Grand Army Plaza at the entrance of Prospect Park.
Here is a rundown of the technical problems we encountered in the course of the game:
The web app was slow. Pages took a long time to load from the server with each new question or with new map tiles when you tried to move around the map. This was not caused by the rate of the iPhone 3G connection (other pages, such as Google, loaded perfectly rapidly), but because of the 20 or 30 people simultaneously accessing the site over-and-over. The problem was worst when all the teams were standing together and trying to register at the start of the game and was slightly mitigated over the course of 90 minutes as we spread out around the course.
The app was unresponsive. Dragging and zooming around the map was sludgy and painful. Redraws took forever.
The GPS location was jumpy and inaccurate. Far worse than I’ve ever seen it in the iPhone Maps application or any native iPhone app that uses location.
The app sucked battery life. In an hour and a half of playing it killed one phone completely dead from about a 30% charge and was close to killing a second one.
The app’s map UI was non-standard and confusing. It used satellite-only tiles which made navigation difficult; accidentally tapping the blue “you are here” dot would zoom the map all the way out, which was extremely annoying; it had extra chrome within Mobile Safari’s chrome leaving only tiniest number of pixels available for the actual map.
At one point, someone in our team accidentally closed the app’s tab while trying to do a google search as part of the game. We had to suffer through lost state within the game that required four or five page loads to get back to where we were and hence about 5 minutes of sitting still with 5 people all hovering around a phone.
Overall, the sludginess of the web app experience meant that, in a game designed to take us out into the neighborhood, make us more attentive to the landmarks around us, and force us to collaborate with a group of strangers, we spent a large amount of the time staring at the phone’s tiny screen, waiting for pages to load. The technology’s flaws overcame the game’s virtues and severely damaged the experience.
I’m sure mobile web app performance experts could explain a number of ways in which Pathfindr’s developers could have improved their app to avoid some of the problems I mention here. However, one of the main talking points for mobile web apps is always that developers “already know how to make them”. And talking to the one of the developers after the game, he touted the fact that they’d been able to build the first version of the game platform in only a few days’ development time. If there are a bunch of new skills that an average web developer will have to learn to build an effective mobile web app, then that goes a long way to mitigating the advantages of their existing experience and it makes the argument for web apps seem a lot more like an argument for familiarity rather than ease of use.
Again, the Pathfindr team was professional and polished. Their app had obviously been the subject of a great deal of loving care. Give them the benefit of the doubt. The problems with the app didn’t derive from their incompetence, but from current limits in mobile browser technology in this kind of situation.
Also, as someone who’s just started learning iPhone development, there was nothing in the game that was beyond the level of the first four or five chapters of a good intro iPhone development book, for example Big Nerd Ranch’s iPhone Programming Guide, which I’m currently in the process of working through.
In less than a weeks’ time, a dedicated developer just getting started on the iPhone could have built a native app with much the same functionality as Pathfindr’s web app that would not have suffered from any of the problems I describe above. And for an experienced iPhone dev, the process would have been more like the one or two days Pathfindr themselves took.
I can’t speak for Android, but at least in the case of iPhone development, as a newbie iPhone dev and a highly experienced web dev, I think the idea that native development is so much more difficult than web development is largely FUD. While native development means learning Java or Objective-C and getting up to speed with someone else’s large API, there’s no server to maintain on an ongoing basis, there’s only one language rather than the three or four that are typical for a web application (HTML, CSS, Javascript, and at least one server-side language), and there are no scaling issues to consider with the acquisition of multiple users. The “ease of use” factor with web apps exists solely for a developer community that already knows how to make web apps. For new developers or anyone willing to learn something new, I think the level of difficult argument is actually a lot more evenly balanced than it might appear from the outside.
What about device independence? In the case of Pathfindr’s audience at Come Out and Play, at least, nearly every single participant used an iPhone. There may have been one or two Android players, but I didn’t see them. They’d advertised the game as “smart phone-only” and said they’d tested it on Android, iPhone, and Palm Pre Plus. A few people with Blackberries came thinking they might qualify, but had to be turned away or added onto teams of players with iPhones. With the incredible battery usage my team saw, I can’t imagine that an Android team would even have been able to finish the course (especially with what I’ve heard about the HTC 4G).
The result on the ground, then, was an application that was the worst of both worlds: device-limited without the benefits of device-specific design. It had all the performance disadvantages of a web app that I’ve outlined above with none of the imagined benefits of cross-device compatibility.
Advocates of web applications as the solution for mobile development need to do some observations of real world scenarios like this before they convince more developers that native apps are unnecessary or evil in some way. The new creative and surprising uses to which a big games festival like Come Out and Play puts mobile apps is a terrific proving ground for development platforms.
For my part, I want mobile apps to be weirder and more fun than their desktop ancestors and to explore all the new affordances offered by the position of phones and new devices in urban space and in our lives. And I’ll take whatever development platform keeps the battery running, the GPS coordinates flowing, and the sensors sensing. Right now that means native apps.
Hi Greg,
Once again I really can’t disagree with your technical observations (other than where you claim that a native app would remove the need for a game server), but the reason I feel compelled to comment now is that you seemed to have missed a word that I’ve consistently been using throughout our conversations- “prototype.” The version of the game you played was a prototype for the forthcoming native experiences.
Why did we bring a prototype to Come Out & Play?
We did this because we’re planning to build a bigger platform, but we needed user validation w/ a focused audience sample to give us an idea of what to keep as is, what to build on, and what to leave behind. We had six teams complete the course (1 didn’t check-in to the final check point), and we had great conversations with each of them about what they liked and what needed work. Based on those conversations and a few other observations, we were able to refine our Feature Roadmap and have begun work on the native experience you say that we so desperately need.
Why did we prototype in HTML, Javascript, & CSS?
The first rule of prototyping is build it quickly. The second rule of prototyping is that once you learn what you need to learn from the prototype, throw it away. Had we prototyped in Ruby or as a native app, the temptation would have been to recycle some code. Sometimes recycling code goes alright, but I’ve found the best results for a product come when you force yourself to trash the prototype and start over.
…a few other things I’ll add is that the game’s description & sign-up form clearly stated that this was an iPhone experience. It was playable on Android devices with browser geo-location, but it wasn’t the experience’s intended context.
Furthermore, and keeping in mind that I do respect and value your obviously informed opinion, I’d ask you to examine your own bias behind the statement of “it ruin the experience.” 5 teams finished ahead of you. 4 of those teams were placed squarely in the “would play again without modification” category and the 5th was a “would play again with slight modification.” I realize and acknowledge that more than 6 teams started the game, but user abandonment is bound to happen and it was likely do to the experience of playing a prototype as opposed to any of the foundation tenants of the gameplay.
My final point is that what you have labeled as “naive belief in the hype around web apps” is actually part of a disciplined and iterative design process. However I am happy to have provided you with a soapbox upon which to broadcast an argument you seem to have been formulating for some time now.
Dakota —
I appreciate your comments and, like I said in the post, none of my criticisms were aimed at you guys in particular. I could tell from the design polish on the app that it was part of a well-considered professional development process. And I can definitely appreciate the value of rapid prototyping and user testing.
However, the point I was trying to make was that the techncal limitations imposed by the use of a web app in this particular environment meant that, at least for my group, we ended up testing the technology more than the gameplay. The experience for us ended up being about waiting for page loads and wrestling with map UI rather than racing.
If you wanted to build a really simple and cheap prototype I might have used paper maps and envelopes or some other non-technical solution that would have been even easier to fabricate and would have kept the focus entirely on the game.
I hope that you don’t take my criticism too personally as that’s not the way it was intended. In regards to me using your game as a “soapbox” for an argument I’ve been long formulating, I’ll just tell you a little about myself (this being my blog many other readers might know this already, but you don’t seem to have clicked around much). I was a web developer for years building my own startups as well as applications for other people in PHP and then Ruby on Rails for five or six years before leaving that world to do more art- and physical computing-driven work at NYU ITP last year.
In other words I was a web evangelist (zealot, even) for a long time but have gradually, over the last year, come to realize that when it comes to some of the really weird interesting creative applications that we want to build for mobile devices, applications that treat mobile devices more as programmable pieces of mobile hardware than as generic user agents, native apps have a lot of advantages. To the degree to which your application is user agent independent, web tehnologies are superior. The more hardware requirements you have in mind, the more you need a native app. When you begin treating the device like a package of sensors and outputs rather than a generic user agent the web app approach stops being viable as a prototypin tehnique and starts being simply the wrong paradigm.
Again, I’m not disagreeing with you. In fact if you take a look at my track record, you’ll see that I’ve basically written the same words:
B&A Article: http://tinyurl.com/ykjvdl3
GaTech Masters Thesis: http://tinyurl.com/35cxs5f
What I guess I am taken back by is the fervor by which you wish to apply a technical criticism to a prototype esp. since I am fairly certain that IPT shares a similar outlook on prototyping as Georgia Tech (where I did my Masters) does. I feel bad that you didn’t have the same enjoyable experience as some of the other groups did, but that happens.
We did paper prototype earlier in the process, but that obviously doesn’t allow us to test things like what to expect in terms of Dilution of Precision for the area we were targeting and whether or not the way we were thinking of organizing games made sense in terms of menu interactions (which it didn’t, and is super good to know prior to doing UX on the go-to-market product).