Archives for : March2012

Times Change, People Change, Hairstyles Change… Security Changes

I’m getting a lot of hits on Sunday’s App Rejections Are A Lousy Way to Communicate Policy Changes, courtesy of a link from Daring Fireball, which is pretty magnanimous considering I kind of slam Gruber halfway through. That blog argued that suddenly rejecting apps for accessing the UDID is a terrible way to handle a policy change, that Apple’s usually far better about controlling the message, and that developers shouldn’t hear about this stuff from Twitter and TechCrunch.

Unfortunately, a lot of the conversation here and elsewhere has been hijacked by a focus on the UDID specifically and privacy issues in general, largely by uninformed readers who are quick to blame everyone who didn’t see this coming.

I’m going to use this blog to explain what the UDID is and the problems around it, from the POV of a developer who’s both chosen to use it and inherited code that uses it.

Take this with the grain of salt that I Am Not A Security Expert (IANASE), or perhaps I Am Not Graham Lee (IANGL).

What Is the UDID?

The Unique Device Identifier is exactly what it says on the tin: a long string of characters that uniquely identifies one iOS device. It only identifies the device, not the person using it (I believe it stays the same after a device wipe, but I don’t feel like wiping my iPad to test that). Any user can find his or her UDID by connecting their device to iTunes and option-clicking the Serial Number.

Since the first public SDK in iPhone OS 2.0, the UDID has been available to developers by calling -[UIDevice uniqueIdentifier]. The reason you would want to do so is typically to identify one instance of your app running in the wild, or perhaps as a stub to create other unique IDs (for example, a document-creating app that IDs each document as UDID plus the time it was created).

Imagine you want to know how often your users use your app. You could set up a URL to log startups, and then hit it from your app. But if you got 30 hits, you wouldn’t know if that was 30 devices running your app once, or one device running it 30 times. If each call sends the UDID, then you’d be able to tell.

So What’s The Problem

Most of us would agree that this scenario is pretty benign. And in fact, many apps gather a lot more metrics than this — what features are used most, which ones are used together, etc.

Let’s set aside for the moment of whether gathering usage metrics is OK (with or without the user’s permission). So far, this doesn’t seem bad. In fact, we still don’t know who’s running the app — no matter how much information we collect on a given UDID, all we know is that one device exists that is being used in that way.

Let’s say we have 9 apps that collect metrics like this, each phoning home metrics to the developer or some third-party’s server. Now what if a 10th app does the same thing, but it for some reason is able to collect personal information (maybe it’s subscription based, maybe it uses a Facebook login, whatever). That app is clearly able to correlate metrics to a given user. But since it shares the same UDID with the other 9 apps, we can now associate the activities in those apps with a specific individual.

Can you say “unintended consequences”? This is, as I understand it, the problem with the UDID, and why Apple has deprecated its use.

They Should Have Known!

No, the issue is that no one should have thought it was ok to use information that could identify or otherwise compromise another person’s privacy. It should have been obvious that once this got out Apple would need to do something about it. It should have been obvious that you should not have been doing it at all.

Icouldseeitcoming, commenting on App Rejections Are A Lousy Way To Communicate Policy Changes

My point in the above story is that the potential for misuse of the UDID does not come from the string itself. Remember, it conveys no personally identifying information. The problem has arisen over time, in an unexpected way, based on many apps working together (not necessarily with their explicit intent or permission).

Was Apple negligent in offering access to the string in the first place? All indications are that they actually gave privacy some serious thought as they developed the iPhone SDK. Consider the Address Book. No, not the Path debacle (we’ll get to that). Over on Mac OS X, there is a method called -[ABAddressBook me] (also the C function ABGetMe()) that returns the user’s address book card. In the C-based iPhone Address Book API, there is no equivalent call to get the user’s record. Clearly, Apple did put some thought into what third-party developers should and shouldn’t have access to, and decided that being able to personally identify the user of the phone was a bad idea. Despite benign uses this prevents, many of us would consider this a good decision.

So given that, let’s consider this absolutist position that “no one” should have thought it OK to use the UDID. Let’s just use Occam’s Razor: do we really believe that Apple, all the ad networks and analytics companies, and the majority of iOS developers are just stupid and negligent? That’s a huge pill to swallow. The alternative hypothesis — that the sense of what is and isn’t acceptable privacy policy has changed over the last four years, particularly in light of how things have actually played out among 700,000 apps — is far more plausible.

To that end, let me tell you another story…


Apple’s guidance in the iOS 5 deprecation statement for the -[UIDevice uniqueIdentifier] call is to generate a CFUUID and persist that in the NSUserDefaults. I’d be inclined to use the Keychain so it survives app deletion and reinstalls, but same difference. The upshot is, the one app that just needs to log app launches still has what it needs: a unique identifier of one instance of the app. When each app creates its own CFUUID, the 10 apps in our above example phone home with 10 different unique identifiers, so one can’t be used to compromise the identity of another. So far, so good.

So what is a CFUUID? It’s Apple’s C API for working with Universally Unique Identifiers (UUIDs). From Apple’s docs:

UUIDs (Universally Unique Identifiers), also known as GUIDs (Globally Unique Identifiers) or IIDs (Interface Identifiers), are 128-bit values guaranteed to be unique. A UUID is made unique over both space and time by combining a value unique to the computer on which it was generated—usually the Ethernet hardware address—and a value representing the number of 100-nanosecond intervals since October 15, 1582 at 00:00:00.

So that’s good, we can’t trace the IDs back to the… hey, wait a minute, what was that? Combining the network hardware address (aka, the MAC address) with the time the UUID was created? Doesn’t that mean that a group of UUIDs created on the same device would all have the same MAC address? So if you can get that MAC address out of the UUID, then even though the 10 apps that phone home 10 different UUIDs, we could get the MAC address that’s common to all of them, identify the user and we’re right back where we started! What the hell? Rabble! Rabble!

Well, as it turns out, Apple’s documentation is wrong. Look back at the Wikipedia entry again and notice that UUIDs have versions. The description above is for version 1, which was abandoned for exactly this reason:

This scheme has been criticized in that it is not sufficiently “opaque”; it reveals both the identity of the computer that generated the UUID and the time at which it did so.

Read in and we find that there is a version digit in the UUID, the character after the second hyphen. Now click to enlarge the screenshot below, from a sample app I showed at CocoaConf Chicago, and which generates hundreds or thousands of UUIDs a second in a (futile) search for duplicates:

UUID Starfield app - shows that iOS/OSX UUIDs are all version 4 and never duplicate

Notice that the digit after the hyphen is always a 4, meaning these are version 4 UUIDs, which are just really huge pseudo-random numbers, and therefore don’t reveal anything about who or what created them. Problem solved.

The reason I bring this up is that it was not at all “obvious” that putting the MAC address in the UUID was a bad idea, at least not so bad that it held up ratification of the standard (or, alternately, it wasn’t a problem until UUIDs started being used for purposes where revealing the creator’s identity was an issue). Lots of smart people worked on this stuff; it wasn’t thought to be a problem until later.

Like bugs, security problems are not things people create on purpose, and it’s insulting to insinuate that. They show up later, after reconsideration, after systems evolve, after third-party attacks.

You Want Evil? I’ll Show You Evil!

Yeah, you guys are right. How could anyone have expected that something called a “unique device identifier” might be used to track people.

Icouldseeitcoming, commenting on App Rejections Are A Lousy Way To Communicate Policy Changes

The UDID is neither necessary nor sufficient to track users. An app could use a hand-rolled UUID to track a device, as is Apple’s recommendation, and is in a position to log every tap and keystroke and phone it home to the analytics server. Heck, I had an AV Foundation demo a few years ago that encoded screen-grabs to a QuickTime movie — it wouldn’t take much more to provide a live video stream of a user’s interaction with my app back to an IP address of my choice, all without the user’s knowledge or assent.

That’s assuming I’m being evil on purpose of course. What removing the UDID hopes to improve is cases where apps inadvertently provide a way to correlate activity in different apps, which in turn could be linked to a real person if any of the apps capture personally-identifying information. Even now, there are other ways this could be done: apps could share hand-rolled UUIDs via URLs, document exchange, or the Keychain (if the apps were all signed with the same credentials and share a bundle id stub), though this requires a far greater degree of deliberate cooperation than the inadvertent UDID case.

There are other APIs with privacy implications too, such as free access to the Address Book (as epitomized by the Path case a few weeks back, and Facebook years earlier). An app that incorporates a UIWebView could also be vulnerable to any known Safari or JavaScript attack vectors. And whole apps are based around using Core Location to rat out your current location.

So to make a big deal out of the UDID as this obvious privacy problem seems badly ignorant. Its problematic aspects come from unintended consequences, not the nature of what it is.

And since any app can itself collect any information on your interactions with it, UDID or not, and phone it home with a network connection, the only perfect way to avoid being tracked at all is to go to the Settings application, turn on Airplane Mode, and never turn it off.

App Rejections Are a Lousy Way to Communicate Policy Changes

Following Twitter reports from earlier in the week, MacRumors and TechCrunch now report that Apple is rejecting apps that use the unique device identifier (UDID), by means of the -[UIDevice uniqueIdentifier] call.

This in itself is not surprising: iOS 5 deprecates the method, adding “Special Considerations” that advise developers to create a one-off CFUUID to identify a single install of the app on a single device.

So far, so good. I even had a section of my talk at last week’s CocoaConf Chicago that talked about CFUUIDs and the need to migrate any UDID dependencies to them now.

The problem is the timing. Apple’s established pattern has been to deprecate a function or method in one major version and, at the speediest, remove the call in the next major version. Many developers, myself included, expected that we had until iOS 6 to get off of the UDID.

But instead, without warning, the app review process is being used as an immediate death-penalty for the uniqueIdentifier call.

This is a problem because we’ve all had about six months to get off of UDID, and while that’s surely enough to get a simple app migrated — indeed, I have cases where switching it out is a 5-line fix — it is not necessarily the case that everyone can be expected to have already done this.

The real problem isn’t with developers; it’s with whom we develop apps for. Our clients don’t know or care what a UDID is, nor are they aware of a single line in Apple’s documentation saying “stop using this”. Sure, it’s our job to be on top of it. But let’s imagine apps with long development cycles — big apps, or academic apps that rev for the new school year in the Fall and are largely dormant the rest of the year. It’s entirely plausible and reasonable that developers of these apps have “get off UDID” as a high-priority item in their bug trackers, but are waiting for budget and approval to start working. And what if it’s not a simple process? What if an app has some deep dependency on access to the UDID, both in the app and on a server somewhere, meaning that two different teams are going to need to deal with losing access to uniqueIdentifier, and will need to come up with a plan to migrate user records over to a new id scheme?

Well, they just lost their chance.

Captain Hindsight is in full effect in the MacRumors forums, loudly asserting that developers should have known this was coming, have had plenty of time, etc. I get that it’s the natural defensiveness about Apple, but it gets worse… because this isn’t the only case of Apple using app rejections to carry out policy changes.

Thanks perhaps to my many posts about in-app purchase, I recently heard from a group of developers who’d gotten a galling rejection. They have an app with a subscription model, and used the new “auto-renewing subscription” product. This product is far superior to the original subscriptions that I have repeatedly described as broken, as they cannot restore between a user’s devices, and do not carry state to indicate what was subscribed to and when. Auto-renewing subscriptions fix these problems, and the In-App Purchase and iTunes Connect guides had (seemingly until a couple weeks ago), clearly disparaged use of the old subscriptions in favor of the new auto-renewing subscriptions.

So imagine the surprise of my colleagues when their app was rejected for using auto-renewing subscriptions. The reason given was that they were using it for a different business plan like a data-plan model, and auto-renewing subscriptions are, according to the reviewer, reserved only for content subscriptions like Newsstand magazines. I have never seen anything to this effect in any Apple I-AP documentation. Nevertheless, the developers had to switch to the shitty, broken, old subscriptions.

In both of these cases, we see Apple breaking with their own documentation or with long-established practice with no warning, and instead using app rejections as a tool to communicate and carry out new policies. This is wretched for developers, who get caught scrambling to fix problems they didn’t know they had (or didn’t expect just yet).

It’s also terrible for Apple, because the aggrieved developers initially control the message as they flock to blogs and Twitter, leaving it to loyalist commenters and bloggers like Gruber and the Macalope to mount a rear-guard gainsaying defense. To see Apple — of all companies! — not controlling the message is astounding.

All it takes is clarity. If they’re going to make such a major change, they’ve already got our attention via e-mails, the developer portal, and many other channels. They could and should clearly state the what, why, when, and how of policy changes. “We’re getting rid of UDIDs, because they constitute a privacy risk. We’ll reject any app that calls -[UIDevice uniqueIdentifier] as of March 23, 2012.” Not that hard. They’ve done it before: a few years back, Apple required streaming media apps to use HTTP Live Streaming if they streamed more than 10MB of data — this was communicated via announcements on the developer portal a month or so before its implementation, and nobody got caught by surprise, nobody complained.

Apple has developed a reputation for capriciousness in its app review process, and a cavalier attitude towards its developer partners. It’s not undeserved. As Erica Sadun once cleverly put it, “Apple is our abusive boyfriend.”

Facebook for iOS Pigs Out

Last night, I was thwarted from adding a show to my iPad because I was nearly out of space:

iTunes usage: iPad nearly full

I started deleting large, rarely-used apps (goodbye for now, GarageBand!), and ultimately trimmed the playlist of songs that syncs with the iPad. But as I looked through my per-app usage, this caught my eye:

iOS Facebook using 421 MB

That’s right: Facebook, a network-based application (one that is typically accessed as a web page) is using nearly half a gigabyte for documents.

No. No you don’t. I deleted Facebook and reinstalled from the App Store to reclaim the space. But then I thought, what the hell is it doing with all that space?

Fortunately, I still had Facebook on my iPhone, where it’s only burning a little over 100 MB. I took a look with iExplorer, and dumped Facebook to my Mac:

Contents of Facebook for iOS app, including 100 MB of cache

So look at that: 12 MB for the app, over 100 MB of Caches, nearly all of it in FBURLCache. Bad enough that it burns a seemingly unlimited amount of space, but let’s flip back to iExplorer and look at the file dates:

FBURLCache file dates

Yep, the oldest files in this cache are three months old. What are the odds that I’m going to want any of those files, and even if I do, that I’m going to get any benefit from caching them locally?

It gets better. After poking around the 1.5 MB cacheinfo.plist index file and its 6,000 entries, I dug into one of the cache files with HexEdit, which immediately revealed it as a simple .plist. Off to the Property List Editor we go!

Facebook cache file property list

So, great… for a “Recommend” button on some web page I accessed back in January, Facebook has cached the entire NSHTTPURLResponse. 1,700 bytes of cache for a 339-byte GIF. And apparently it’s done this for every element of every web page I’ve viewed with the Facebook app.

Unbelievable. I rarely view a web page more than once through the FB app, and it’s highly doubtful anyone is getting more benefit from caching these files to be worth the space they’re consuming. It seems there may be a 3-month roll-off (or maybe that’s when I last reinstalled the app?), but that’s a uselessly long period: it’s rare you can roll back to posts that old on Facebook. And I’m left to assume that there’s no limit to the size this cache is allowed to grow to. At over 400 MB, the cache on my iPad was nearly the size of two standard-def TV shows… something that would do me a lot more good than caches of links I’ll never view again.

Should there be an option to turn the cache off, or at least to cap its size or or set its expiration date? Of course there should. But given that this app is the same one that giddily uploaded all the entries in my contacts list years ago, I probably shouldn’t expect better.

What You Missed at CocoaConf Chicago (2012)

OK, here’s my usual post-conference blog entry with slides and sample code links. In a break with tradition, I’m actually getting it up the day after I got home from CocoaConf, rather than weeks later.

“Core Audio Cranks It Up” is a talk I first gave at an earlier conference, so the links to slides and sample code are in What You Missed At Voices That Matter iOS, Fall 2011.

New for CocoaConf, I did a talk called Core What?, which started from Twitter discussions with potential attendees about topics including a grab-bag of Core Foundation and other C-level oddities and novelties, and all the gruesome PDF lessons I learned late last year.

Core What?

For what it’s worth, this is another conference presentation I created entirely on my iPad. This presentation also includes an off-the-cuff reference to Bodacious Space Pirates (although one stage of the build was lost in export to PDF). You’re welcome.

We also did a “Reverse Q&A” panel, in the style introduced by Harmonix’s PAX East 2011 Reverse Q&A. This made it a lot easier to get the discussion rolling quickly, and featured much more insight from the audience than you’d get from a typical panel format. Feedback on Twitter has been positive, including from CocoaConf’s itself, which vows to use the format again in the future. But maybe next time I’ll stick to the handheld mic that I shared with audience members, because I think my super-sensitive lavalier was the source of the feedback whenever I was under a speaker.

Finally, like any good trip, this one included trips to Fry’s Electronics and Ted’s Montana Grill. Because if there are two things I need more of in my life, it’s computer parts and bison-burgers.

Point and Laugh: The End of WebM

As hinted last week, Mozilla has finally seen the writing on the wall and decided to support H.264 in the <video> tag in Firefox. Probably inevitable given that

  1. Mozilla is making a new push into mobile
  2. The default standard for web video has become H.264 <video> for mobile and Flash for the desktop
  3. Adobe is abandoning Flash for mobile

Taken together, these three points mean that continuing to reject H.264 <video> (and implicitly depend on Flash to support the video content that actually exists in the wild) might well leave mobile Firefox unable to play any meaningful amount of video at all. And certainly somebody somewhere must have realized that H.264 is the most popular encoding format within Flash movies, meaning that Mozilla’s attempt to push developers into other codecs simply wasn’t working.

Not that <video> is a cure-all; Streaming Media’s Jan Ozer makes the case that it’s not there, and probably never will be, at least not for non-trivial uses.

But as Streaming Media’s Troy Dreier points out in today’s article, at least this will rid us of the distraction of WebM: “This move drives a major nail in the WebM coffin, making WebM only slightly more relevant than Ogg Theora.”

Google fans among my Tweeps and +1’ers argue that WebM only failed because Google didn’t push it hard enough, but I think they’re wrong: WebM was always a dumb idea, one whose only advantage was political correctness among a holier-than-thou group of developers, few of whom had particularly deep knowledge of digital media beyond patent and licensing politics. Looking back at my earlier anti-WebM screed, my only regret is not slamming it even harder than I did.

Guess it’s just as well that I never created an webm category on the blog… turns out I wouldn’t ever need it.