Secret APIs

Discussing Apple’s Java deprecation, Java creator James Gosling blogged about the background of Java on the Mac, saying “the biggest obstacle was their use of secret APIs. Yes, OS X has piles of secret APIs. Just like the ones that Microsoft had that contributed to their antitrust problems.”

In a recent Q&A at Google, available on YouTube, he elaborates further, around 43 minutes in (embedded YouTube clip will take you right there, otherwise read the blockquote):


At Sun, we had worked with them to try to take it over. But there were all kinds of issues, and it was mostly things like, you know, to integrate properly into the Mac OS, there were a bunch of secret APIs. And in their integration, there were all these secret APIs, and they wouldn’t tell us what they were, we just knew they were there. And then, you know, it’s sort of like half their brain wanted to give us the code, half their brain is like “no no no no no, we can’t”. So, nyah, that was all kind of spastic.

The fact that Dr. Gosling brings up “secret APIs” repeatedly when talking about the subject makes me think that he really wants to make this point that Apple’s use of secret APIs and its intransigence has been a major problem for Java on the Mac.

But… is it true? How big a deal are secret APIs in OSX and iOS anyways?

Nobody denies that there are undocumented and otherwise secret APIs throughout both OSX and iOS. They are easily found through techniques such as reverse-engineering and method swizzling. On OSX, they can be called, provided you can figure out their proper usage without documentation. Technically, this is also possible on iOS, although use of non-public APIs will get your app rejected by the App Store, so it’s largely pointless.

The benign explanation for secret APIs is that they’re used internally but haven’t been fully vetted for use by third-parties. We’ve all written code we’re not proud of and wouldn’t want others calling, or at least written utility functions and methods that were only thought through for certain uses and aren’t known to be appropriate for general use. An interesting example is iOS’ UIGetScreenImage function. As a devforums thread indicates, Apple started allowing use of this private API in 2009 because there wasn’t a good public alternative, with the proviso that its use would be disallowed once a suitable public API was released. This occurred with the arrival of AV Foundation in iOS 4.0, and direct calls to UIGetScreenImage are again grounds for App Store rejection.

Aside from technical grounds, another reason for secret APIs is legal entanglements. There was an example of this in one of my earliest blogs: Apple licensed AAC encoding for OS X and for its own apps on Windows (iTunes, QuickTime Player), but not for third-party apps on Windows. According to Technical Q&A QA1347, a developer who wanted to provide this functionality on Windows would need to license the AMR encoding separately from VoiceAge, then provide proof of that license to Apple in order to get an SDK that would allow their code to make the secret call into QuickTime’s encoder.

But what can we say about Dr. Gosling’s complaints about secret APIs and Java? Certainly it plays well to the passions and politics of the Java community, but I’m not yet convinced. We know that most of Java actually ports to the Mac pretty easily: Landon Fuller’s “Soy Latte” project ported JDK 6 to the Mac in just a few person-weekends, and was later incorporated into OpenJDK’s BSD Ports subproject. But that left out some hard parts with intense native entanglements: sound, and the UI (Soy Latte, like most vanilla Java ports, relies on X11). Gosling acknowledges this in his blog, saying of these secret APIs that “the big area (that I’m aware of) where these are used is in graphics rendering.”

However, does this seriously mean that porting the Java graphics layer — Java2D, AWT, and Swing — is impractical or impossible without access to these secret APIs? It can’t be. After all, SWT exists for Mac as well, as a third-party creation, and it does the same things as these missing pieces of OpenJDK. In fact, SWT is more tightly-coupled to native code, as its whole approach is to bind Java objects to native peers (originally in Carbon, later in Cocoa), while Swing is all about avoiding native entanglements and instead painting look-alike widgets. Furthermore, I think Java’s rendering pipeline was switched over to an OpenGL implementation a while back, and that’s a public API that exists on OSX. So this kind of begs the question: what does Java need that isn’t provided by a public API? It doesn’t seem like graphics can be the problem.

The conspiracy theorists could argue that Apple has its own APIs that are more performant than the public APIs. Maybe, but what would be the point? Microsoft was roundly criticized for this in the 90’s, but Microsoft had more cases where their own products competed directly with third parties, and therefore could have incentive for their OS team to give a secret hand to the applications team. With Apple, software is their second-smallest revenue segment, and there are fewer cases where the company competes directly with a third-party rival (though there are clearly cases of this, such as Final Cut versus Premiere). Often, Apple’s software serves a strategic role – iLife may be more useful for selling Macs than for selling itself on DVD to existing Mac owners. So sure, Apple could be using secret APIs to give itself a leg up on competitors, but it’s hard to see how that would really be in their self-interest.

Having said all this, I’m still thwarted by a private API I needed this Summer: the “suck into a point” animation isn’t exposed by a Cocoa API on OSX, and asking for help on cocoa-unbound didn’t turn up an answer. Apparently, it’s possible on iOS, but via an undocumented method. Why this isn’t public on OSX or iOS, I can’t imagine, particularly given that Apple’s apps have made it a fairly standard behavior, meaning users will expect it when you use the round close button on a free-floating view. Oversight? Not ready for public consumption? Apple just being dicks? Who knows!

Of course, that brings up the last point about secret APIs. At the end of the day, they’re almost always conveniences. If something is possible at all, you could probably just do it yourself. I don’t know exactly what transforms are involved in the suck-to-close animation, but it’s surely possible to create a reasonably close approximation with Core Animation. Similarly, instead of calling QuickTime’s secret AAC encoder on Windows, you could license some other library or framework, or write your own. It might not be easy or practical, but if Apple can move the bits in some specific way, it must at least be possible for a third-party to do the same.

Comment (1)

  1. […] 3, 2011 Chris Adamson wrote today about private APIs in the Mac OS X Java AWT. He also links back to James Gosling’s post about […]

Leave a Reply

Your email address will not be published. Required fields are marked *