Archives for : January2011

Up Next…

My original plan for being featured on the iDevBlogADay blogroll was to be able to share some of the work I’m doing on the Core Audio book. I figured that as I worked through new material, that would translate into blog entries that could then get the word out about the book.

Unfortunately, I think what’s happening is that I’ve been working on iDevBlogADay entries instead of working on the book. And that’s not going to fly, at least if we want to get done in time for WWDC (two of which this book has already missed).

So, given that, and given that there are 50 other iOS developers waiting for a turn, I’m going cede my spot on iDevBlogADay, return to the waiting list, and hopefully apply that time to getting the book done.

If you want to keep following me, please do… previously, my blogging has tended to come and go in waves of inspiration, rather than the steady schedule that comes with participation in iDevBlogADay, so just grab the RSS feed URL or create a bookmark or just follow me on Twitter.

A Bit O’ Honey

Today’s announcement of the new features in Android 3.0 (Honeycomb) showed a feature I truly didn’t expect to see: support for HTTP Live Streaming.

Given Google’s decision to drop H.264 support from Chrome – a move that I denounced a few weeks back and would simply characterize here as batshit crazy – the idea of embracing HLS has to be seen as surprising, given that the MPEG codecs are the only commonly-used payloads in real-world HLS. The format could handle other payloads, but in practice, it’s all about the MP4s.

And that, of course, is because the target audience for HLS is iOS devices. Apple says they have an installed base of 160 million iOS devices out there now, and even the earliest iPhone can play an HLS stream. Moreover, App Store terms require the use of HLS for non-trivial streaming video applications. So there’s more and more content out there in this format. Android is wise to hop on this bandwagon, and opt in… unless of course they turn around and expect content providers to switch to WebM payloads (one would hope they’re not that dumb).

I don’t think I’d previously thought of the iOS base as a target for media providers, but found myself thinking: could the iOS base be bigger than Blu-Ray? A little searching shows it’s not even close: as of last Summer, Blu-Ray had a US installed base of just under 20 million, while iOS devices of all stripes number 40 million in the US (coincidentally making it the largest US mobile gaming platform as well). And while Blu-Ray had a good Christmas, iPad sales were insane.

Not every iOS user is going to stream video, and most content providers will need to develop custom apps to use the feature (Netflix, MLB, etc.), but those that do are already making big investments in the format. No wonder Google is opting in now… trying to get all the content providers to support an Android-specific format (other than Flash) would surely be a non-starter.

Now if Apple and the content providers could just work the kinks out…

Why Does iOS Video Streaming Suck? Part I

Experiment for you… Google for iphone youtube and look how pessimistic the completions are: the first is “iphone youtube fix” and the third is “iphone youtube not working”

Now do a search for iphone youtube slow and the completions all seem to tell a common story: that it’s slow on wifi. Moreover, there are more than 4 million hits across these search terms, with about 3.6 million just for “iphone youtube slow”.

Related searches show 630,000 hits for iphone netflix slow, 835,000 for iphone ustream slow and 9.6 million for the generic iphone video slow.

Surely something is going on.

I noticed it with my son’s YouTube habit. He often watches the various “let’s play” video game videos that people have posted, such as let’s play SSX. I keep hearing the audio start and stop, and realized that he keeps reaching the end of the buffer and starting over, just to hit the buffer again. Trying out YouTube myself, I find I often hit the same problem.

But when I was at CodeMash last week, even with a heavily loaded network, I was able to play YouTube and other videos on my iPhone and iPad much more consistently than I can at home. So this got me interested in figuring out what the problem is with my network.

Don’t get your hopes up… I haven’t figured it out. But I did manage to eliminate a lot of root causes, and make some interesting discoveries along the way.

The most common advice is to change your DNS server, usually to OpenDNS or Google Public DNS. Slow DNS is often the cause of web slowness, since many pages require lookups of many different sites for their various parts (images, ads, etc.). But this is less likely to be a problem for a pure video streaming app: you’re not hitting a bunch of different sites in the YouTube app, you’re presumably hitting the same YouTube content servers repeatedly. Moreover, I already had OpenDNS configured for my DNS lookups (which itself is a questionable practice, since it allegedly confuses Akamai).

Another suggestion that pops up in the forums is to selectively disable different bands from your wifi router. But there’s no consistency in the online advice as to whether b, g, or n is the most reliable, and dropping b and n from mine didn’t make a difference.

Furthermore, I have some old routers I need to put on craigslist, and I swapped them out to see if that would fix the problem. Replacing my D-Link DIR-644 with a Netgear WGR-614v4 or a Belkin “N Wireless Router” didn’t make a difference either.

In my testing, I focused on two sample videos, a YouTube video The Prom: Doomsday Theme from a 2008 symphonic performance of music from Doctor Who, and the Crunchyroll video The Melancholy of Haruhi Suzumiya Episode 3, played with the Crunchyroll iPhone app, so that I could try to expand the problem beyond the YouTube app and see if it applies to iOS video streaming in general.

And oh boy, does it ever. While the desktop versions of YouTube and Crunchyroll start immediately and play without pauses on my wifi laptop, their iOS equivalents are badly challenged to deliver even adequate performance. On my iPad, the “Doomsday” YouTube video takes at least 40 seconds to get enough video to start playing. Last night, it was nearly five minutes.

If anything, Crunchyroll performs worse on both iPhone and iPad. The “Haruhi” video starts almost immediately, but rarely gets more than a minute in before it exhausts the buffer and stops.

So what’s the problem? They’re all on the same network… but it turns out speeds are different. Using the website and the Mobile Speed Test app, I found that while my laptop gets 4.5 Mbps downstream at home, the iPad only gets about 2 Mbps, and the iPhone 3GS rarely gets over 1.5 Mbps.

I took the iPhone for a little tour to see if this was consistent on other networks, and got some odd results. Take a look at these:

The top two are on my LAN, and are pretty typical. The next two after that (1/22/11, 2:45 PM) are on the public wifi at the Meijer grocery/discount store in Cascade, MI. The two on the bottom are from a Culver’s restaurant just down 28th St. Two interesting points about these results. Again, neither gives me a download speed over 1.5 Mbps, but look at the uplink speed at Culver’s: 15 Mbps! Can this possibly be right? And if it is… why? Are the people who shoot straight-to-DVD movies in Grand Rapids coming over to Culver’s to upload their dailies while they enjoy a Value Basket?

As for Meijer, the ping and upload are dreadful… but it was the only place where Crunchyroll was actually able to keep up:

See that little white area on the scrubber just to the right of the playhead? It’s something I don’t see much on iOS: buffered data.

So what really is going on here anyways? For one thing, are we looking at progressive download or streaming? I suspect that both YouTube and Crunchyroll use HTTP Live Streaming. It’s easy to use with the iOS APIs, works with the codecs that are in the iDevices’ hardware, and optionally uses encryption (presumably any commercial service is going to need a DRM story in order to license commercial content). HLS can also automatically adjust to higher or lower bandwidth as conditions demand (well, it’s supposed to…). Furthermore, the App Store terms basically require the use of HLS for video streaming.

A look at the network traffic coming off the phone during a Crunchyroll session is instructive:

whois tells us that 65.49.43.x block is assigned to “CrunchyRoll, Inc.”, as expected, and it’s interesting to see that most of the traffic is on port 80 (which we’d expect from HLS), with the exception of one request on 443, which is presumably an https request. The fact that the phone keeps making new requests, rather than keeping one file connection open, is consistent with the workings of HLS, where the client downloads a .m3u8 playlist file, that simply provides a list of 30-second segment files that are then downloaded, queued, and played by the client. Given the consistent behavior between Crunchyroll and YouTube, and Apple’s emphasis on the technology, I’m inclined to hypothesize that we’re seeing HLS used by both apps.

But oh my goodness, why does it suck so much? The experience compares very poorly with YouTube on a laptop, which starts to play almost immediately and doesn’t stop after exhausting the buffer 30 seconds later. Whether you use Flash or the HTML5 support in YouTube (I’ve opted into the latter), it always just works, which is more than can currently be said of the iOS options, at least for me (and, if the Google hit count is right, for a couple million other people).

One other thing that doesn’t wash for me right now: remember when Apple started streaming their special events again? I blogged that it was a good demo of HLS, and I watched the first one with the iPhone, iPad, and Mac Pro running side-by-side to make a point that HLS was good stuff, and it all worked. How can the live stream hold up so well on three devices, yet a single archive stream falls apart on just one of the iOS devices?

I actually really like what I’ve seen of HLS: the spec is clean and the potential is immense. I even wondered aloud about doing a book on it eventually. But I can’t do that if I can’t get it working satisfactorily for myself.

What the hell is going on with this stuff?

If I ever get a good answer, there’ll be a “Part II”.

Do Over

One thing I’m coming away from CodeMash with is a desire to clean up a lot of my old habits and dig into tools and techniques I’ve long known were available, but haven’t used. In some ways, I’m still stuck in my iPhone OS 2 ways in an iOS 4 world.

Daniel Steinberg has taken a pretty extreme position, but one that makes sense: he no longer has any private instance variables in his header files, since the current SDK allows you to put them in the implementation. Combined with the use of a class extension in the .m for helper methods, this makes it possible for the header to be exactly what it’s supposed to be: an exposure of the public interface to your class, with no clues about the implementation underneath.

To my mind, Daniel was also the winner of the “mobile smackdown” session, in which one presenter each from the iOS, Windows Phone 7, and Android camps was given 15 minutes to develop a trivial Twitter app that could manage a persistent list of user names and, when tapped, nagivate to that user’s page. I say Daniel won because his iPhone app was the only one to complete all the features in time (actually, Daniel needed an extra 30 seconds to finish two lines of code). The Windows Phone presenter never made it to adding new names to the list, and the Android guy didn’t get around to showing the user’s page. One of Daniel’s wins was in using the “use Core Data for storage” checkbox: by graphically designing a data model for his “Twitterer” class, he picked up persistence and his table view in one fell swoop. Now that I think of it, I don’t remember how, or if, the other platforms persisted their user lists. I don’t use Core Data often, but after this demonstration, I’m much more inclined to do so.

There was a whole session on unit testing for iOS, something I just explored on my own for my first day tutorial (and even then, I was using it as much for illustrating the use of multiple targets in an Xcode project as for the actual testing of features). I’ve never been religious about testing, particularly given that GUIs have long proven difficult to make fully testable, but with a testing framework buit into Xcode (not everyone’s favorite, but it’s a start), it’s well worth rethinking how I could use it to get some measure of test coverage and fight regressions.

All of this makes me worry about the status of the iPhone SDK Development book I wrote with Bill Dudney. That was an iPhone OS 2 book that slipped far enough to be an early iPhone OS 3 book, with the addition of new chapters for important new frameworks like Core Data and Game Kit. But with iOS 5 surely looming, some of it is starting to look pretty crusty. In particular, the arrival of Grand Central Dispatch means that means that it’s no longer safe to blithely ignore threads, as we did, since there are scenarios where you can have even simple code that unwittingly manages to get off the main thread, which means trouble for UIKit. Furthermore, new frameworks demand blocks for completion handlers, so that’s something that now needs to appear early (and given that the block syntax is pure C, readers will need to be acclimated to C earlier than they used to). And I’ve long wanted to move the debugging and performance chapters (my favorites, actually) much earlier, so readers can figure out their own EXC_BAD_ACCESS problems. Not that I can currently even plan on a rev to that book – I still have four chapters to go on Core Audio, and would need a long and difficult conversation with the Prags besides. But I certainly see where my guidance to new developers has changed, significantly, in the last few years.

Betweeen Christmas break, a week of CodeMash prep and home office reorganization, and CodeMash itself, I feel like I’ve been off for a month (and my MYOB First Edge financial status would seem to agree). I feel ready to start anew this week, and making a clean break with the past suits this mood nicely.

An Encoder Is Not A State Machine

I’m pleasantly surprised that Google’s removal of H.264 from Chrome in favor of WebM has been greeted with widespread skepticism. You’d think that removing popular and important functionality from a shipping product would be met with scorn, but when Google wraps itself with the “open” buzzword, they often seem to get a pass.

Ars Technica’s Google’s dropping H.264 from Chrome a step backward for openness has been much cited as a strong argument against the move. It makes the important point that video codecs extend far beyond the web, and that H.264’s deep adoption in satellite, cable, physical media, and small devices make it clearly inextricable, no matter how popular WebM might get on the web (which, thusfar, is not much). It concludes that this move makes Flash more valuable and viable as a fallback position.

And while I agree with all of this, I still find that most of the discussion has been written from the software developer’s point of view. And that’s a huge mistake, because it overlooks the people who are actually using video codecs: content producers and distributors.

And have they been clamoring for a new codec? One that is more “open”? No, no they have not. As Streaming Media columnist Jan Ozer laments in Welcome to the Two-Codec World,

I also know that whatever leverage Google uses, they still haven’t created any positive reason to distribute video in WebM format. They haven’t created any new revenue opportunities, opened any new markets or increased the size of the pie. They’ve just made it more expensive to get your share, all in the highly ethereal pursuit of “open codec technologies.” So, if you do check your wallet, sometime soon, you’ll start to see less money in it, courtesy of Google.

I’m grateful that Ozer has called out the vapidity of WebM proponents gushing about the “openness” of the VP8 codec. It reminds me of John Gruber’s jab (regarding Android) that Google was “drunk on its own keyword”. What’s most atrocious to me about VP8 is that open-source has trumped clarity, implementability, and standardization. VP8 apparently only exists as a code-base, not as a technical standard that could, at least in theory, be re-implemented by a third party. As the much-cited first in-depth technical analysis of VP8 said:

The spec consists largely of C code copy-pasted from the VP8 source code — up to and including TODOs, “optimizations”, and even C-specific hacks, such as workarounds for the undefined behavior of signed right shift on negative numbers. In many places it is simply outright opaque. Copy-pasted C code is not a spec. I may have complained about the H.264 spec being overly verbose, but at least it’s precise. The VP8 spec, by comparison, is imprecise, unclear, and overly short, leaving many portions of the format very vaguely explained. Some parts even explicitly refuse to fully explain a particular feature, pointing to highly-optimized, nigh-impossible-to-understand reference code for an explanation. There’s no way in hell anyone could write a decoder solely with this spec alone.

Remember that even Microsoft’s VC-1 was presented and ratified as an actual SMPTE standard. One can also contrast the slop of code that is VP8 with the strategic designs of MPEG with all their codecs, standardizing decoding while permitting any encoder that produces a compliant stream that plays on the reference decoder.

This matters because of something that developers have a hard time grasping: an encoder is not a state machine. Meaning that there need not, and probably should not be, a single base encoder. An obvious example of this is the various use cases for video. A DVD or Blu-Ray disc is encoded once and played thousands or millions of times. In this scenario, it is perfectly acceptable for the encode process to require expensive hardware, a long encode time, a professional encoder, and so on, since those costs are easily recouped and are only needed once. By contrast, video used in a video-conferencing style application requires fairly modest hardware, real-time encoding, and can make few if any demands of the user. Under the MPEG-LA game-plan, the market optimizes for both of these use cases. But when there is no standard other than the code, it is highly unlikely that any implementations will vary much from that code.

Developers also don’t understand that professional encoding is something of an art, that codecs and different encoding software and hardware have distinct behaviors that can be mastered and exploited. In fact, early Blu-Ray discs were often authored with MPEG-2 rather than the more advanced H.264 and VC-1 because the encoders — both the devices and the people operating them — had deeper support for and a better understanding of MPEG-2. Assuming that VP8 is equivalent to H.264 on any technical basis overlooks these human factors, the idea that people now know how to get the most out of H.264, and have little reason to achieve a similar mastery of VP8.

Also, MPEG rightly boasts that ongoing encoder improvements over time allow for users to enjoy the same quality at progressively lower bitrates. It is not likely that VP8 can do the same, so while it may be competitive (at best) with H.264, it won’t necessarily stay that way.

Furthermore, is the MPEG-LA way really so bad? Here’s a line from a review in the Discrete Cosine blog of VP8 back when On2 was still trying to sell it as a commercial product:

On2 is advertising VP8 as an alternative to the mucky patent world of the MPEG licensing association, but that process isn’t nearly as difficult to traverse as they imply, and I doubt the costs to get a license for H.264 are significantly different than the costs to license VP8.

The great benefit of ISO standards like VC-1 and H.264 is that anyone can go get a reference encoder or reference decoder, with the full source code, and hack on their own product. When it times come to ship, they just send the MPEG-LA a dollar (or whatever) for each copy and everyone is happy.

It’s hard to understand what benefits the “openness” of VP8 will ever really provide. Even if it does end up being cheaper than licensing H.264 from MPEG-LA — and even if the licensing body would have demanded royalty payments had H.264 not been challenged by VP8 — proponents overlook the fact that the production and distribution of video is always an enormously expensive endeavor. 15 years ago, I was taught that “good video starts at a thousand dollars a minute”, and we’d expect the number is at least twice that today, just for a minimal level of technical competence. Given that, the costs of H.264 are a drop in the bucket, too small to seriously affect anyone’s behavior. And if not cost, then what else does “openness” deliver? Is there value in forking VP8, to create another even less compatible codec?

In the end, maybe what bugs me is the presumption that software developers like the brain trust at Google know what’s best for everyone else. But assuming that “open source” will be valuable to video professionals is like saying that the assembly line should be great for software development because it worked for Henry Ford.

The Dark Depths of iOS

CodeMash starts Wednesday in Sandusky, with half-day iOS tutorials from Daniel Steinberg and myself, followed by two days of sessions. My Thursday session is The Dark Depths of iOS, and is a rapid-fire tour of the non-obvious parts of the iOS APIs.

Researching for this has proven an interesting exercise. I had the idea for the talk from the occasional dive into Core Foundation for functionality that is not exposed at higher levels of the iOS and Mac stacks. A simple example of this is the CFUUID, the Universally unique identifier defined by RFC 4122. More than once, I’ve needed an arbitrary unique ID (other than, say, the device ID), and am happy to use the industry standard. But it’s not defined in Foundation or Cocoa, so you need to use Core Foundation and its C API.

Another example I knew about before starting this talk was the CFNetwork sub-framework, which provides a much more complete networking stack than is available in Cocoa’s URL Loading System. CFNetwork allows you make arbitrary socket connections, work with hosts (e.g., to do DNS lookups), accept connections, etc. Basically, if what you need from the network can’t be expressed as a URL, you need to drop down at least to this level. It has an advantage over traditional BSD sockets in that it integrates with the Core Foundation view of the world, most importantly in that its reading and writing APIs use the asynchronous callback design patterns common to Apple’s frameworks, rather than blocking as the traditional C APIs would.

Stuff like those, along with Accelerate and Keychain, are things I knew I wanted to talk about, and did the first few slides on Core Media and Core Services by pointing out reasonably findable information from Apple’s architecture docs.

The eye-openers for me were down in the “System” level of Core OS… the various C APIs that iOS inherits from its open-source roots in FreeBSD and NetBSD (i.e., Darwin). These aren’t substantially documented in Xcode (though they do enjoy syntax highlighting and sometimes code-completion). The documentation for these is in the man pages, which of course makes sense for long-time Unix programmers, but maybe less so today… if I’m writing in Xcode for a separate iOS device, the Mac command-line is a rather counterintuitive location to look for API documentation, isn’t it?

So the trick with calling the standard C libraries is finding out just what’s available to you. Apple has links to iOS Manual Pages that collect a lot of these APIs in one place, but they are largely unordered (beyond the historical Unix man page sections), and being programmatically generated, Apple can only offer a warning that while some APIs known to be unavailable on iOS have been filtered out, the list isn’t guaranteed to be completely accurate. There’s also a consideration that some of these APIs, while they may exist, are not particularly useful given the limitations under which third-party iOS applications operate. For example, all the APIs involving processes (e.g., getting your PID and EUID) and inter-process communication are presumably only of academic interest — the iPhone is not the campus timeshare mainframe from 1988. Similarly, ncurses is probably not going to do much for you on a touch display. OK, maybe if you’re writing an ssh client. Or if you really need to prove that Angry Birds could have worked on a VT100.

Another way of figuring out what’s there — if less so how to actually call it — is to go spelunking down in <iOS_SDK>/usr/lib and <iOS_SDK>/usr/include to get an idea of the organization and packaging of the standard libraries. Had I not done this, I might not have realized that there is a C API for regular expression matching (regex.h), XML parsing with libxml2 (both DOM and SAX), zip and tar, MD5 and SHA (in CommonCrypto/) and other interesting stuff.

On the other hand, there are cases where code libraries are available but don’t have public headers. For example, libbz2.dylib are libtidy.dylib are in usr/lib but don’t seem to have corresponding entries in usr/include. That begs the question of whether you could call into these BZip and Tidy libraries and remain “App Store safe”, given the apparent lack of a “public” API, even though you could easily get the headers for these open-source libraries from their host projects. Heck, kebernet pointed out to me that tidy.h and bzlib.h are available in the iPhoneSimulator.platform path, just not iPhoneOS.platform.

It would be nice if there were better visibility into these libraries, though their nature as unrepentant C probably scares off a lot of developers, who will be just as well off scouring Google Code for a nice Objective-C alternative (provided the license they find is compatible with their own). My takeaway is that there’s even more functionality at these low levels than I expected to find. I’ll probably at least consider using stuff like libxml2 and regex.h in the future.

Apparently, you *are* entitled to your own facts now

Everything you read in newspapers is absolutely true, except for that rare story of which you happen to have first-hand knowledge. ~Erwin Knoll

I ended up spending a fair amount of time disproving an obviously wrong newspaper story last night. It didn’t work.

In college, I was a member of the Stanford Band, a group I keep up with via Facebook and alumni e-mails. They were featured in a Miami Herald front page story this weekend, about their antics and their upcoming performance at the Orange Bowl pregame.

Imagine my surprise when the San Francisco Chronicle reported that the Band was banned from performing at halftime. Surprised, because the story is totally wrong.

The Chron story sources an MSNBC story, which itself cites no sources, and whose URL suggests it is a local affil item submitted to MSNBC. The MSNBC story’s facts are all from the Miami Herald write, and is likely its only source, and uncredited at that. Here’s the last two grafs of the Herald write:

South Florida is not exactly Arkansas, but cautious Orange Bowl organizers have reduced the opportunity for indignity by keeping both college bands off the field at halftime; they’ll be restricted to brief, six-minute pregame shows.

Stanford Band bosses are keeping mum about their plans, saying only that the show is titled Recent Events in the Pro Sports World in Miami. Look out, LeBron.

This omits a crucial fact — Orange Bowl halftimes are always gala affairs that do not involve the marching bands — but this is written around in an amusing tone that’s consistent with the rest of the article.

The MSNBC affil didn’t pick up that fact, or on the light tone, and took those last grafs for their lead:

Fearing an en masse pants drop, or just wanting to protect their newest celeb athlete, LeBron James, from OJ Simpson-style mockery, the Orange Bowl administrators have decided to keep the bawdy Stanford Band from performing at halftime.

MSNBC picked a few highlights of the Band’s antics for the body of their article, then returned to the “banning” in their last grafs:

Denying the Stanford Band a stage also denies Virginia Tech one. The Orange Bowl will only allow the teams’ bands to perform in six-minute bursts before kickoff.

The Band’s show is entitled “Recent Events in the Pro Sports World in Miami.”

Whose talents could possibly be targeted, er, featured?

Using the MSNBC write as its only source – and with apparently no vetting of the facts via a local call to Stanford’s athletics department – the Chronicle turns the lightweight kicker from the Herald into a hard lead:

Orange Bowl administrators, determined to make tonight’s matchup between No. 5 Stanford and No. 12 Virginia Tech less entertaining, have decided to bar Stanford’s irreverent band from performing at halftime.

The move comes after the band announced its show was entitled: “Recent Events in the Pro Sports World in Miami.”

This is where I got involved. Hopping into the article’s comments section, I posted a series of followups, determined to prove the article false. The last comment I posted linked to four sources that could completely dispel the story:

I also e-mailed the writer of the article. I would have thought this would be enough to get the article – demonstrably and totally wrong – retracted.

Instead, it’s still on SFGate’s front page the next morning:

Obviously, this has hit a nerve with me. I can’t help it: I used to be an editor. When ESPN ScoreCenter sent me a push notification of the game’s final score, including the text “A. Luck(STAN) 4TD, 0INT”, my first thought was “Bullshit, he threw a pick in the second quarter.” Loyalties be damned, A is still A.

To see a major newspaper so sloppy and so obviously wrong is shameful. When I was writing and editing at CNN, if I had ever used a single source, which cited none of its own sources, with no vetting and no common sense fact-checking, I’d have been busted back to separating carbons in the printer room within a week.

Frankly, I’m going to get a good chuckle the next time one of the Chron’s 27 liberal columnists complains about obvious falsehoods on Fox News. It should also rankle that Wikipedia gets this more right than the Chron does. From a overnight edit to the Stanford Band article:

Despite Twitter rumors to the contrary, the band was not banned from performing during the 2011 Orange Bowl halftime. The Orange Bowl traditionally has major-label recording artists perform the halftime show, not school marching bands (The Goo Goo Dolls performed the 2011 show). They did, however perform during pregame, which was briefly shown on the game’s national broadcast. The theme of that show was “Recent Events in the Pro Sports World in Miami.”

Yesterday was a really rough day for me in terms of getting things straight. I also, perhaps foolishly, hopped into discussions of a Detroit Free Press article discussing the possibility that Stanford coach Jim Harbaugh could be the one to turn around the University of Michigan’s slumping football program. There’s a lot of enthusiasm for that here in Michigan, and I’ve been trying to throw cold water on it. Sure, I’m motivated by the implicit slight to Stanford. But moreover, it’s not a given that Harbaugh would want to give up a top-5 program that he’s built in order to start over with a reclamation project, one that he pointedly insulted a few years ago. It would, you’d think, at least take a dump truck of money. Moreover, a number of NFL teams are openly courting Harbaugh. I posted comments with links to articles indicating the 49ers, Broncos, and Panthers were openly pursuing him, along with a nice New York Times profile that covered Harbaugh’s options.

The result? A lot of people calling me various names, and one poster in particular who insisted repeatedly that it didn’t matter because Michigan had already hired Harbaugh a month ago. Now how does this even pass Occam’s Razor? Why would the NFL teams waste time and money, and a chance to land coaches who are actually available, if U-M had already secured Harbaugh’s services, or even thought they had? Why would anyone even be talking about it if it were a done deal? It can’t be, not yet anyways, and yet this guy stuck to his guns.

And yet, how is that any different from the Chron? There’s no penalty to being obviously wrong. When the facts aren’t on your side, just yell louder. The Big Lie works, in part because people believe what they want to believe. The Chron’s totally wrong article has been shared to Facebook nearly 1,700 times as of this writing.

We all know the internet is a breeding ground for ignorance, that you can find seemingly reputable sources for whatever stupid nonsense you care to believe (Obama’s not really American, 9/11 was an inside job, the CIA created AIDS to kill black people, etc.), but it’s still ghastly to see it in action.

My local paper recently declared it’s getting tough with trolls in its forums, and while I wish them the best, I wonder if it isn’t better in the long run to just ditch user comments altogether. I find I enjoy the consistent voice of an author – Daring Fireball is the obvious example of this in the Mac/iOS world – whereas I rarely find anything of value in feedback forums, just a mudslinging scrum among various partisans.

Comments should allow for readers to communicate back to publishers, and could provide a valuable means correcting bad information. And as professionals, publishers should want to be right: it’s the only thing that distinguishes them from any random Joe with a website.

But that’s where we started. I tried correcting them. It didn’t work. And the Chron is still happily collecting hits on an article that’s demonstrably and totally wrong.

Secret APIs

Discussing Apple’s Java deprecation, Java creator James Gosling blogged about the background of Java on the Mac, saying “the biggest obstacle was their use of secret APIs. Yes, OS X has piles of secret APIs. Just like the ones that Microsoft had that contributed to their antitrust problems.”

In a recent Q&A at Google, available on YouTube, he elaborates further, around 43 minutes in (embedded YouTube clip will take you right there, otherwise read the blockquote):


At Sun, we had worked with them to try to take it over. But there were all kinds of issues, and it was mostly things like, you know, to integrate properly into the Mac OS, there were a bunch of secret APIs. And in their integration, there were all these secret APIs, and they wouldn’t tell us what they were, we just knew they were there. And then, you know, it’s sort of like half their brain wanted to give us the code, half their brain is like “no no no no no, we can’t”. So, nyah, that was all kind of spastic.

The fact that Dr. Gosling brings up “secret APIs” repeatedly when talking about the subject makes me think that he really wants to make this point that Apple’s use of secret APIs and its intransigence has been a major problem for Java on the Mac.

But… is it true? How big a deal are secret APIs in OSX and iOS anyways?

Nobody denies that there are undocumented and otherwise secret APIs throughout both OSX and iOS. They are easily found through techniques such as reverse-engineering and method swizzling. On OSX, they can be called, provided you can figure out their proper usage without documentation. Technically, this is also possible on iOS, although use of non-public APIs will get your app rejected by the App Store, so it’s largely pointless.

The benign explanation for secret APIs is that they’re used internally but haven’t been fully vetted for use by third-parties. We’ve all written code we’re not proud of and wouldn’t want others calling, or at least written utility functions and methods that were only thought through for certain uses and aren’t known to be appropriate for general use. An interesting example is iOS’ UIGetScreenImage function. As a devforums thread indicates, Apple started allowing use of this private API in 2009 because there wasn’t a good public alternative, with the proviso that its use would be disallowed once a suitable public API was released. This occurred with the arrival of AV Foundation in iOS 4.0, and direct calls to UIGetScreenImage are again grounds for App Store rejection.

Aside from technical grounds, another reason for secret APIs is legal entanglements. There was an example of this in one of my earliest blogs: Apple licensed AAC encoding for OS X and for its own apps on Windows (iTunes, QuickTime Player), but not for third-party apps on Windows. According to Technical Q&A QA1347, a developer who wanted to provide this functionality on Windows would need to license the AMR encoding separately from VoiceAge, then provide proof of that license to Apple in order to get an SDK that would allow their code to make the secret call into QuickTime’s encoder.

But what can we say about Dr. Gosling’s complaints about secret APIs and Java? Certainly it plays well to the passions and politics of the Java community, but I’m not yet convinced. We know that most of Java actually ports to the Mac pretty easily: Landon Fuller’s “Soy Latte” project ported JDK 6 to the Mac in just a few person-weekends, and was later incorporated into OpenJDK’s BSD Ports subproject. But that left out some hard parts with intense native entanglements: sound, and the UI (Soy Latte, like most vanilla Java ports, relies on X11). Gosling acknowledges this in his blog, saying of these secret APIs that “the big area (that I’m aware of) where these are used is in graphics rendering.”

However, does this seriously mean that porting the Java graphics layer — Java2D, AWT, and Swing — is impractical or impossible without access to these secret APIs? It can’t be. After all, SWT exists for Mac as well, as a third-party creation, and it does the same things as these missing pieces of OpenJDK. In fact, SWT is more tightly-coupled to native code, as its whole approach is to bind Java objects to native peers (originally in Carbon, later in Cocoa), while Swing is all about avoiding native entanglements and instead painting look-alike widgets. Furthermore, I think Java’s rendering pipeline was switched over to an OpenGL implementation a while back, and that’s a public API that exists on OSX. So this kind of begs the question: what does Java need that isn’t provided by a public API? It doesn’t seem like graphics can be the problem.

The conspiracy theorists could argue that Apple has its own APIs that are more performant than the public APIs. Maybe, but what would be the point? Microsoft was roundly criticized for this in the 90’s, but Microsoft had more cases where their own products competed directly with third parties, and therefore could have incentive for their OS team to give a secret hand to the applications team. With Apple, software is their second-smallest revenue segment, and there are fewer cases where the company competes directly with a third-party rival (though there are clearly cases of this, such as Final Cut versus Premiere). Often, Apple’s software serves a strategic role – iLife may be more useful for selling Macs than for selling itself on DVD to existing Mac owners. So sure, Apple could be using secret APIs to give itself a leg up on competitors, but it’s hard to see how that would really be in their self-interest.

Having said all this, I’m still thwarted by a private API I needed this Summer: the “suck into a point” animation isn’t exposed by a Cocoa API on OSX, and asking for help on cocoa-unbound didn’t turn up an answer. Apparently, it’s possible on iOS, but via an undocumented method. Why this isn’t public on OSX or iOS, I can’t imagine, particularly given that Apple’s apps have made it a fairly standard behavior, meaning users will expect it when you use the round close button on a free-floating view. Oversight? Not ready for public consumption? Apple just being dicks? Who knows!

Of course, that brings up the last point about secret APIs. At the end of the day, they’re almost always conveniences. If something is possible at all, you could probably just do it yourself. I don’t know exactly what transforms are involved in the suck-to-close animation, but it’s surely possible to create a reasonably close approximation with Core Animation. Similarly, instead of calling QuickTime’s secret AAC encoder on Windows, you could license some other library or framework, or write your own. It might not be easy or practical, but if Apple can move the bits in some specific way, it must at least be possible for a third-party to do the same.