Archives for : web

Facebook for iOS Pigs Out

Last night, I was thwarted from adding a show to my iPad because I was nearly out of space:

iTunes usage: iPad nearly full

I started deleting large, rarely-used apps (goodbye for now, GarageBand!), and ultimately trimmed the playlist of songs that syncs with the iPad. But as I looked through my per-app usage, this caught my eye:

iOS Facebook using 421 MB

That’s right: Facebook, a network-based application (one that is typically accessed as a web page) is using nearly half a gigabyte for documents.

No. No you don’t. I deleted Facebook and reinstalled from the App Store to reclaim the space. But then I thought, what the hell is it doing with all that space?

Fortunately, I still had Facebook on my iPhone, where it’s only burning a little over 100 MB. I took a look with iExplorer, and dumped Facebook to my Mac:

Contents of Facebook for iOS app, including 100 MB of cache

So look at that: 12 MB for the app, over 100 MB of Caches, nearly all of it in FBURLCache. Bad enough that it burns a seemingly unlimited amount of space, but let’s flip back to iExplorer and look at the file dates:

FBURLCache file dates

Yep, the oldest files in this cache are three months old. What are the odds that I’m going to want any of those files, and even if I do, that I’m going to get any benefit from caching them locally?

It gets better. After poking around the 1.5 MB cacheinfo.plist index file and its 6,000 entries, I dug into one of the cache files with HexEdit, which immediately revealed it as a simple .plist. Off to the Property List Editor we go!

Facebook cache file property list

So, great… for a “Recommend” button on some web page I accessed back in January, Facebook has cached the entire NSHTTPURLResponse. 1,700 bytes of cache for a 339-byte GIF. And apparently it’s done this for every element of every web page I’ve viewed with the Facebook app.

Unbelievable. I rarely view a web page more than once through the FB app, and it’s highly doubtful anyone is getting more benefit from caching these files to be worth the space they’re consuming. It seems there may be a 3-month roll-off (or maybe that’s when I last reinstalled the app?), but that’s a uselessly long period: it’s rare you can roll back to posts that old on Facebook. And I’m left to assume that there’s no limit to the size this cache is allowed to grow to. At over 400 MB, the cache on my iPad was nearly the size of two standard-def TV shows… something that would do me a lot more good than caches of links I’ll never view again.

Should there be an option to turn the cache off, or at least to cap its size or or set its expiration date? Of course there should. But given that this app is the same one that giddily uploaded all the entries in my contacts list years ago, I probably shouldn’t expect better.

Point and Laugh: The End of WebM

As hinted last week, Mozilla has finally seen the writing on the wall and decided to support H.264 in the <video> tag in Firefox. Probably inevitable given that

  1. Mozilla is making a new push into mobile
  2. The default standard for web video has become H.264 <video> for mobile and Flash for the desktop
  3. Adobe is abandoning Flash for mobile

Taken together, these three points mean that continuing to reject H.264 <video> (and implicitly depend on Flash to support the video content that actually exists in the wild) might well leave mobile Firefox unable to play any meaningful amount of video at all. And certainly somebody somewhere must have realized that H.264 is the most popular encoding format within Flash movies, meaning that Mozilla’s attempt to push developers into other codecs simply wasn’t working.

Not that <video> is a cure-all; Streaming Media’s Jan Ozer makes the case that it’s not there, and probably never will be, at least not for non-trivial uses.

But as Streaming Media’s Troy Dreier points out in today’s article, at least this will rid us of the distraction of WebM: “This move drives a major nail in the WebM coffin, making WebM only slightly more relevant than Ogg Theora.”

Google fans among my Tweeps and +1’ers argue that WebM only failed because Google didn’t push it hard enough, but I think they’re wrong: WebM was always a dumb idea, one whose only advantage was political correctness among a holier-than-thou group of developers, few of whom had particularly deep knowledge of digital media beyond patent and licensing politics. Looking back at my earlier anti-WebM screed, my only regret is not slamming it even harder than I did.

Guess it’s just as well that I never created an webm category on the blog… turns out I wouldn’t ever need it.

In-App Purchase and Rent-Seeking

In economics, rent-seeking occurs when an individual, organization or firm seeks to earn income by capturing economic rent through manipulation or exploitation of the economic or political environment, rather than by earning profits through economic transactions and the production of added wealth.
Rent-seeking, from Wikipedia

Most of my tweets today have been about in-app purchase, first with the news of Sony’s Reader app being rejected on the basis of its purchase model, and then Apple’s clarification / de facto policy change stating that apps that offer user-purchasable content must offer Apple in-app purchase in addition to whatever other purchase methods might be available.

Matt Ingram at GigaOM nearly nails the issue in a section header:

The Landlord Will Get His Share

What Apple seeks to collect is rent, specifically monopoly rent, the ability to extract excess payments not as a consequence of a mutually-beneficial trade, but because it owns the damn store and it sets the damn rules.

I have long argued that in-app purchase can be a pretty raw deal for developers. Slide 50 from my Voices That Matter (Seattle 2010) talk, In-App Purchase: Best/Worst Thing Ever puts this in perspective:

App Store In-App Purchase
Product Hosting
Audit Trail
Purchase UI
Purchase Restoration (sometimes)
Local Purchase Storage
Apple’s Cut 30% 30%

To clarify what I mean by each of these:

  • Product Hosting: Apps are hosted on Apple’s servers. In-App Purchases go on your server (unless they’re bundled with the app and simply unlocked)
  • Audit Trail: The audit trail for an app purchase is entirely verifiable through Apple. With In-App Purchases, the developer gets a digital receipt that he or she needs to log on their own server, after validating it against an Apple webservice.
  • Purchase UI: For apps, the description, screenshots, user ratings, pricing, and purchase process are all handled by iTunes or the iOS App Store. For I-AP, the developer needs to create a UI for all of this, beyond a simple “are you sure” alert provided by Store Kit once a purchase is initiated.
  • Purchase Restoration: Users can easily re-download their old apps to a new device for free with iTunes or the App Store, once they’ve logged back into their iTunes account. For In-App Purchases, developers are expected to provide restoration of “durable” and “subscription” products, but Store Kit’s -[SKPaymentQueue restoreCompletedTransactions] method only restores the former. There is no programmatic way to discover old subscription purchases on a new device. There’s also no way to get an identifier for the iTunes account, so you don’t even have anything to associate with the subscription purchase on your own server.
  • Local Purchase Storage: iOS is responsible for storing the apps on the device’s file system. The developer is responsible for storing in-app purchases, ideally in a form that will survive crashes, wipes and restores, etc.

If you sell a $10 subscription, you do almost all the work, and yet Apple still feels it is entitled to a $3 cut for doing little more than running a credit-card swipe. Obviously, I don’t think they earn it.

If Apple’s now going to force the issue, it’s truly a sad and ugly moment for them and for all of us. How far does this concept go, after all? Consider websites that have premiere memberships, like or Crunchyroll. Both of them have dedicated iOS apps, since the full functionality of their Flash-heavy sites don’t work in Mobile Safari. Is Apple going to insist that these memberships be made available for purchase by I-AP as well? Or will users not be able to use iOS devices to access the online content they’ve paid for? How is that good for the platform?

It’s enough to make you hope that content providers switch to an ad-supported model and let Google make all the money. That would be just desserts, with a cherry on top.

Life Beyond The Browser

“Client is looking for someone who has developed min. of 1 iPhone/iPad app.  It must be in the App Store no exceptions.  If the iPhone app is a game, the client is not interested in seeing them.” OK, whatever… I’ll accept that a game isn’t necessarily a useful prerequisite. But then this e-mail went on: “The client is also not interested in someone who comes from a web background or any other unrelated background and decided to start developing iPhone Apps.”

Wow. OK, what the hell happened there? Surely there’s a story behind that, one that probably involved screaming, tears, and a fair amount of wasted money. But that’s not what I’m interested in today.

What surprised me about this was the open contempt for web developers, at least those who have tried switching over to iOS development. While I don’t think we’re going to see many of these “web developer need not apply” posts, I’m still amazed to have seen one at all.

Because really, for the last ten years or so, it’s all been about the web. Most of the technological innovation in the last decade arrived in the confines of the browser window, and we have been promised a number of times that everything would eventually move onto the web (or, in a recent twist, into the cloud).

But this hasn’t fully panned out, has it? iOS has been a strong pull in the other direction, and not because Apple wanted it that way. When the iPhone was introduced and the development community given webapps as the only third-party development platform, the community reaction was to jailbreak the device and reverse-engineer iPhone 1.0’s APIs.

And as people have come over, they’ve discovered that things are different here. While the 90’s saw many desktop developers move to the web, the 10’s are seeing a significant reverse migration. In the forums for our iPhone book, Bill and I found the most consistently flustered readers were the transplanted web developers (and to a lesser degree, the Flash designers and developers).

Part of this was language issues. Like all the early iPhone books, we had the “we assume you have some exposure to a C-based curly-brace language” proviso in the front. Unfailingly, what tripped people up was the lurking pointer issues that Objective-C makes no attempt to hide. EXC_BAD_ACCESS is exactly what it says it is: an attempt to access a location in memory you have no right to touch, and almost always the result of following a busted pointer (which in turn often comes from an object over-release). But if you don’t know what a pointer is, this might as well be in Greek.

And let’s think about languages for a minute. There has been a lot of innovation around the web programming languages. Ruby and Python have (mercifully) replaced Perl and PHP in a lot of the conventional wisdom about web programming languages, while the Java Virtual Machine provides a hothouse for new language experimentation, with Clojure and Scala picking gaining some very passionate adherents.

And yet, none of these seem to have penetrated desktop or device programming to any significant degree. If the code is user-local, then it’s almost certainly running in some curly-brace language that’s not far from C. On iOS, Obj-C/C/C++ is the only provided and only practical choice. On Mac, Ruby and Python bindings to Cocoa were provided in Leopard, but templates for projects using these languages no longer appear in XCode’s “New Project” dialog in Snow Leopard. And while I don’t know Windows, it does seem like Visual Basic has finally died off, replaced by C#, which seems like C++ with the pointers taken out (i.e., Java with a somewhat different syntax).

So what’s the difference? It seems to me like the kinds of tasks relevant to each kind of programming is more different than is generally acknowledged. In 2005’s Beyond Java, Bruce Tate argued that a primary task of web development was mostly about doing the same thing over and over again: connecting a database to a web page. You can snip at specifics, but he’s got a point: you say “putting an item in the user’s cart”, I say “writing a row to the orders table”.

If you buy this, then you can see how web developers would flock to new languages that make their common tasks easier — iterating over collections of fairly rich objects in novel and interesting ways has lots of payoff for parsing tree structures, order histories, object dependencies and so on.

But how much do these techniques help you set up a 3D scene graph, or perform signal processing on audio data captured from the mic? The things that make Scala and Ruby so pleasant for web developers may not make much of a difference in an iOS development scenario.

The opposite is also true, of course. I’m thrilled by the appearance of the Accelerate framework in iOS 4, and Core MIDI in 4.2… but if I were writing a webapp, a hardware-accelerated Fast Fourier Transform function likely wouldn’t do me a lot of good.

I’m surprised how much math I do when I’m programming for the device. And not just for signal processing. Road Tip involved an insane amount of trigonometry, as do a lot of excursions into Core Animation.

The different needs of the different platforms create different programmers. Here’s a simple test: which have you used more in the last year: regular expressions, or trigenometry? If it’s the former, you’re probably a web developer; the latter, device or desktop. (If you’ve used neither, you’re a newbie, and if you’ve used both, then you’re doing something cool that I probably would like to know about).

Computer Science started as a branch of mathematics… that’s the whole “compute” part of it after all. But times change; a CS grad today may well never need to use a natural logarithm in his or her work. Somebody — possibly Brenda Laurel in Computers as Theatre (though I couldn’t find it in there) — noted that the French word for computer, ordinateur, is a more accurate name today, being derived from root word for “organize” rather than “compute”.

Another point I’d like to make about webapps is that they’ve sort of dominated thinking about our field for the last few years. The kind of people you see writing for O’Reilly Radar are almost always thinking from a network point of view, and you see a lot of people take the position that devices are useful only as a means of getting to the network. Steve Ballmer said this a year ago:

Let’s face it, the Internet was designed for the PC. The Internet is not designed for the iPhone. That’s why they’ve got 75,000 applications — they’re all trying to make the Internet look decent on the iPhone.

Obviously I disagree, but I bring it up not for easy potshots but to bolster my claim that there’s a lot of thinking out there that it’s all about the network, and only about the network.

And when you consider a speaker’s biases regarding the network versus devices operating independently, you can notice some other interesting biases. To wit: I’ve noticed enthusiasm for open-source software is significantly correlated with working on webapps. The most passionate OSS advocates I know — the ones who literally say that all software that matters will and must eventually go open-source (yes, I once sat next to someone who said exactly that) — are webapp developers. Device and desktop developers tend to have more of nuanced view of OSS… for me, it’s a mix of “I can take it or leave it”, and “what have you done for me lately?” And for non-programmers, OSS is more or less irrelevant, which is probably a bad sign, since OSS’ arrival was heralded by big talk of transparency and quality (because so many eyes would be on the code), yet there’s no sense that end-users go out of their way to use OSS for any of these reasons, meaning they either don’t matter or aren’t true.

It makes sense that webapp developers would be eager to embrace OSS: it’s not their ox that’s being gored. Since webapps generally provide a service, not a product, it’s convenient to use OSS to deliver that service. Webapp developers can loudly proclaim the merits of giving away your stuff for free, because they’re not put in the position of having to do so. It’s not like you can go to and check out the source to AdWords, since no license used by Google requires them to make it available. Desktop and device developers may well be less sanguine about the prospect, as they generally deliver a software product, not a service, and thus don’t generally have a straightforward means of reconciling open source and getting paid for their work. Some of the OSS advocates draw on webapp-ish counter-arguments — “sell ads!”, “sell t-shirts!”, “monetize your reputation” (whatever the hell that means) — but it’s hard to see a strategy that really works. Java creator James Gosling nails it:

One of the key pieces of the linux ideology that has been a huge part of the problem is the focus on “free”. In extreme corners of the community, software developers are supposed to be feeding themselves by doing day jobs, and writing software at night. Often, employers sponsor open-source work, but it’s not enough and sometimes has a conflict-of-interest. In the enterprise world, there is an economic model: service and support. On the desktop side, there is no similar economic model: desktop software is a labor of love.

A lot of the true believers disagree with him in the comments. Then again, in searching the 51 followups, I don’t see any of the gainsayers beginning their post with “I am a desktop developer, and…”

So I think it’s going to be interesting to see how consensus and common wisdom industry changes in the next few years, as more developers move completely out of webapps and onto the device, the desktop, and whatever we’re going to call the things in between (like the iPad). That the open source zealots need to take a hint about their precarious relevance is only the tip of the iceberg. There’s lots more in play now.

The Other iOS Programming Language

My latest contract project has me doing a bunch of custom work with a UIWebView: we have XHTML content that we want to render in our app, but with some fairly extensive changes to its presentation, such as paginating the content like a book, and intercepting taps on links. Given the option of using and customizing the built-in WebKit rendering, versus parsing the XHTML myself, laying it out, etc., the choice was a no-brainer.

The trick, then, is in how to extend and customize the functionality. As a long-time curly-brace application developer, my natural instinct is to impose control from the Cocoa side, perhaps by subclassing UIWebView to achieve custom behavior (although this is specifically discouraged by the documentation), tying in delegates where possible, perhaps even employing some render hackery (like using an offscreen UIWebView and then blitting its pixels into some visible view). But this really isn’t the right way to do it: for starters, it still gives you no access to the DOM, which is where nearly all the value of your HTML content is.

I suspect younger readers already know what the right answer is: insert your own JavaScript, and work inside the UIWebView. This is pretty straightforward to do — you can load the HTML source into a string and then munge it as necessary, such as by strategically adding references to your own CSS stylesheets or JavaScript (.js) files, and then load that modified source into the UIWebView, along with an appropriate base URL to help resolve relative paths. I say this would be a natural conclusion for younger developers because I suspect that most young developers start with either Flash or JavaScript, as these environments deliver immediate visual results and aren’t hard to get into (plus, JavaScript is free). Developers my age sometimes wish that computers still came with an introductory programming environment like a flavor of BASIC or HyperCard, overlooking the fact that today’s dominant starter language is included with every browser.

What makes JavaScript programming practical in an iOS app is the method -[UIWebView stringByEvaluatingJavaScriptFromString:], which does exactly what the wordy method name says: it tells a UIWebView to execute arbitrary JavaScript contained in an NSString parameter, and return the result as an NSString. You can easily try it out on a UIWebView in your own application like so:

	[myWebView stringByEvaluatingJavaScriptFromString:
		@"alert ("hello JavaScript");"];

With this door opened between the JavaScript and Cocoa worlds, you have two-way access to the DOM and how it is rendered. For example, you can pull out the web view’s selected text by writing a simple JavaScript function, and calling it from Cocoa. Or slurp it all in by walking the DOM, appending all the textContent, and returning one big NSString. Or collect all the links with document.elementsByTagName('a') and index them in Cocoa.

But don’t stop there. With the richness of JavaScript, you can employ all the tricks you see in rich web applications, such as rewriting the DOM on the fly, scrolling around programmatically and finding element coordinates, etc. Plus, since you’re only running against one pseudo-browser (iOS’ built-in WebKit framework), you don’t have to work around the incompatibilities and quirks of multiple browsers like regular web developers do.

As I’ve worked on this project, I’ve settled into a “Render unto Caesar…” strategy. Meaning that anytime I need to access contents of the DOM, or change how it is rendered, I know I’m writing a JavaScript function, because that’s what owns the web content and its rendering. For the rest of the app, it’s still Cocoa.

There are hard parts, not the least of which is that fact that I’m still pretty green when it comes to JavaScript and messing around with the DOM. Mozilla Dev Center is a very useful resource for this, despite some search and link breakage, far more so than Apple’s Safari Dev Center.

The other problem is that debugging JavaScript in a UIWebView is notoriously difficult. Unlike desktop browsers, there is no “developer mode” you can drop into in order to inspect elements, see the results of your actions, or even log messages efficiently. Worse, the smallest syntax error will break execution of your function (and major syntax errors can break your whole .js file), so there is often little recourse but to jam alert() calls into your code just to see how far the interpreter gets before quietly dying. That said, iOS Safari behaves very much like its desktop equivalent, so it is possible to do some amount of development and debugging in desktop Safari’s developer mode, or the WebKit nightly build, before switching back to the UIWebView. That said, I only do this for extreme cases of debugging — I wouldn’t want to develop a bunch of new code against WebKit nightly only to find it doesn’t work the same way on the production version of iOS.

Anyone with a browser knows how rich web applications can be, and so you implicitly know that anything you can do in Safari can also be done inside a UIWebView. Sometimes it makes sense to do exactly that.

A Big Bet on HTTP Live Streaming

So, Apple announced yesterday that they’ll stream today’s special event live, and everyone immediately assumed the load would crash the stream, if not the whole internet, myself included. But then I got thinking: they wouldn’t even try it if they weren’t pretty damn sure it would work. So what makes them think this will work?

HTTP Live Streaming, that’s why. I banged out a series of tweets (1, 2, 3, 4, 5, 6, 7, 8, 9) spelling out why the nature of HTTP Live Streaming (which I worked with briefly on a fix-up job last year) makes it highly plausible for such a use.

To summarize the spec: a client retrieves a playlist (an .m3u8, which is basically a UTF-8’ed version of the old WinAmp playlist format) that lists segments of the stream as flat files (often .m4a’s for audio, and .ts for video, which is an MPEG-2 transport stream, though Apple’s payload is presumably H.264/AAC). The client downloads these flat files and sends them to its local media player, and refreshes the playlist periodically to see if there are new files to fetch. The sizing and timing is configurable, but I think the defaults are like a 60-second refresh cycle on the playlist, and segments of about 10 seconds each.

This can scale for a live broadcast by using edge servers, which Apple has long depended on Akamai (and others?) for. Apple vends you a playlist URL at a local edge server, and its contents are all on the edge server, so the millions of viewers don’t pound Apple with requests — the load is pushed out to the edge of the internet, and largely stays off the backbone. Also, all the local clients will be asking for the same handful of segment files at the same time, so these could be in in-memory caches on the edge servers (since they’re only 10 seconds of video each). All these are good things.

I do wonder if local 3G cells will be a point of failure, if the bandwidth on a cell gets saturated by iPhone clients receiving the files. But for wired internet and wifi LANs, I suspect this is highly viable.

One interesting point brought up by TUAW is the dearth of clients that can handle HTTP Live Streaming. So far, it’s iOS devices, and Macs with QuickTime X (i.e., running Snow Leopard). The windows version of QuickTime doesn’t support HTTP Live Streaming (being based on the “old” 32-bit QuickTime on Mac, it may effectively be in maintenance mode). Open standard or not, there are no handy HTTP Live Streaming clients for other OS’s, though MacRumors’ VNC-based workaround (which requires you to manually download the .m3u8 playlist and do the refresh yourself), suggests it would be pretty easy to get it running elsewhere, since you already have the ability to play a playlist of segments and just need to automate the playlist refresh.

Dan Leehr tweeted back that Apple has talked a good game on HTTP Live Streaming, but hasn’t really showed much. Maybe this event is meant to change that. Moreover, you can’t complain about the adoption — last December, the App Store terms added a new fiat that any streaming video app must use HTTP Live Streaming (although a February post seems to ratchet this back to apps that stream for more than 10 minutes over the cellular network), so any app you see with a video streaming feature almost certainly uses HLS. At WWDC, Apple boasted about the MLB app using HLS, and it’s a safe bet that most/all other iOS video streaming apps (Netflix, Crunchyroll, etc.) use it too.

And one more thing to think about… MLB and Netflix aren’t going to stream without DRM, right? That’s the other piece that nobody ever talks about with HTTP Live Streaming: the protocol allows for encrypting of the media files. See section 5 of the spec. As much as Apple and its fanboys talk up HTML5 as a rival to and replacement for Flash, this is the thing that should really worry Adobe: commoditizing DRM’ed video streaming.

Eek! A Patent!

Mike Shaver’s blog about Mozilla forgoing H.264 support in the HTML <video> tag is getting a lot of play, predictably from the /. crowd.

It’s a shame, because it’s a childish and facile argument. Here’s the gist of it.

For Mozilla, H.264 is not currently a suitable technology choice. In many countries, it is a patented technology, meaning that it is illegal to use without paying license fees to the MPEG-LA. Without such a license, it is not legal to use or distribute software that produces or consumes H.264-encoded content.

In short, the very idea that something is patented and requires licensing is prima facie proof that it is intolerable. Going on:

These license fees affect not only browser developers and distributors, but also represent a toll booth on anyone who wishes to produce video content. And if H.264 becomes an accepted part of the standardized web, those fees are a barrier to entry for developers of new browsers, those bringing the web to new devices or platforms, and those who would build tools to help content and application development.

Yeah, can you imagine if any other technology were encumbered by patents? They’d have to pass on the costs to customers too! Imagine if television were patented, or if automobiles involved 100,000 patents… surely those products could never exist and never be affordable.

There is a case to be made against patents and intellectual property as a whole, but this blog doesn’t make it. Instead, it blithely refuses to acknowledge that we do live in a world of IP, decrying its costs as if they are out of the ordinary or unjust. Ultimately, it flees back to the intellectual dead-end of “everything should be in Ogg”, a stance so untenable that even ESR conceded it was a non-starter, four years ago.

A final irony: by refusing to support H.264, Mozilla bolsters the primary alternative for video on the web: the Flash plug-in, which is not just patent-encumbered but proprietary, available only from Adobe and only in those environments where it serves the company’s strategic ends. Shaver, to be fair, admits that proprietary plug-ins are bad too, but declines to say they’re far more bad than patent-encumbered standards. Instead, he holds out for a pollyannaish vision of completely free web video technology, naming Ogg as his moral standard-bearer, without acknowledging Ogg’s lamentable but obvious feet of clay.

Battlestar Bingo

In advance of Friday night's finale of Battlestar Galactica, here's a little game to play to see if they tie up all the loose ends:


The board is written with JavaScript from 80-some possible items, so reload if you get a bad board. Tested on WebKit and Firefox. Thanks to Michael Ivey, Robert Cooper, and Michael Stemmle for trying out the prototype.

WebKit fixed fast

I’ve been using the WebKit nightly build as my preferred browser for about a year (ever since they put in the HTML5 <video> tag support, which of course is now ubiquitous in modern browsers).

On Sunday, I found a crashing bug, filed it, tracked the bug it turned out to be a duplicate of, and as of last night’s build, the bug is fixed.

Damn, that’s fast. Thanks guys, you rock.

Java Video: Past the Point of Caring

I used to care about Java video (c.f., parts I, II, and III of an ONJava blog series about it). I don’t anymore.

It’s not just the shifting of my interest to the iPhone, it’s that Java’s powers that be have chosen their way, and it’s not something I think will work. No, change that: it won’t achieve any of the things that I care about. It may achieve the things they care about. Which, frankly, is demoing well at JavaOne.

At this year’s show, Sun announced they’d licensed some On2 codec, and showed off a “video cloud” demo of dozens of videos being played on panels being rotated about a 3D space.

People who don’t know better were impressed. Which is the point. What Sun needed to show was that it heard the concerns of Java developers about how badly the platform has lagged in multimedia support, and by showing a gee-whiz video tech demo, they mollified that crowd somewhat. Of course, this crowd of developers assumes it understands media because they all have iPods and have watched YouTube. it’s sort of like me presuming to be a security expert because I remember passwords and use login screens.

Problem is, as the problem always is with Java, Sun is talking to the customers they already have. Their solution is targeted to the developers already using Java. Were they talking to media application developers? Probably not, because their solution lacks credibility to anyone who actually works in the field.

Here, in a nutshell, is the problem. By choosing a non-standard video codec, they opt out of the economies and ecosystems that have developed around mainstream codecs like H.264 and VC-1. With the MPEG standards, for example, there is competition among encoders, as only decoding is standardized. Any product or process that produces a compliant bitstream is fair game. As a result, the bitrates needed to produce broadcast quality MPEG-2 dropped from 8 Mbps in the late 90’s to about 2-3 Mbps in the last few years. H.264 is expected to see the same evolution, driven by competition between encoders. With a single On2 encoder, Java Media Components won’t enjoy this kind of improvement. The no-brand codec also opts out of compressionist expertise.

And there’s also the question of whether the encoder is appropriate for multiple purposes, as H.264 has proven to be (albeit with so many optional features that the different profiles might well have been different codecs in another era). To my way of thinking, you generally need three functional styles of codecs, which often requires three totally different codecs:

  • One-to-many playback – cases where the content is encoded once, and played back many times. The encoding may be thought of as “asynchronous”, in that the encoding may require much more time, CPU power, compressionist expertise, etc., than the playback. Example: DVD.
  • One-to-one telephony – cases where encoder and player are more or less equals, needing a codec that can be encoded and decoded with the same resources (CPU power, time, etc.). Example: video conferencing
  • Production – cases where the content will be heavily edited and processed. Codec here needs to be scrubbable, be easy to effect and edit, not be subject to generation loss, etc.

In general, I don’t think you find one codec that does all of these things well, and the fact that you see H.264 used for both playback and videoconferencing may speak to its high level of configurability. Presumably, you’re not using B-frames for video conferencing, due to the compression-time expense and the introduction of latency.

So having said all this, the idea that one codec is going to take care of all Java Media needs is highly implausible, especially since it’s not H.264.

The other thing that’s a big gotcha with Java Media is that the HTML5 <video> tag is poised to utterly commoditize media playback anyways. Why endure the hassle of trying to get Java runtimes on end-user systems, or have your developers and designers master JavaFX, when legions of JavaScript-savvy Ajax developers are getting video support provided to them with an industry-standard tag? Even if <video> doesn’t end up with a de jure official codec, it’s highly likely that H.264 will fill that role, as browser makers can rely on QuickTime and Flash, among other libraries, to render it. And if not, the prospect of browser-sniffing and sending different <video> tags to different browsers may still be more appealing than trying to goad users into installing Java.

To the Java crowd, this looks like a perfectly reasonable way to bring modern media support for the platform. It won’t matter, and they probably won’t think to ask why.