Rss

Archives for : carbon

What’s New, Blue Q?

One-time self-described “World’s Greatest Compressionist” Ben Waggoner posts a pointed question to the quicktime-api list:

http://www.apple.com/macosx/what-is-macosx/quicktime.html

What I’d like to know is if QuickTime X is going to be available for Windows and older versions of Mac OS X.

It’s an important issue, because despite iTunes’ insistence on installing QuickTime on Windows, the future of that product seems completely unknown. For years, every question I’ve seen about the future of QuickTime on Windows has been met with absolute silence from Apple. Yeah, I know, “Apple does not comment on unannounced products,” and all… Still, Apple has left this technology in limbo for a remarkably long time. I recall asking ADC reps about QuickTime for Windows back at Leopard Tech Day Atlanta in 2006, as I was considering calling it from Java with JNI, and (as previously noted), I got no reply at all. And every other public question I’ve seen about the future of QuickTime on Windows has gone similarly unanswered, for years.

Smell that? That’s the scent of Abandoned Code Rot. We got that from QuickTime for Java for a few years before they managed to finally deprecate it (though they apparently haven’t gotten the message out).

It wouldn’t be too surprising to see QT for Windows fall by the wayside… Apple probably cares more about the popularity of its favorite formats and codecs (AAC and H.264) than of the QuickTime APIs and QuickTime’s interactive features like Wired Sprites that have been clearly and unequivocally beaten by Flash.

But if that’s true of Windows, is it also true on the Mac? QuickTime developers are right to be a little worried. The old C-based QuickTime API remains a 32-bit only option, intended to be replaced by the Objective-C QTKit. But in the four years since its introduction in Tiger, QTKit has only taken on part of the capabilities of the old QuickTime API. With Leopard, you could finally do capture and some significant editing (e.g., inserting segments at the movie or track levels), but raw sample level data was unavailable for any track type other than video, and some of the more interesting track types (like effects and especially tweens, useful for fading an audio track’s volume between specific times) are effectively useless in QTKit.

With Snow Leopard, the big news isn’t a more capable QTKit API, it’s QuickTime X. And as Apple’s QuickTime X page points out, QTX is all about a highly-optimized playback path (using decode hardware if available) and polished presentation. Great news if you’re playing 1080p movies on your computer or living room PC, not so much if you want to edit them: if you want to edit anything, you’re back in the old 32-bit QuickTime (and the code is probably still written in C against the old APIs, given QTKit’s numerous limitations). You don’t see a 64-bit Final Cut Pro, now do you? (BTW, here’s a nice blog on that topic.)

When you all install Snow Leopard tomorrow and run the QTX-based QuickTime Player, you’ll immediately understand why the $30 QuickTime Pro (which bought you editing and exporting from the Player app and the plug-in) is gone. Follow up in the comments tomorrow (after the NDA drops) and we’ll discuss further.

If I were starting a major new multimedia project that wasn’t solely playback-based — imagine, say, a podcast studio that would combine the editing, exporting, and publishing tasks that you might currently perform with Garage Band, iTunes, and FTP — I would be very confused as to which technology to adopt. QuickTime’s cross-platform story seems to be finished (QTJ deprecated, QTW rotting away), and everything we hear on the Mac side is about playback. Would it be safer to assume that QuickTime doesn’t have a future as a media creation framework, and drop down to the engine level (Core Audio and Core Video)? And if not QuickTime… then what?

Oh, and as for the first question from the quicktime-api thread:

… How about Apple throwing us a bone as to what QuickTime X will offer those of us that use QT and QTSS?

From what I can tell, Apple has all but ditched QTSS in favor of HTTP Live Streaming, supported by QuickTime X and iPhone 3.0.

Opt-in Complexity

Last month, Erica posted a blog heralding the introduction of AVAudioPlayer in iPhone OS (and SDK) 2.2. She writes:

When the SDK finally did roll around, its Audio Queue approach to handling audio playback and recording proved to be an extreme disappointment. A quick glance through the Audio Queue Services programming guide reveals both the power and complexity of the service. Involving pages of low-level programming just for even the simplest audio requests, Audio Queues are perfect for serious performance-driven developers but lack the easy-to-use hooks that Celestial had provided. With Celestial, you could load a URL and then just play it.

Erica makes an excellent point here that gets overlooked: Audio Queue Services is powerful, as well as complex. Granted, with audio, we have something of an 80/20 scenario: presumably, about 80% of the iPhone developers with any use for audio need only about 20% of the Audio Toolbox’s functionality (namely, playback, volume and pan controls, and maybe level metering). So they’re probably very happy to have AVAudioPlayer.

But what about the other 20%? There’s a group of audio developers for whom simple playback is not enough. These are the guys and gals who want to:

  • Stream audio from the network
  • Pick out Shoutcast metadata from said stream
  • Apply effects
  • Inspect the audio format
  • Inspect metadata
  • Edit
  • Convert between formats
  • Perform monitoring other than peak/average power level
  • Perform arbitrary DSP (e.g., FFT frequency matching for a Karaoke Revolution / Rock Band type game)

Now how are you going to design an API to make them happy, while not drowning the basic developer with a hundred method signatures they won’t be able to make heads or tails of?

Intriguingly, Apple’s APIs on the Mac and iPhone largely don’t even try. Instead, the complex stuff gets segregated down to the lower levels of the SDK stack — Core Media and Core Services — while Cocoa and Cocoa Touch’s higher-level abstractions provide the most broadly used functionality.

In the case of audio, that means the developer with simple playback needs can stay in Obj-C and use AVAudioPlayer and not worry about things like latency or effects. When he or she is ready to opt in to more complexity, the first step is to use the C-based Audio Session API to describe how the app interacts with the rest of the system (can it mix its sounds with music playing from the iPod app, for example… does it want to be notified when the output path chances, like when the user removes the headphones, etc.). And if the developer needs more power, then they choose complexity and move on to Audio Toolbox (or perhaps even Core Audio… a DevForums thread and a blog by developer Michael Tyson report extremely low latency by using the RemoteIO audio unit directly).

This isn’t just true of the media APIs. You also see it in Foundation versus Core Foundation. The first chapter I did for the Pragmatic Programmers’ iPhone book was an omnibus I/O chapter (which later became separate chapters on file and network I/O), and while working on the networking portion, I wrote an example that used Cocoa’s NSHost class and NSStream‘s getStreamsToHost:port:inputStream:outputStream: method. It worked fine on the simulator, but started giving compiler warnings when I finally got my certificate. Search for the method in the documentation and switch between the Mac OS X and iPhone Doc Sets to see the problem: NSHost and getStreamsToHost:port:inputStream:outputStream: are not part of the public iPhone API (a hint of the reason why is on DevForums). Hilariously, it was only after I’d gone on to rewrite it with the lower-level, procedural-C CFNetwork that I decided to take a step back and say “you know what, the Obj-C URL Loading System is going to be enough for 80-90% of our readership’s networking needs.” Again, the functionality of opening a stream to an arbitrary port on an arbitrary host is there, but if you’re the 1 in 10 developers who really really needs to do that, then you’re going down to CFNetwork and using something like CFStreamCreatePairWithSocketToHost().

Need time zone awareness? NSTimeZone is your friend. Need to know every time zone that the device supports? Get to know CFTimeZoneCopyKnownNames(). Again, a niche-ier feature lives down at the Core Foundation level, and isn’t wrapped by an equivalent call in Foundation, though it’s easy enough to switch to procedural C and make the one-off lower-level call.

It’s an interesting trait that the Mac and iPhone stacks work this way, opting in to complexity and keeping the higher-level APIs sparser and simpler, and you have to wonder whether it’s a conscious design decision or a happy accident. After all, a key reason to put so much functionality in the lower-level procedural-C layers — aside from performance benefits from not having to do Obj-C message dispatch — is that these C APIs can be called equally easily from Carbon or Cocoa apps. But of course, the whole idea of Carbon/Cocoa compatibility is irrelevant on the iPhone, where Carbon is nonexistent. In a purely iPhone world, the only reason to have the complex stuff be C-only is to move the tricky, nichey, sophisticated stuff out of the way, optimizing the Obj-C APIs for the most common uses.

Advantage: it does make Cocoa a pleasure to work with. Disadvantage: non-trivial apps are almost surely going to need to make these low-level calls sooner or later, and switching between Obj-C and procedural C on a line-by-line basis takes some getting used to. Still, making the complex stuff an opt-in ultimately makes the SDK both more approachable and more interesting.

[Cross-posted to O’Reilly’s Inside iPhone]

Product versus Process

Daring Fireball, on rumor that Apple’s Finder App to get Cocoa Rewrite for Snow Leopard:

Everyone out there with a stiffy for the “rewritten in Cocoa” Snow Leopard Finder needs to get a grip. Cocoa is just an API. It is not some sort of magic technology where you just sprinkle a ton of square brackets in your source code and you instantly get a better UI.

I don’t think that’s the big deal. I think the big deal is that with Apple finally facing up to a really big, nasty Carbon-to-Cocoa rewrite, they’re either going to appreciate the pain they’re insisting that their partners go through with their legacy products (Word, Photoshop, etc.), or (more optimistically) they’re going to develop tools and techniques as part of their internal transition, and make those available in some future tool or SDK.

Granted, this is the sound of me clinging to a soon-to-be-wrong prediction that Carbon would get deprecated this year:

Apple has some still-viable pre-OS X apps (iTunes anyone?) that are presumably Carbon, and with the Intel transition done, I wonder if they’re not spending the first half of 2008 converting those to Cocoa, developing needed migration tools along the way, with the intention of rolling into WWDC 2008 able to say “it’s not that hard, we did it, here’s how, and here’s stuff to help.”

WWDC wishlist and predictions

About this time next week, I’ll be queued up on 4th Street (or Mission? or worse?) for the WWDC Stevenote. Speculation about what will be announced is sure to be a hot topic on the Mac blogs this week, so here are some things I think we will see, and things I’d like to see announced at WWDC:

Predictions

  • Deprecation of Carbon – I’ve already made my case for this one. With Carbon not going 64-bit, and all of Apple’s hardware already 64-bit, formal deprecation is just another shoe that must drop eventually. How they handle it is another matter entirely… has Apple already Cocoa-ized iTunes and Final Cut, and if so, how did it go?

  • Mac OS X 10.6 announced/previewed, and Intel-only – I predicted 10.6 would be Intel-only back in 2005, and though I got some hate (see my follow-ups), it’s impossible to imagine Apple including PowerPC support in an OS that’s probably at least a year off, and therefore four years after the last PPC Macs shipped.

  • A rebranded .Mac – Not my prediction; I ‘m just nodding my head in assent with the rest of the Mac community, noting the research reported by the likes of Mac Rumors and Daring Fireball. Pity if it’s true, though. I have long disliked .Mac, and don’t see a relaunch changing my mind about it. I’m not sure that Apple should be in the service business (don’t we always tout how they’re a hardware company?), especially when there are more, better, and cheaper or even free alternatives to the .Mac services.

    The real annoyance is that useful functionality is obviously omitted from Mac OS X or iLife in order to provide that functionality on a paid-subscription basis through .Mac. Consider that to sync mail, bookmarks, and iCal events between multiple Macs, you have to pay for .Mac and send your data through their servers; it’d be more elegant to keep the data within your LAN by just letting the Macs discover each other with Bonjour. And for the savvy user, Apple’s nefarious skullduggery! doesn’t seem worth the price: why not just use IMAP Gmail and put events on Google Calendar, and then sync by making all of your Macs be clients of the web service?

  • A 3G iPhone – Well, duh.

  • iPhone nano – The original iPod established buzz at a high price, then offered a less-expensive version (first the mini, later the nano and shuffle) that more people could afford. Apple can make their 10 million iPhone goal easily if they have a model that goes for $199. Question is, what do you take out to get there?

Wish list

  • A new Cocoa Touch device – Depending on how hard Apple wants to push, the iPhone OS could be the basis of many new lines of products, such as the Wii-like Apple TVii I proposed a while back. But imagine whole new devices, and how they’d work with iPhone OS. I’d love to see an automobile information system powered by iPhone OS: speakerphone, mapping/navigation, audio system, a separate rear screen for movies and other video, all wifi-enabled so your car syncs your podcasts while it’s parked in the garage overnight. If I were GM, Ford, or Chrysler, I’d throw whatever money I had left at Apple to get this in my company’s vehicles, as soon as possible (since every article I ever read on the Detroit Three says the cars we actually want from them won’t be out for another three years).

    One reason I don’t see this happening is focus. In a Fortune interview, Jobs stressed the importance of saying no to good ideas, so Apple can instead focus on a small number of key efforts (compare to, say, Sun, which says “yes” to pretty much everything, but puts insufficient resources behind them, hoping that 100 uncoordinated 3-person teams will eventually produce something revolutionary). So my gut feeling is that they’ll hold off on cool new iPhone OS devices for another year until iPhone itself is truly established.

  • More editing APIs in QTKit – Since we can’t use the classic C-based QuickTime API in 64-bit apps, QTKit is obviously the future of QuickTime. I guess I expected that by now, QTKit would offer more of the the advanced functionality of the API it is replacing. Editing in QTKit is still primitive — you get copy-and-paste, but not management of the edit list, or track modifiers like tweens. I don’t see how you write something like Garage Band or Soundtrack’s rubber-bands for level and pan adjustment with QTKit, short of putting together your own “media document” abstraction to map modifications in pan or volume to times in the media (perhaps skipping QT and working with CoreAudio instead?), and that’s kind of the point of QuickTime. If QuickTime’s going away someday, QTKit really needs to add a lot of missing pieces.

  • A Mac Apps Store in iTunes – Granted, I’m presuming the success of the iPhone SDK app store, but for a lot of indie developers, the value-proposition is highly appealing: give Apple 30% of your proceeds and you don’t have to worry about licensing, versioning, distribution, hosting download servers, etc. Plus, the store is going put your app’s description in front of more people than an indie ever could. So… wouldn’t this be awesome for the indie Mac developer, too? Wouldn’t it be great for users to be able to try, purchase, and maintain smaller and simpler Mac apps through iTunes, just like they do their music, movies, and iPod games? Please tell me Apple is at least thinking about it; I would probably start writing my own commercial Mac apps if this were available.

I might have more wish-list items later, but this is a start. I’m also interested to see what the rest of the Mac community is watching for.

Nefarious Skullduggery!

Interesting story about how Adobe won’t be able to get Photoshop 64-bit on Mac for the current release, and are looking at a difficult effort porting it to Cocoa. Daring Fireball has an extensive analysis of the technical issues and how things have played out. My bet, posted in January and still on the table, is that Apple will deprecate Carbon at WWDC and roll out migration tools, based on their own experiences as they migrate apps like iTunes and Final Cut.

Still, the other thing that’s popping up in some quarters is that the unavailability of a 64-bit version of Carbon is part of a scheme to “force” developers to use Cocoa. Even if true — why wouldn’t Apple rather have one application framework to enhance and test rathter than two? — this is hardly an issue of coercion.

What it really evinces is the lazy cynicism that all big tech companies are evil, so just like we bashed on Microsoft when they were the top dog, it’s time to bash Apple now that they’re successful. Apple’s decisions can no longer receive the benefit of the doubt, cannot possibly be the product of rational decision making or allocating limited resources. No, every time they backpedal on Carbon or don’t ship a Java 6 implementation, it’s prima facie evidence of nefarious skullduggery!

In fact, I’m adding a skullduggery! category right now to track blog entries about stuff like this.

Over on java.net, Fabrizio — who was constantly bashing Apple for not shipping JDK 6 until I guess he tired of it — typifies this line of thinking:

It’s also appalling to me to learn that strategic software manufacturers such as Adobe […] don’t get early warnings from Apple about its close technologies such Carbon and Cocoa.

Of course, if Apple did only its strategic partners about major OS strategies, and left all other developers out in the cold, we’d have a bunch more posts about the nefarious skulduggery! of that.

This makes me understand even better how important is to work with open technologies.

Yes, because open-source projects have such consistent and reliable software roadmaps, and will surely be the basis of Adobe’s products going forward.

Speaking of which, while praising Java for being 64-bit, it’s worth noting that neither Fabrizio nor anyone else thinks for a second that Java would be a viable technology for Photoshop. Even if Adobe has to do a massive rewrite to go to Cocoa, and just doing it in Java instead would give them 64-bit capability and an insta-port to every desktop platform, even us Java developers don’t even consider the idea of major apps being written in Java anymore.

A 2008 Prediction: Carbon deprecated

So let me just toss this out there as a prediction for the new year: I think this will be the year that Apple lays out a road map to move all Mac OS X applications off of Carbon.

The hints are obvious and substantial, not the least of which is the fact that Carbon isn’t supported in 64-bit mode. But moreover, Apple has always preferred Cocoa, going all the way back to the “NeXT takeover of Apple”… Cocoa (née NextStep) has generally gotten the spotlight in WWDC keynotes and other developer communications. Carbon was offered as a migration path for Mac OS 9 apps to move to OS X and pick up modern benefits like protected memory, but remember: it was primarily meant as an upgrade path. It might not have existed if not for the big app makers balking at the thought of total rewrites for what was then called Rhapsody, especially given Apple’s poor outlook for the future in the late 90’s.

It’s also worth noticing that a lot of Carbon has already gone away. Some of the biggest and most significant libraries that made up Carbon are deprecated. QuickDraw was deprecated for 10.4 (Tiger), and Sound Manager has been deprecated in 10.5 (Leopard). So Carbon code-bases are probably already seriously in play.

The win for Apple is obvious: they’d have much to gain by only having one primary application framework rather than two (POSIX and Java aren’t “primary” for the purposes of this consideration). Less code to write, less code to test, less duplication of effort.

Thing is, rewriting Office or Photoshop or anything else for Cocoa isn’t likely to be a lot easier now than it was when Microsoft and Adobe first balked at the idea almost 10 years ago. Or is it? The difference now is that Carbon apps should have moved by now from QuickDraw to Quartz, from Sound Manager to Core Audio, from FSSpecs to modern equivalents… there might be a lot less work to do. The use of C++ is tolerated in Cocoa (in the form of Objective-C++), so app logic doesn’t necessarily need to be rewritten with “those crazy square braces”.

Apple has some still-viable pre-OS X apps (iTunes anyone?) that are presumably Carbon, and with the Intel transition done, I wonder if they’re not spending the first half of 2008 converting those to Cocoa, developing needed migration tools along the way, with the intention of rolling into WWDC 2008 able to say “it’s not that hard, we did it, here’s how, and here’s stuff to help.”