Rss

Radio On The TV

With the book done, and taking time off from conferences, I’ve wanted to do some coding of my own, to catch up on stuff I’ve missed out on over the last year or so. Naturally, top of the agenda is tvOS, and the various changes to the media frameworks.

So to warm up, I decided to take my old web radio project from the CocoaConf Core Audio class, and port it to tvOS. I figured that along the way, I’d also rewrite it in Swift and update some of the crustier parts of the code.

tl;dr: it works fine, but there are things I’d do differently a second time around.

Radio On The TV main UI

tvOS has many of the media frameworks from iPhone, notably AV Foundation, AVKit, Media Player, Audio Toolbox, and even Core Media and Core Video (among the few things missing is Video Toolbox, but you weren’t planning on encoding video on the Apple TV, right?). A quick glance shows they’re pretty substantial subsets of their iPhone/iPad equivalents. Probably the most obvious thing missing is capture functionality — there are no capture classes in AV Foundation, and I assume grabbing Bus 1 of the RemoteIO Audio Unit is not going to do anything. I suppose it makes sense with no obvious supported way to connect capture hardware to the Apple TV, but maybe at some point we’ll be able to pair bluetooth headsets (e.g., for gaming) and be able to capture then.

All we need for the web radio example is Foundation, UI Kit, and Audio Toolbox:

  • Foundation lets us download data from a URL
  • UI Kit provides the UI, of course
  • Audio Toolbox parses audio streams and plays them

Overview

For the web player, we can use a combination of an Audio File Stream to parse the audio as it streams into the app, and an Audio Queue to play it. We cut Audio File Streams from the book when we were on a do-or-die deadline to wrap it up, and while I thought it was limited at the time (I thought those kind of Shoutcast-style stations were on the way out), podcast-streaming clients like Overcast have proven me wrong. You can find really old versions of this code in my post from CocoaConf Columbus 2012, or the version from CocoaConf Portland 2012 where I put an AudioQueueProcessingTap on it to pitch-shift web radio (note that the Portland code got me some side-eye from Apple, so a much corrected version came out in early 2013.

Anyways, quick recap: with an Audio File Stream, you just throw buffers of data at it until it can figure out the data format of the stream, and then parse the data enough so that it’s ready to start sending you packets of audio data. These are in Core Audio’s usual format for compressed audio — a void* of data and an array of AudioStreamPacketDescriptions that map out just what’s in the big data buffer — and you could then convert these to PCM with Audio Converter Services, or write them to a file with Audio File Services.

Or, we could play them, with Audio Queue Services. The data format, an AudioStreamBasicDescription gives us what we need to set up an Audio Queue, and then as we receive packets, we can just throw them at the queue for play-out.

Aside: Yes, it would be easier to do this with an AVPlayer. In fact, we have a 5-line playground to do just that in the first chapter of iOS 9 SDK Development. But the point here is to learn the low-level bits, in an interesting way.

I made a couple of modernizations to this project from its previous incarnations. First, I switched from NSURLConnection-style downloading to NSURLSession, because jeez, it’s 2016, y’know? Also, the old version required you to know whether you were getting MP3 or AAC so it could properly hint AudioFileStreamOpen; this version uses the MIME type from the initial response, so it should be able to handle either encoding gracefully.

Swift

So my other decision was to use Swift. Actually, I’d tried to port this code to Swift 1.0 back in mid-2014 and gave up, because Swift seemingly didn’t have a means of providing the function pointers needed by so much of the Audio Toolbox API. For example, when you set up an Audio File Stream, you need two function pointers: a function to get callbacks as properties of the stream change (usually as the initial parse figures out things like the data format), and another to receive packets. Audio Toolbox also uses lots of void*, which Swift typically denigrates with types like UnsafeMutablePointerDieDieDie or whatever.

It can be done in Swift 2.0. Should it? That’s another question entirely.

Let’s start with this bad boy, the function that creates an AudioFileStreamID:


public func AudioFileStreamOpen(inClientData: UnsafeMutablePointer,
    _ inPropertyListenerProc: AudioFileStream_PropertyListenerProc,
    _ inPacketsProc: AudioFileStream_PacketsProc,
    _ inFileTypeHint: AudioFileTypeID,
    _ outAudioFileStream: UnsafeMutablePointer) -> OSStatus

Look horrible? Well, maybe the C declaration is friendlier:


OSStatus AudioFileStreamOpen ( void *inClientData,
    AudioFileStream_PropertyListenerProc inPropertyListenerProc,
    AudioFileStream_PacketsProc inPacketsProc,
    AudioFileTypeID inFileTypeHint,
    AudioFileStreamID _Nullable *outAudioFileStream );

The idea here is that you provide a single state pointer to be provided to the callbacks (inClientData), function pointers to be called for properties changing and packets being received, a hint about the data format if you can provide one, and a pointer to receive the created AudioFileStreamID (we have to do an in-out dance here, because the return value is reserved for an error code of type OSStatus).

Thinking of this in C is natural. In Swift, not so much. For starters, how do you even create an UnsafeMutablePointer? The state we need to keep track of (the data format, whether we’ve received enough data to start the queue, etc.) is so simple, it’s tempting to use a struct. But we need pass-by-reference semantics here, so a class makes more sense:


class PlayerInfo {
    var dataFormat : AudioStreamBasicDescription?
    var audioQueue : AudioQueueRef?
    var totalPacketsReceived : UInt32 = 0
    var queueStarted : Bool = false
    weak var delegate : PlayerInfoDelegate?
    var state : PlayerState = .Initialized {
        didSet {
            if state != oldValue {
                delegate?.stateChangedForPlayerInfo(self)
            }
        }
    }
}

Callbacks

Then there are the callbacks. How do we do these? In C, these are typedef‘ed function pointers. You would then write a free function with that signature and provide the name of the function to AudioFileStreamOpen. In Swift, there’s a typealias, like this one for AudioFileStream_PropertyListenerProc


typealias AudioFileStream_PropertyListenerProc = (UnsafeMutablePointer,
    AudioFileStreamID,
    AudioFileStreamPropertyID,
    UnsafeMutablePointer) -> Void

Notice what this is? It’s a closure. All you have to do is provide a closure that receives those parameters and returns Void. You could bust out the curly braces right in the middle of AudioFileStreamOpen and declare your property- or packet-handling code right there, as the value of the parameter. Of course, that would look like ass, so you should probably use a method, right?

Actually, not so much. While it’s highly tempting to write a closure inside your class and give it a name (making it a property of the class), you can’t actually capture any state (like your object’s properties) for a closure like this.

Error when accessing a property of self inside a data proc closure

So you end up being just as well off writing your closure like it’s a free function, but give it a name, so it’s now a global variable. Whee!


let streamPropertyListenerProc : AudioFileStream_PropertyListenerProc = {
    (inClientData : UnsafeMutablePointer,
    inAudioFileStreamID : AudioFileStreamID,
    inAudioFileStreamPropertyID : AudioFileStreamPropertyID,
    ioFlags : UnsafeMutablePointer) -> Void in
  // very C-like innards omitted
}

Once the callback closures are written, or at least stubbed out, we can actually make the AudioFileStreamOpen call:


err = AudioFileStreamOpen(&playerInfo,
    streamPropertyListenerProc,
    streamPacketsProc,
    streamTypeHint,
    &fileStream)

Notice how we have to use the & character for the UnsafeMutablePointers.

It’s also a little weird inside the closures when we receive that inClientData object. To access the void*, we use Swift’s memory variable, like this:


let playerInfo = UnsafeMutablePointer<PlayerInfo>(inClientData).memory

And no, I never would have known to do that had I not seen it in AlesTsurko’s port of the Learning Core Audio book code to Swift 2.0. So, thanks again for that!

So… Your Point

In the end, this code does work (although I still have some trouble getting the button in the split view’s detail pane to update appropriately). You can check it out at github.com/invalidstream/radioonthetv.

Having gone through this one time, I don’t think I would write Audio Toolbox code in Swift again. It calls for weird second-guessing of how C maps to Swift, and it ends up being deeply unintuitive even if you’re experienced calling Audio Toolbox from C. I think it would make a lot more sense if you’re going to write a lot of Audio Toolbox code to go ahead and do that in C (or C++, or maybe even Obj-C), and really wall it off from the rest of your Swift code, so you only need to make a couple function calls from Swift. Going the other direction is tricker; if the audio code needs to call back to your UI, maybe use NSNotifications, or write just enough C-to-Swift bridge code to support a delegate scheme or completion-handler closures. But at any rate, making a lot of calls into Audio Toolbox directly from Swift really doesn’t feel good at all.

Render unto C what is C’s, and render unto Swift what it Swift’s, I guess.

Leave a Reply

Your email address will not be published. Required fields are marked *