Rss

Asynchronous Unit Testing in Swift: The Good, Ugly The Bad, And The

Janie Clayton-Hasz and I are working on the unit testing chapter for the still-unannounced book, and we’ve had enough fun that we decided to share a little bit of what we’re up to.

In the previous edition, I wrote a testing chapter based on Bill’s iCloud recipes project, and it was a nightmare for a couple of reasons. First, a completion handler that was supposed to be called from -[UIDocument closeWithCompletionHandler:] wasn’t, at least not in iOS 6. Second, iCloud sucks (c.f., “First”). And finally, the whole idea of testing something that takes an unknown amount of time is an interesting problem, one that OCUnit was not built to handle.

So it was really cool that Janie did the research and came back with promising results about asynchronous unit testing in iOS 8 / Xcode 6. Then we jumped into the chapter and… well, it’s not as pretty as the WWDC video would have you believe. It works, but sometimes you have to play a little dirty to get it there.



That Would Be Truly Wonderful

For the sake of this blog, I’ve written a new example, AsynchTestThing (.zip, 40 KB), which is in Swift and therefore requires Xcode 6.

It’s a tabbed app that loads web pages one of two ways. The first tab is a mini web-browser: you get a URL text field, a “Go” button, and a UIWebView. Looks like this… oh, wait, the new NDA says I can talk about new stuff but I can’t show screenshots. OK, enjoy my iPad stylus art:

layout of vc 1

Then in the second tab, there’s a navigation thing where the first view controller gives you a list of Madoka Magica characters in a UIPickerView — hey, this is Janie and me, what did you expect, fragrant baking? go watch it on Crunchyroll, Netflix, or iTunes — and a “Go” button. When you tap Go, you navigate to another view controller, which loads a web page for that character and strips out the document.title from the DOM and puts it in a label, for reasons that we’ll explain shortly. (the layout on this view is kind of messed up… it’s like IB isn’t accounting for the nav bar, or maybe I’ve again forgotten the magic call that makes iOS 7+ nav bars not be all translucent and sucky). More stylus art:

navigation between vc's 2 and 3

Run the app in Xcode 6 with Cmd-R, and the unit tests with Cmd-U.


I’m Not Afraid of Anything Anymore

OK, here’s the deal with this app, and why it’s interesting to test: loading stuff from the web takes time. So we want to verify that our web view is loading what we expect it to.

Unit testing is generally pretty straightforward: in your test classes, write methods that start with the word test, do some XCTAssert… calls in them. If you return normally, the test passes.


// always passes!
func testLuigiWinsByDoingAbsolutelyNothing() {
    
}


This Just Can’t Be Right

So here’s the problem with asynchronous testing. We start a test method, we kick off some long-running action, and we’re done. Oh wait… crap… we never tested anything!

OK, let’s tell the queue to sleep for 5 seconds! No, wait, our tests are running on the main queue, so we’re blocking the queue that’s doing the work we want done.

OK, then we’ll dispatch_async our tests. No, that doesn’t work either, because then our test… method returns immediately after the dispatch and claims success, even if the block/closure that was dispatched fails an assertion later.


I’d Never Allow That To Happen

Fortunately, this is where Xcode 6’s asynchronous testing comes in. It allows us to create XCTestExpectation objects, which are not tests but timers. We create expectations with XCTestCase‘s expectationWithDescription(), which just takes a string to describe what we’re waiting for. Then, prior to the end of the test… method, we call waitForExpectationsWithTimeout(), passing in a timeout period and a completion handler closure. This prevents the test method from exiting until either the timeout expires, or some asynchronous test code calls fulfill() on the expectation object, which unblocks it.

So the only thing we need to worry ourselves with is how to make those asynchronous calls. If you look at the “Testing in Xcode 6” session from WWDC 2014, you might notice their example uses the UIDocument class, whose openWithCompletionHandler() offers a very convenient way to execute some code — such as our test code — in the asynchronous completionHandler closure.

Gee, it sure would be a shame if we had to use this with APIs that weren’t so asynchronicity-friendly, wouldn’t it?


Miracles and Magic Are Real

OK, first off, how are we going to get our GUI loaded up and testable? With pure dirty tricks, of course. First off, we need the UIStoryboard. There are two approaches we’ve tried, and haven’t really settled on either. We can either load it directly:


storyboard = UIStoryboard(name: "Main", bundle: nil)

Or we can get the one that the simulator loads as the main app:


storyboard = UIApplication.sharedApplication().keyWindow.
             rootViewController.storyboard

Janie and I haven’t decided which of these we like better. She’s leaning towards the first, and that’s what our Twitter friends voted for too, so that’s probably going to be what we use.

Either way, the next trick is that you can load any scene from your storyboard by giving its view controller a “Storyboard ID” in the identity inspector, and then just instantiate that VC by name:


if let firstVC =
    storyboard!.instantiateViewControllerWithIdentifier
    ("FirstVC") as? FirstViewController {
  // ... do stuff with firstVC
}


There’s No Way I’ll Ever Regret It

So how do we test that the web view is loading? We have to wait for it to finish loading, and then somehow test that we got the right thing. On the latter point, let’s say that digging into the DOM and checking the document.title will suffice.

As for the waiting, UIWebView has a UIWebViewDelegate that gets notified of loading and navigation. And thanks to the fact that Swift doesn’t let us hide things in a .m file’s class extension, our test can easily see the view controller’s webView property and happily reassign its delegate. Of course, if the VC was counting on some other object (possibly itself) serving as the delegate and doesn’t expect that to change, well… maybe this is why some languages have access modifiers, like Java’s private and protected.

So, when we have the VC, we force the view to load (since we’ve yanked the VC from its usual existence in a UIWindow hierarchy), reset the delegate to our test class, and tell it to load the default URL:


  func testFirstVCWithHomura() {
    firstVCTestExpectation =
      expectationWithDescription(
      "test first VC with homura's URL")
    if let firstVC = storyboard!.
        instantiateViewControllerWithIdentifier("FirstVC")
        as? FirstViewController {
      firstVC.loadView()
      firstVC.webView.delegate = self;
      firstVC.handleGoButtonTapped(firstVC.goButton)
    }
    waitForExpectationsWithTimeout(5.0, handler: nil)
  }

Then, in the delegate method webViewDidFinishLoad(), we can assume we have a fully-loaded web page, and can extract stuff from the DOM with UIWebView‘s stringByEvaluatingJavascriptInString():


  func webViewDidFinishLoad(webView: UIWebView!) {
    // becoming the web view's delegate is a dirty, filthy
    // trick to get asynchronous testability
    let domTitle = webView.
        stringByEvaluatingJavaScriptFromString("document.title")
    let targetString = "Homura Akemi"
    let range = domTitle.rangeOfString(targetString)
    // TODO: there has got to be a better way of working with
    // Range than this!
    XCTAssert(distance(range.startIndex, range.endIndex) > 0,
      "Didn't find (targetString) in HTML document title")
    firstVCTestExpectation!.fulfill()
  }

So the delegate gets the string, looks for the expected substring, and does an XCTAssert with the substring range. Whether that succeeds or fails, we call fulfill() on the firstVCTestExpectation to make sure it doesn’t timeout, and instead let the the testFirstVCWithHomura() method exit normally.


I Was Stupid, So Stupid

Well, this is great, except for two things. It only works because web views expose their asynchronicity through the UIWebViewDelegate, and Swift is willing to let our test go gallavanting through the view controller’s properties and reassign that delegate.

What if we couldn’t do that?

That’s what the second tab is for. It’s built to be more of a pain in the ass. The picker provides a URL to the third VC, which has a URL as a public property, and loads that URL in viewWillAppear().

It doesn’t expose its web view as a property, instead getting it by looking up a tag that was set in the Storyboard. (OK, smartypants, you could still call the webViewByTagSoWeCanKeepItSecret() method, but it would take two minutes for me to take that out, so assume it’s off-limits). Furthermore, the VC needs to be the web view’s delegate so it can set the label at the top of the view. Since we’re testing the value of that label, reassigning the delegate will break the label, and the test will fail. Now what?


I Won’t Rely On Anyone Anymore

Recap: we want to test that when we set the URL on the ThirdViewController and load its view, that the label will eventually get set with text we expect (eg, the name of the character chosen in the picker), as loaded by the web view.

Well, let’s at least start by creating an expectation, setting the URL, and making the VC start loading the web view:


  func testThirdVCWithMami() {
    thirdVCTestExpectation =
      expectationWithDescription(
      "test first VC with mami's URL")
    if let thirdVC = storyboard!.
        instantiateViewControllerWithIdentifier("ThirdVC")
        as? ThirdViewController {
      thirdVC.loadView()
      let url = NSURL(string:
        "http://wiki.puella-magi.net/Mami_Tomoe")
      thirdVC.url = url
      thirdVC.viewWillAppear(false)

Now we just need to wait. How do we do that? Well, for lack of any genuinely asynchronous API, we’ll create our own. GCD’s dispatch_after will run a closure after a set period of time, so we’ll create a closure to check the label’s value 4 seconds in the future.


      // give the web view some time to load before
      //  we grab its DOM
      // TODO: figure out how to multiply a float
      //  by NSEC_PER_SEC
      let dispatchTime = dispatch_time (DISPATCH_TIME_NOW,
                         4 * (NSEC_PER_SEC.asSigned()))
      dispatch_after (dispatchTime,
        dispatch_get_main_queue(),
        {
          (Void) -> (Void) in
          if let vcTitle = thirdVC.titleLabel.text {
            let targetString = "Mami Tomoe"
            let range = vcTitle.rangeOfString(targetString)
            // TODO: there has got to be a better way of
            // working with Range than this!
            XCTAssert(
              distance(range.startIndex, range.endIndex) > 0,
              "Didn't find (targetString) in doc title")
          } else {
            XCTFail("Didn't get titleLabel.text")
          }
          self.thirdVCTestExpectation!.fulfill()
        }
      )

So that code will run 4 seconds in the future. But in the present, we’re still in the testThirdVCWithMami() function. Like with the previous test, we tell the expectation to wait for someone (the closure) to fulfill() it.


    }
    waitForExpectationsWithTimeout(5.0, handler: nil)
  }
}

And that’s basically what it takes. Using dispatch_after like this seems like it should be a general-purpose recipe to do asynchronous testing on stuff that takes time and doesn’t offer a good asynchronous API to perform those tests. It’s just not very pretty. In fact, not pretty at all. And it will be slow for lots of tests because we have to hard-code times instead of letting our test code get called right when the asynchronous process completes. But, usable? Maybe?


The Only Thing I Have Left To Guide me

Because you see, there’s one other thing that’s kicking around in my head about all of this. Tests run on the main thread, so on one hand I feel like I can just test stuff willy-nilly. But what if they didn’t? Or, given how much I use dispatch_async to put UIKit work back on the main queue, how much can I really count on setting things and having them immediately testable in the same iteration of the run loop?

What if asynchronicity isn’t the exception, but rather the rule?

I think we’d better get used to asynchronous testing, then, don’t you?

Previous Post

Next Post

Comments (3)

  1. Matt Brooks

    Have you thought about monitoring for notifications in your tests? It appears that the web view fires off a notification when it is done loading the page called: WebViewProgressFinishedNotification

    Reference:
    http://stackoverflow.com/questions/12120772/how-to-know-when-a-webview-has-finished-rendering

  2. Jim Adams

    Have you tried this with Xcode 8 and iOS 10? I am doing stuff similar to you and everything just broke with the new Xcode.

  3. Chris Adamson

    Jim–
    I haven’t tried this two-year-old code, but we are covering testing in the upcoming iOS 10 book and the expectation stuff works fine, so you can just watch for that when it’s out (you don’t have to buy the book; you could always just grab the sample code).
    –Chris

Leave a Reply

Your email address will not be published. Required fields are marked *