Using Crittercism SDK to effectively find & fix crashes

This post is talking about iOS apps, but I'm sure most of it applies to Android as well...

Having a crash report framework for your app is not really optional.

While investing in the QA phase before submitting your app can find and eliminate a lot of the crashes, there will always be something you'll miss, and there's nothing like millions of potential users to help you find and fix these annoying crashes when they happen.

Crittercism is a great example of a framework that makes sure you are going to get all the needed data from your users, AND may also be very helpful during the development & QA phase (in case you are not connected with a debugger and an exception breakpoint and a crash occurred).

The framework is really simple to use and the documentation is very succinct, so I'm not going to elaborate too much about the basic integration and features (such as symbolicated stack traces for each crash), but I DO want to tell you about some of my favorite features in the framework, that got us at JoyTunes to choose and use it for our iOS apps.

1. Live Data

This is a feature that is actually useful for app analytics regardless of crash monitoring, and it behaves much better than similar features in many analytics frameworks do.

You get live stats of how many app loads and how many of them crashed (filterable by app version).
This is cool because you can use it for live monitoring of app loads when an event occurs like a broadcast push notification.

In the same category, it's worth mentioning that new crashes appear in the list (and you get an e-mail) immediately as they happen. Especially useful for the QA phase when you don't have to wait hours to view the stack trace of a crash.

2. Crash Trends

Another great app analytics related feature. You can watch graphs of your DAU, MAU, App Loads, etc.
The most useful graph is the % of app loads that crashed, with separates graphs per app version.
This is really useful to make sure that you lowered the percentage by fixing crashes in certain versions, and make sure the new features you added didn't raise the percentage...

3. Breadcrumbs

While often looking at the stack trace alone is enough to understand what is going on and why the crash happened, there will be times you will find yourself staring at the stack trace not getting what were the circumstances that caused this line to crash.

This is where breadcrumbs kick in.

Using breadcrumbs effectively will let you track problematic user paths that may reproduce a crash in your development environment.

Since we are using Flurry for advanced analytics features, we simply added a method that reports an event both to Flurry and to Crittercism as a breadcrumb, and then we call this method instead of directly calling to the Flurry API.
This way each new flurry event is automatically reported as a breadcrumb as well.

The only caveat here is that while flurry supports parameters, Crittercism does not, so if parameters are important for you to understand what the user was doing, you should simply append the relevant info to the event's name when sending a breadcrumb.

4. User Metadata

By default, Crittercism creates a guest profile for each user of your app, and lets you know what crashes are affecting the same user.

This is useful by itself, but with User Metadata you get much more!

You can assign a username, an e-mail, and every other piece of metadata to a Crittercism user, and then you can link crashes analytics to each

This is useful for many things. For instance:

- If you get a support ticket with a description of a crash, you can "Search by User" in Crittercism and get all the info about the crash the ticket is referring to.

- If breadcrumbs data was not enough to identify the problem, and you want to further investigate the user's usage in your database/in other frameworks, you can do it by the username.

- You can contact all the users that suffered from a crash and let them know of a workaround or an app update that solves it.

- You can set up custom metadata fields to help you identify crashes that occur only to specific types of users (like the Diagnostics tab in a crash page helps you identify crashes that occur only for specific device types or iOS versions...)

---

That's it for this post.
Crittercism has a lot to offer besides the features I mentioned, like an in-app support forum, so you should definitely check out their website and see everything for yourselves.

But bottom line - I think Crittercism is a great affordable framework for monitoring and fixing crashes, and you should definitely consider it for any of your apps - just make sure you make the most out of it!

Developing "Piano Dust Buster"

So... "Piano Dust Buster" - the game I've been working on in the past few months in JoyTunes - is live in the iPad App Store for two weeks now. This is very exciting to me, so I decided to write a blog post about it, sharing some thoughts about this game and how it was to develop it.

About the game

The game is a piano song game, that you can play both in touch mode and with your own real piano.
The idea is to give a tap-tap/guitar hero like experience, while being on the more educational side (e.g. piano keys are real-life sized + there's a game mode with a musical staff that you can learn reading sheet music through) + instead of playing with a toy guitar, you can play with your own real instrument, so you're actually practicing.

Here's the game trailer, so you can get a hunch of what's the game like:

For more info, visit the game's homepage: www.joytunes.com/piano
The link to the game: http://itunes.apple.com/us/app/piano-dust-buster-song-game/id502356539?mt=8

Techincal Facts

• We chose to develop the game in pure objective-c and not some cross platform framework like Adobe AIR. Mainly for performance reasons (recognizing which real piano note was played in real time) and it also allowed us to give a native look-and-feel for menus etc.
• The chosen game engine was The Sparrow Framework. The main reason for choosing this framework is because JoyTunes's existing games were written in ActionScript and Sparrow is imitating the ActionScript SDK pretty well.
  I also found it much easier to learn than the popular alternative - cocos2d.
• A major part of the game was developed in TDD (which I'm sure is not something you can say about many games in the app store).

Lessons Learned

Since this is both the first iOS app and the first game I developed officially, I learned A LOT and have a tons to share:

• A good second programmer can increase productivity in >x3 factor
  I started to develop "Piano Dust Buster" as a sole programmer, and about 2-3 months later another rockstar game developer (@noamgat) joined JoyTunes and started to work with me on this project.
  I was actually lucky enough to have known Noam previously, and we actually took a very thorough programming course together, so we also had a real good understanding of each other.
  
  The increase in development pace was unbelievable.
  In the first few weeks, we did pretty much only pair-programming. This was my way of mentoring him both on the existing codebase, and on Objective-C and development tools. The benefits of pair programming are well discussed here, and I can really relate. You would think that pairing with someone new to the technology and to the codebase would slow me down, but it really did the opposite - I found myself spending much less time on pondering what would be a good design here and what would be a good behavior there, which were taking a very large percentage of my time when working alone.
  Needless to say that when Noam got the hang of things our velocity was improving constantly. We still consulted each other about tricky decisions and could get through them quickly, and of course dividing the more straightforward tasks between us helped us finishing them much faster.

• It's amazing to work on something that you have a real passion towards
  
  My 2 greatest passions in life are software development and music.
  I felt so amazing being both a part of the musical production team and the development team of this game. Also, I could solve problems like an algorithm to locate a note correctly on a musical staff, while applying advance TDD techniques such as Uncle Bob's Transformation Priority Premise - a classic combination of my 2 big passions.
  A piece of advice: If you are developers and have an additional passion in life (like I do), when the opportunity arrives to work on something that relates to this passion, don't hesitate and take it.
  I left a really fun job at IBM in order to join JoyTunes, working more hours for less money, and I don't regret it for a second, because of all the things I learned and the amazing fun I had combining my loves while developing this game.

• Prototyping is a powerful and important tool.
  
  Requirements from a game are much fuzzier than developing a server-side application, especially the requirements from the UI layer.
  You rarely have a good idea of what is the "correct" behavior, and how things should look and interact on screen.
  Therefore, you have to just go ahead and write a lot of half-baked prototypes for different game modes and interactions, then give it to a bunch of people and see how they feel about it. You will quickly get a grasp of what's working and what's not.

• Start usability testing early
  Due to the fact that you can know if your menus and your game levels are behaving "correctly" until the players interact with them, usability testing is a crucial phase before releasing.
  For us, it was especially important to see how children of different age groups interact with the game, and how people with various piano experience level do.
  I can say we found tons of issues in our usability tests, and I actually wish we started the tests in a much earlier phase, because we had to delay our release date in order to fix all the critical usability issues.

• Don't take every criticism and feedback too hardOnce again, it's hard to define what feels "right" when playing a game, especially when it comes to a game with a lot of visual art and background music.
  You need to remember that different people have different opinions and different taste. The more usability you do, the more different opinions you are going to get. It's going to be impossible to please everybody, and it's going to be impossible for you to release the "perfect" game that has no usability issues at all.
  Heck, I'm sure if I look at successful games like "Angry Birds" from a judgmental point of view, I'll easily come up with 2 pages of things they could have done better.
  So just know your 80-20 rule, and know which feedback is critical and should be fixed immediately, and which feedback is what you are aware of and are willing to live with. You will never release otherwise.

• Analytics are super importantWhat's a better usability test than the data arriving from all the app store users of your game?
  There are a lot of good analytics frameworks out there that can help you with this task. We used Flurry.
                                                  
  In order to get useful data though, you need to make sure you collect analytics about pretty much everything, and you need to design it carefully so you would be able to extract the essence of your problematic use patterns and what's causing them from the data.
  Also, we often forget to test the analytics part of our code, since it's not something you usually look for testing in the QA phase. However, if you have a bug there when you release to the App Store (which we did), you would be stuck with the bug until your next update, which could be quite painful.

• Continuous Deployment is a pain in iOS development
  Nothing would make me happier than the ability to add a tiny feature and simply deploy it to all our users and see its impact immediately.
  Unfortunately, since releasing an update for your app includes a review process that takes approximately 1 week, continuous deployment is not an option. Also, an update has other hidden meanings like losing all you current version's ranking and reviews. This makes an update something you really need to plan carefully, which is problematic for many reasons (see any continuous deployment presentation/book).
  A solution for this issue that is only partial: Use the TestFlight API to automatically build and deploy to your up-to 100 beta testers after each commit. At least you'll get some feedback. However, what's 100 beta testers compared to tens of thousands users in the app store?

Spread the word!

That's it for now. As you can see, I had a blast working on this game, and we have tons of features and big plans for it in the future. I hope you enjoyed this post and I think I'll post some more thoughts of this kind in the future.

If you have an iPad, it would be really cool if you download the game and give it a nice review in the app store, and if you don't - it would be really cool if you would share this to someone who has :)

Xcode vs. AppCode

Some Background
(mostly some personal info about me - skip if you're only interested in the comparison):
I've always been an Eclipse fan.
Eclipse was always my favorite IDE for Java, and therefore when I started developing in Python PyDev was the obvious and excellent choice.
I tried a little bit of Jetbrains' IntelliJ and PyCharm. They both seemed very impressive, but to be honest - whole the project settings there were a little bit scary for me, plus they cost money and I was happy with the free Eclipse so I didn't see a good reason to make the move.


Upon joining JoyTunes, I started to develop in 2 new technologies for me: ActionScript3, and Objective-C.

Excuse #6 - We are such good programmers, we don't need tests!

Welcome to the 6th excuse in the Testing: Why Bother? series.

I'm going to address this excuse a little bit differently than the other ones and be more personal about it.

Reading the headline of this excuse may sound dubious to some of you, but I assure you it's not something I made up. I actually heard this excuse while I was looking for a job a while back and was interviewing for a startup company.

Something I tend to do during interviews (and I think all job candidates must do as well), is to be a little bit of an interviewer myself - asking some questions about the nature of the team I'm about to join, to see if there's a good fit to what I'm looking for in a work environment.

For me, one of these questions will always be "How do you feel about unit tests?"
I often get an answer like: "we don't really write those, but we are open for change in that matter". This is actually the common answer I get and one of the answers that I live with quite happily. However, I once got a completely different answer, that inspired some of the excuses in this series, and this excuse in particular, as I did not meet any other interviewer so confident to claim such a statement.

"Our recruitment process is one of the hardest there is. It's simply impossible to get accepted if you're not a 'rockstar', and therefore we have a team of amazing developers. This way we ensure excellent code quality and zero bugs, and therefore writing tests is simply superfluous".

While I believe it's true that a challenging job candidate screening process is an excellent way to ensure better quality products in any field and any company, I found this reasoning a bit absurd, for 3 reasons:

1) Consider the opposite situation
The first odd thing about this claim, is that by this reasoning, all mediocre developers should write a lot more tests than those 'rockstars', because they NEED those tests in order to reveal all the bugs they cause. 

However, you actually must be a decent developer in order to be successful in writing good tests that reveal bugs, or else, the writing tests is too hard excuse is probably justified in your case.

Also, developers who write tests are usually the ones who care enough and are passionate enough about software development in order to make that extra effort, and IMO that alone makes them better in what they do.
I mean, can you honestly tell me there's a development job you wouldn't hire names like Uncle Bob Martin and Kent Beck, etc. to do?

So, I might have not proved my interviewer's claim wrong by this explanation, but I think you'll agree that by looking at the argument from this angle you see it's quite paradoxical, as it's not likely that if the entire team were lousy developers the team's attitude towards tests would improve.

On a side note about this reason, there was a similar claim by Tomas in the comment of my previous post in the series. I think you'll find the discussion interesting.

2) Everyone has bugs!
I'm sure that was the immediate response for most of you.
You are simply misleading yourself if you think that the people you hired aren't going to have bugs.

We are all human, we all make mistakes, and heck - I'm sure even if we build a perfect robot programmer it would have some bugs in its software, as the definition of a bug might be simply a change in requirements, and not a human error of forgetting to deal with a null pointer.

It's true - better developers are often quicker on catching bugs. they tend to make less mistakes and get to working software faster (therefore - they are better developers), but sometimes the way they achieve that special ability of writing bug-less code quickly, is simply by testing it. Does this by definition make them worse developers? I think not.

3) REGRESSION!
If you are following my list of excuses, you should be familiar with this "excuse killer" by now.
It's not only about finding the bug now, it's about ensuring the programmer who comes after you is not going to encounter it again.
Sure, now your team is assembled from amazing programmers who know all about their code and never make mistakes. But what's going to happen when a new programmer enters the project? Even if she is a rockstar developer herself, she still might lack the understanding of what the original author of that piece of code had in mind, and therefore might make a mistake in interpretation = a bug.

So, I hope you take something from this post. If it's the idea of asking "how do you feel about unit tests" in your job interview (great ice breaker on the "do you have any questions for us?" phase), if it's understanding that better developer != doesn't write tests, or if it's just getting to know me better by a personal story about a job I didn't take.

Anyways, it would be awesome if you'll spread the word about my blog, subscribe to my feed and follow me on twitter :)

Excuse #5 - The Frequent Refactoring Excuse

Welcome to the 5th excuse in the Testing: Why Bother? series:
"We refactor our code so frequently, that the time we invest in tests just isn't worth it - they are going to change and be irrelevant anyhow"


This excuse is one of my favorites, because it's so ironic, yet surprisingly quite common.

You are constantly refactoring your code? Without tests?? How can you be certain you didn't break anything???
As I mentioned in previous posts, the number one reason for automated tests as I see it, is REGRESSION. It gives the programmer the confidence of changing a piece of code without being afraid of possible side-effects, because if change introduces new bugs - good regression tests will immediately find them.

I totally agree that constantly improving your code quality and the design is a very important phase of developing dynamic software that responds well to changes in requirements. But it's that simple - you should NEVER refactor without tests to back you up.
Quoting from Martin Fowler & Kent Beck's great book Refactoring: Improving the Design of Existing Code: "If you want to refactor, the essential precondition is having solid tests."


Furthermore, for most refactoring operations, when you use refactor-friendly IDEs, performing a simple refactoring operations such as rename, change method signature, etc. will be almost painless, as it will make the necessary changes in your tests' code as well.

"Yeah but... Still... Every time I change my code I feel like I have to re-write all my tests, and then the ROI of testing vs. debugging seems to not be worthwhile any more".
Well... If you are talking from actual experience, and these are more than just fears, you may be "doing it wrong", and should ask yourself the following questions:

1) Are my tests too much low-level?
Often, tests that you feel you need to maintain too much and change for every little change in production code, present a code smell.
Is this test passing if your implementation was with a for loop, but when you replaced it with recursion it failed? This means you were testing too low.

If a refactoring step caused a test that shouldn't care about inner implementation but only about the final outcome to fail, try re-writing the test by using your code at a higher level (e.g. test the public method and not the "helper" that should be only used internally)

2) Do I have irrelevant data in my tests?
This does not refer only to refactoring, but to a very common cause for high-maintenance test code, are tests that fail because you changed the value of data that should be irrelevant to the functionality you are checking for.
I highly recommend the following post by @jbrains for further information about what the problem exactly is and how to deal with it.

3) Are your refactoring steps too big? (Is it actually a "rewrite"?)
I often got to hear the frequent refactoring excuse referred to as frequent "re-writes". Rewrites, as opposed to refactoring, is something I suffered a lot of pain from in the past.
Instead of working your way one small change at a time, you decide to "throw all this subsystem's code away and start over".

This usually turns out to be an adventure you regret every moment of getting into: all the old bugs (and many new ones) you encountered while developing the subsystem in the first place come back, and slowly but steady, the spaghetti situation in your code reoccurs, until you decide to have another rewrite - which starts the same cycle once again.

I don't believe on rewriting working software. I believe in refactoring in small steps, making sure nothing was broken after each step by running all tests. This way - the pain of maintaining the tests now becomes a piece of cake - as you know exactly what was changed in the latest step and you can deal with it.

If you don't have tests - well... I would start by adding them, making the minimal changes in production code to make it testable, and only then getting into this whole refactoring process (well.. that's my point in this whole post).

EDIT: actually, this is a bit harsh, in some cases rewrite may be the right thing to do. Read Nimi's comment and my response for it.

In conclusion

Tests should be a prerequisite for any refactoring adventure, so this excuse is full of irony, and if you really feel the pain of maintaining your tests while refactoring, not writing tests is definitely not the solution.

Another problem this excuse reveals, is why do you feel you are frequently changing everything in your code in the first place? Should you develop a better response-to-change-in-requirements mechanism? Or maybe you don't invest enough time to ensure quality of code in the first round of writing it? Maybe if you were using TDD it would help?

Think about it...

In the meanwhile, you should make sure you read all the previous excuses in the series, stay tuned for more, and follow me on twitter.

Excuse #4 - Running the tests takes forever

So, after a long break (been focusing my time on reading about new technologies rather than writing - started a new job @ JoyTunes...) I'm back with the next excuse in the "Testing, Why Bother?" series.

"Running the tests takes too much time, 
so we never run them anyhow."




Well, this excuse is less common than the other ones on the list, and to tell the truth - if that's your case, you're in a pretty good spot - because, at least you have tests to run!

However, if you have tests and you're not running them, they obviously have little merit, so the effort of writing them goes to waste, which is definitely a shame.

So here are a few tips on how you can solve this tricky situation.

You don't have to run the tests yourself!

The word "automated" in "automated testing" is the key. It means the whole process must be automatic, not only the tests themselves.
If you don't have a Continuous Integration (CI) server that runs the tests yet, such as Jenkins, you should hurry up and setup one, because tests are just not the same without it.

If you haven't heard of continuous integration, I highly recommend you read the book:


In a nutshell, it means that for each new revision pushed into the source control management system (such as Subversion, git, ...), the CI server wakes up and starts a "build" for the project. Builds can automate very complicated things that are often done manually in the release process, and that's probably something I can write a whole set of blog posts about, but what is often done at the beginning of the CI build, is running all the automated tests in the repository, and sending e-mails to everyone involved in case the tests failed.

This way, tests that are running for too long are a much lesser pain, as you don't really have to run them manually and wait for them to finish in order to know if they passed or not.

"But I want to be able to run the tests manually as well"

That's a very good point. Tests are not only to be run automatically by a CI server after they are already in the repository. You often want to run them while you're working, from your IDE, in order to know that the latest line of code you changed didn't break any logic that was working previously.

Well, here are a few tip regarding how to speed up your automated tests:

Make sure you don't set up any unnecessary environment

Unit tests should be testing a single component separated from its context. So - if your tests have to set up 5 DB connections, open some sockets, read a bunch of XMLs and instantiate 100 objects just to run a simple functionality test on the tested component - you're doing it wrong!

If you design your code in a way that you don't have to use real environment and you can easily substitute external interfaces with mock objects (e.g. avoid evil DbConnectionManager singletons), it should be easy to save a lot of setup time for your tests.

If however, you feel you MUST use real environment for a specific test - make sure you only set it up once, in the global setup of your Test Suite (e.g. @BeforeClass in JUnit), and not before every test function...


Mock irrelevant operations that take to much time

This is pretty much similar to the previous tip, but this time I'm talking about mocking calls to some kind of 3rd party API or external interface.

Simply replace calls to the external interfaces with a mock that returns a value, without having to wait for the small 3rd party calls to finish before moving on with your test.

One interesting way of achieving this, is by taking an Integration Test that's already testing your component from a higher level, recording all the calls to external interfaces and APIs, and then replaying all the results in a unit-test of your component, that takes much less time.

There are some frameworks that may help you with this, like CaptureMock for python.

Separate the Integration Test Suite from the Unit Test Suite

Unit Tests in line for some CPU time...

This may come as trivial for a lot of you, but I've seen many violations of this principal.
As said, the Unit Test Suite should be no more than a few seconds to run, in order for us to be able to quickly run them and make sure we didn't break anything in our code.

If you have integration tests that take a lot of time - you don't need to run each time. Have a separate TestSuite for them, and make it run only in the CI server - and probably not after each build, but only before release candidates.









In conclusion
Having slow tests is much better than having no tests at all, but is still a real pain - but a very treatable one. If you have any more ideas of how to speed up your tests, don't hesitate and share it in the comments!

Stay tuned for more blog posts in the series, and make sure you read all the previous ones as well. I will try keeping 'em coming. If you haven't already, you should Follow me on twitter.

TDD and Unit-Testing for Python Developers

A few days ago, I gave a 1:30 hours long talk (in Hebrew) in my company about TDD and Unit-Testing for python developers.

The idea was to focus both on what is TDD, including a 45 minutes demo, and to give some concrete tips for unit-testing in python.

Usually when I give a talk about TDD, I use some kind of simple kata as a demo, like the prime factors kata of Uncle Bob. The problem with that, is I usually get responses like "TDD works only for calculators examples". I tried to be a little more original here and demonstrate a more real-life like example, and I tried not to come prepared too much, so I could demonstrate a genuine way of thinking about the design and letting it drive my code.
Looking back, I think this caused the demo to be a little messy - I told the audience to participate and throw ideas in the middle, and this caused things to advanced a little bit too slowly, and we didn't really get to real value by the end of the demo (as opposed to using a simple kata when you get value by the end of 5-10 tests).

For the tips sections, I took most of the material from Michael Foord's great blog and mocking framework: http://www.voidspace.org.uk/, and I think it should be very useful. So thank you for that Michael!

Anyhow, apart from the messy demo - which I think is still educational - I feel like the talk went fine, and decided to share it.

The Talk
Here's an unedited screen and sound capture of the talk, for the Hebrew speakers of you.

If you can't watch it through this shaky flash plugin, you can download it from here

Unfortunately, you can barely hear what the audience is saying so you can't catch up on the audience interaction - I wanted to edit it a bit and add subtitles of these parts but I really can't see myself getting to do so any time soon, so I decided to share it as it is now rather than sharing an edited video never :)
Also, there were some off screen interactions, unfortunately the video camera didn't work well so you wont be able to see them.

If you only want the slides (in English), you can access them here:
https://docs.google.com/present/edit?id=0AaAaoNed4IPFZGZ2cHJocHZfMWQ2NmNmdGdr&hl=en_US

Additional TDD material
Per audience request, some useful links:

Recommended books:



Additional TDD demos:

• A great one hour screencast by TekPub, hosting Brad Wilson: http://tekpub.com/view/ft_tdd_wilson/1
  It's a .NET TDD demo. What's so great about it is it's more of a real-life design example and not some calculator-like algorithmic example.
  
• The string calculator kata in python:
  requirements: http://osherove.com/tdd-kata-1/
  video: http://katas.softwarecraftsmanship.org/?p=128
  
• Some Uncle Bob presentations:
  http://butunclebob.com/files/downloads/Prime%20Factors%20Kata.ppt
  http://butunclebob.com/files/downloads/Bowling%20Game%20Kata.ppt
  
• The bowling kata as a script of a real demo:
  http://www.objectmentor.com/resources/articles/xpepisode.htm
  

Hope you found it useful...

If you did, you should check out some other posts in this blog, and follow me on twitter.

Getting started with iOS development

So... In the last couple of weeks, I've been researching iOS development - reading some blogs, going to some lectures or watching them on iTunes U, searching for answers to common questions on stackoverflow, etc.
This doesn't make me an expert on the subject, but I think if like me - you're thinking about starting with iOS development or mobile development in general, this post might save you some time and make sure you're not missing any of the important technologies available today (August 2011), and the information you need to know in order to get started.

Before You Start...

Do I have to own a Mac in order to develop for iOS?

• In a word: YES.
• There might be frameworks that will let you develop the applications without a Mac, but when you'll want to deploy on a real device, or to the app-store, you're going to need a Mac for that.
• Even if you succeeded in creating a Hackintosh on a VM (I personally had a really hard time trying to do so...), it is not recommended - as you are going to have troubles once you'll need to update your software/OS version, etc.
• I tried to find online solutions for "Mac for rent", and found macincloud.com, but 2 days after sending a request for a trial they announced they closed the site for new users.

Does it cost me money to develop iOS applications?

• The iOS Developer Program costs 99$/year. It lets you have the development environment for free, and the ability to install your apps on up to 100 personal device, and deploying them to the App Store.
• You can however, obtain xCode (the development environment) for 5$ in the Mac App Store, or get the 4.1 version comes free if you have OS X Lion. This will let you develop apps freely, and test them on the iPhone/iPad simulators. If you'd like to test them on a real device as well without paying the 99$, you'll have to jailbreak it first (which is a big No No if you're planning on deploying it to the App Store in the future).

I'm considering deploying my app to android or other platforms as well.
What kind of options do I have?

• Well, it seems as if the most popular choice is simply to re-write your app for the other platforms using the platform's native framework + SDK. This might sound surprising, but understandable once you think about the downsides of the cross-platform solutions available today.
• Notice that because both Java (Android) and Objective-C (iOS) can integrate with C/C++ code, writing at least some layers of your application in C/C++ will make it easier to port it to other platforms.
• This is also true about web applications. Nowadays, all smartphones and tablets have the capability of displaying web content such as HTML5/CSS3/Javascript. Therefore, writing your app as a web application or at least having a web container that runs web content as part of your native app, will make it available for other platforms freely, or at a very low cost. There are also some frameworks that will facilitate your efforts writing such apps, that will be discussed later on this post.
• Even though there are some downsides, there is a variety of available frameworks for developing cross-platform apps, and I'm going to discuss all the notable ones later on this post.

So, what are the downsides of using a cross-platform solution?

• Native Features: Most of the available cross-platform solutions won't let you access all the native features of the hardware. For instance, if you want to record real-time data from your microphone or camera, you'll have to write native code that does it, even if your basic framework is cross-platform.
• Performance: Like cross-platform solutions on the PC, a cross-platform solution in mobile devices comes with a performance overhead, some more than others.
• User Experience: Your UI will fail to address the special look-and-feel and the specific features of a given platform, like the 4 physical buttons that exist in Android vs. the single home button in iPhone, which will surely feel awkward to the user.

Why iOS and not Android?

• While following smule, a great music-related iOS app developers, I found an interesting post they shared about why they don't develop applications to Android:
  http://tech.fortune.cnn.com/2011/05/27/why-its-harder-to-make-money-on-android-than-on-apples-ios/
• Another thing they claimed, is that it's impossible to do real-time audio processing in Android due to high latency: "right now if you create an app for android that just routes the audio from the microphone to the speaker, there will be a noticeable delay, even on the devices with fast processors. This is a problem because if you want to make an app like 'I Am T-Pain' then it becomes impossible from a user experience standpoint to do live vocal processing. Put another way, hearing your voice come out of the speaker delayed creates a user experience that is not on par with that which we've been able to create on iOS. - this is a major problem if -- like I do -- you want to include real-time audio processing in your applications.
• On the other hand, it seems that the majority of the mobile devices in the market is soon going to be Android's: http://www.mofonu.com/2011/07/15/android-devices-130-million-and-growing-%E2%80%93-550000-devices-activated-everyday/

Native iOS Development Technologies

Some useful links to get you started

• I highly recommend Stanford's iOS development videos in iTunes U: http://itunes.apple.com/us/podcast/cs193p-student-final-projects/id395605774?i=90218598
  Which covers both Objective-C basics and the iPhone's SDK. It's from Fall 2010 and there might be newer ones in iTunes U, but I personally watched the first few videos and I am really pleased with them.
• A great blog with a variety of tutorials on iOS and game development: http://www.raywenderlich.com
• Another great source for Objective-C and iOS related tips and issues: http://iphonedevelopertips.com/archives
• If you speak Hebrew, a great intro by Hezi Cohen, focusing on objective C and xCode: http://www.youtube.com/watch?v=GVAWfqTyhcA
• iOS Dev Center: http://developer.apple.com/devcenter/ios/index.action
  You should read the iOS development guide, and all the different guidelines there. It may look long and boring, but it pays off!

Objective-C

• An additional layer to C that introduces objects
• Syntax is ugly (smalltalk like), but not something you can't get used to
• Memory management is better than C, but you still need to be very aware, unlike modern languages. However, iOS 5 will support Automatic Reference Counting (ARC), so this should alleviate the pain. (See http://developer.apple.com/technologies/ios5/)
• Has some very features of modern languages, like automatic getters&setters, nil objects that do not causes a crash if referenced, and dynamic language features - altering existing classes, invoking methods dynamically, etc.
• The iPhone SDK and all APIs are written in Objective-C (Including the UI layers), and as said before - you are probably going to write at least one layer of your application in it, so you definitely should learn Objective-C if you are planning on developing for iOS.
• A GREAT summary about Objective-C's basics: http://www.otierney.net/objective-c.html

IDE (xCode)

• Basically, it's like VisualStudio for iPhone applications. Seems like a decent tool - autocompletion works very nicely, and there are built-in UnitTests support which is great (no green bar though...)
• Comes with a set of tools, like the iPhone simulator which is a very good way to feel how your app is going to appear on the device without actually deploying it, and other important tools like a profiler, a memory debugger, etc.
• Will take some time to adjust to, and from first look - it's powerful, but lacks some advanced refactoring features IDEs like Eclipse have.
• I heard some that claimed that even if you develop in other IDEs, you'll still have to compile them eventually in xCode before submitting to Apple. However, I guess this was true prior to a change in Apple's policy regarding development tools, in September 2010.
• One prominent alternative for xCode is Appcode by JetBrains (IntelliJ dudes): http://www.jetbrains.com/objc/ - haven't thoroughly checked it out yet, so no review.

OpenGL ES

• Official website: http://www.khronos.org/opengles/
• OpenGL ES is a subset of the OpenGL 3D graphics API, designed for embedded devices, including mobile phones. A subset - meaning some of the features of OpenGL such as floating point operations might not be available, and you'll have to replace them with alternatives if you're planning on porting an existing desktop OpenGL application.
• The upside of developing in OpenGL ES, besides being a strong and popular 3D graphics engine, is that it supports many platforms. Implementations always have a platform-specific part, and the OpenGL API which is the same on all platforms for a specific OpenGL API version (which unfortunately may differ between the Android SDK and the iOS SDK). So while your code won't be 100% cross-platform, it should be easily portable, especially if you create proper abstractions between the platform-specific and non platform-specific part.
• A great set of OpenGL ES tutorials for iPhone can be found at http://iphonedevelopment.blogspot.com/2009/05/opengl-es-from-ground-up-table-of.html

Cocos2D

• Link: http://www.cocos2d-iphone.org/
• An open source objective-C based framework for easily developing 2D games for iOS and Mac. Based on OpenGL ES.
• Contains nice features such as scene managements, menus and buttons, tile map support, high score server, and many more.
• Very popular - has a large community and lots of cool games created with the framework.
• The tutorial in the official site seems very effective, and there are lots of other books and tutorials out there.

Cross-Platform Solutions

Web Based Cross-Platform Solutions

• There are a lot of platforms that let you develop native apps in html5/javascript/etc.
• This is a good solution for developers who already experienced in web, and do not wish to learn objective-c or other similar technologies.
• On the other hand, although some of these frameworks allow you access to the hardware of the devices, it is limited in comparison to real native apps. Also, there are major performance limitations, and of course there's the native look-and-feel issue I mentioned earlier.
• The most popular examples of such frameworks are: PhoneGap and Appcelerator Titanium. I must admit my research less focused on them so I can't really say how they are, but I do know they are popular and has a large community, and on the other hand - they have the known limitations mentioned earlier.
• To sum it up, here's a nice presentation by Onavo about the dilema of Native apps vs. Web Apps:  http://www.slideshare.net/onavo/advanced-ios-engineering-the-junction-talk

Rhodes Rhomobile

• Link: http://rhomobile.com/
• A bit similar to the rest of the web-based cross platforms solution, except that it's Ruby based, and should be more powerful regarding hardware capabilities.
• Personally I'm more of a python guy, but from the little I know, ruby is a really fun dynamic language.
• According to their claim, apps written with rhodes are faster than Android apps written in Java, because it's written 
• Hardware capabilities currently include access to GPS, camera (only still pictures), bluetooth, audio playback, barcode recognition, and more. Audio/Video capture is planned in future releases.
• Integrates nicely to all popular IDEs, with a built in unit-testing framework, etc.
• All in all, seems like a good option if your app doesn't require intensive graphics or performance, and you like ruby, or at least prefer it over objective-C.

Marmalade

• Link: http://www.madewithmarmalade.com/
• Until recently known as 'Airplay SDK'
• A cross-platform infrastructure in C++ - mainly targeting game development but according to them "Marmalade’s sweet spot is 'rich' apps, by which we mean any combination of: great graphics; audio processing; use of device APIs such as camera, GPS, and microphone; deep C/C++ codebase; or anything else that raises your app above the level of a simple mobile website wrapper"
• They have a plugin for the existing IDEs: either Visual Studio in Windows, or xCode in Mac - which makes it very comfortable to develop.
• The support for platform-specific code (called EDK) gives an automatic layer of abstraction that makes it very easy to write a cross-platform app that takes advantage of native features.
• There is a very impressive portfolio of games developed with this SDK, like games by Electronic Arts, or PES by Konami.
• All in all, this looks like one of the top choices for developing cross-platform applications.

Unity

• Seems like a very powerful cross-platform game engine. Has good documentation and a good community of Q&A. http://unity3d.com/
• Most of the development is done with an editor, and the engine supports scripts in Javascript, C# and Boo. There are also many native plugins in the Pro version, and you can write them yourself in c/c++ / objective-c / java, etc. (but using it will make your game non-cross-platform of course).
• In general, seems like this is a very strong option for game development, especially if you are developing 3D games for multiple platforms. However, accessing the microphone/camera of the mobile device + other advanced features will require using native plugins.

Flash CS5

• It seems that Flash is a decent cross platform solution for developing native iOS applications. Surprised? So was I. Here is a summary about that: http://www.adobe.com/devnet/logged_in/abansod_iphone.html
• Here are some nice tutorials:http://www.paultrani.com/blog/index.php/2010/11/tutorial-creating-mobile-apps-using-flash-cs5/http://active.tutsplus.com/tutorials/mobile/flash-for-iphone/
• In general - seems like a great solution if you are an experienced Flash developer, or you already have a Flash version of your application that you want to deploy it to the iPhone. The open question is: is the performance going to be good enough?

Corona SDK

• Link: http://www.anscamobile.com/corona
• Based on the Lua scripting language, should be similar to ActionScript. There's a nice blogpost regarding how to port a Flash application to Corona: http://blog.anscamobile.com/2011/01/flash-to-corona-porting-guide/
• Claims to focus on performance and from the showcase seems like there are very impressive games developed with Corona.
• Has access to many of the device capabilities, including camera, microphone, GPS, etc.
• There's a free unlimited trial, but you need to buy a subscription in order to distribute apps.

Afterword

That's about all I have researched and can share with you currently. I hope that with time, after gaining some hands-on experience, I'll have some more useful knowledge to share in this blog. So if you're interested - stay tuned: subscribe to the blog, or follow me on twitter.

I would like to welcome any comments you have about the information written here: if you feel I left some important information out, if you spotted a mistake or two in the data, or if you have a personal opinion about some of the frameworks I mentioned. Please leave your comments as a reply to this post. I promise to take them into account.

---

Edit: Thanks @philxan for mentioning I forgot about MonoTouch

• Official Site: http://ios.xamarin.com/
• Part of the Mono Project (http://www.mono-project.com/Main_Page) - a solution that allows you to write cross-platform C# application.
• Basically, it's a C# framework that integrates with the desktop Mono SDK and the iPhone SDK and lets you create native applications in C# and the .NET framework. Should be interesting mostly if you're looking to port an existing C# application for iOS, or if you're an experienced C# developer and want to save yourself the efforts of learning and adapting to new technologies.
• The IDE is MonoDevelop for OS X - looks a solid IDE, but I can't say I tried it personally.
• Cost begins at 399$ for a personal license...
• Has a version for Android (and naturally Windows Phone) as well
• Here's a good site with code samples: http://monotouchexamples.com

Excuse #3 - Isn't it QA's job to test my code?

Sorry for the delay in publishing this one guys, had been busy, but finally - the 3rd excuse of the "Testing: Why Bother?" series is finally up.

So... We are going to address an excuse that is a bit less commonly asked out loud, but I think many of us programmers secretly think it without even knowing.

I assume you stumbled at least once before across arguments like:

• "Well... I could write a test for this, but QA has to perform some heavy testing on this feature anyhow, so why bother?"
• "I don't have the environment I need to run a test for this right now, and QA have their own testing environment, it would be much more cost-effective to just leave the job of testing this to them"
• "Our QA guys are not doing anything right now anyhow... We are on a lot of pressure and have tons of new features to implement before deadline - why not speed up things a bit by skipping the testing part and leaving it to QA?"

Well, I must admit these are all thoughts that ran through my head at least once or twice.

Here are some of my key points on what I've come to learn from my experience with other programmers and even myself cutting corners about testing the code because we're on a hurry and "QA are going to test it anyhow":

The Longer you wait testing the code - the harder it is to fix it

There are different types of ways collaborating between development and QA teams.
I've seen many kinds of ways, but I think they generally fall into the following 2 categories:

1. Dev and QA teams are working separately, on separate iterations. For example: the dev team are working for 2 weeks on the current release candidate, releasing it to QA, and start working on the next iteration. When QA guys find a bug with the last released version, they report it to the dev guys, and the latter have to decide if they are going to fix it for the upcoming version or not.
   When there's a release candidate QA feels is stable enough - it's being released to customer.
2. There is only one team, made of both QA and dev personnel. Each iteration, the dev guys are working on a small feature, and as soon as they are done with it, 

Now, of course the 2nd option is more "agile", and I personally prefer it, but that's not the issue here. What I want to say - is that if you wait until the QA guys kick in until you get feedback on your feature and know if it works well. This time arrives at least after you decide you're done with the current feature in the better scenario (#2), or might even wait until the end of the iteration (#1), which might be weeks away.

That's awful. Because when the bug will be opened, you already forgot about what you did, how the code exactly work, and you are already keen on finishing the development of the next feature in the list.

If instead you have written a unit test for the feature during (or before) implementing it, you would have a much easier time fixing the bugs that were found.

Every bug might be hiding a long list of more complex ones!

When you release your product with a kind of a "blocker" bug, like a bug in installation or a crash in the beginning of the application, you are blocking the ability of QA to reach the really interesting bugs of your application.

This means that until you fix your installation bug, QA can pretty much go to the beach and catch some sun! So much for efficiency and maximizing productivity. 

If you make sure to heavily test those critical part of the application that will block the ability to use it at all, you avoid these evil scenarios.

Help The QA team focus on what they're good at

Related to the previous point...

As I see it, QA are very important even if you have 100% code coverage in your unit tests. They help you find the nasty bug that you would have never thought of testing automatically, like "your application crashes when I'm running it on Windows 2008 R2 SP1 when the USB dongle was plugged in before you disable the video camera", and additionally - they provide a more user oriented point of view developers sometimes lack.

If you keep the QA team busy with recreating bugs of basic functionality you could have easily found with a simple unit test, you are not taking full advantage of their services.

Collaborate with QA for gain of both sides

This is an important tip, especially if you feel that the pressure on development while QA have some slack.

Something you CAN do, is use QA for assistance while writing your code. You can pair with them while writing the unit tests, and even the code itself. This way - development gains the QA point of view of what their code should do and what to test for, and the QA gains the better understanding of the code and the ability to spot weaknesses in it.

Also, QA usually have a list of tests they run for each build before declaring it a stable product. When you write along with the QA guys automation for some of these tests, and include it in the test suite that runs after every build, you save them the effort and increase confidence of both sides with the stability of the product.

To sum things up

Leaving all the testing to QA will NOT save you time. It will either cause you to delay your deadlines, or simply cause you to release low quality products.

More posts in the series are coming, and some new content!

You should subscribe to this blog and follow me on twitter.

A test with no assert is better than no test at all

Sorry about the delay since my last post.
Now that I've freed some spare time, I hope to get back to the "Testing: Why Bother?" series. So stay tuned.

Anyhow, wanted to share a quick insight I had about what to do when you need quick feedback about a feature that you find hard to write a good test for, through an example.

Some Background

I was pairing with my buddy on a cool feature of our project for some time now.
The feature is basically some kind of a "Device Prettifier", that receives a local device path, performs some inquiries about it, and when its __repr__ (toString() equivalent) is called, it displays all the data it gathered from the inquiries prettily.

So, we actually test-driven-developed the mentioned DevicePrettifier, unit-testing everything by creating mock devices returning whatever we want the DevicePrettifier to do. Basic TDD+Mocking, went very smoothly and was a lot of fun.





The Tricky Part


Unit testing here was not enough for us. We mocked what the inquiries returned according to known examples we know about, and we pretty much tailor-made the responses to the logic we wanted to implement. Perfectly reasonable way to implement a feature using TDD. However, we never tested how the DevicePrettifier behaves on a REAL device, on various operating systems and environments.

So... The correct way to go about this, is to write an integration test. An integration test that uses real devices and that tests how the DevicePrettifier works on them.
The problem: without the proper testing framework that will allow us to automatically allocate hosts and real devices to test, we would have to put a lot of effort on such integration test - effort that might take us too much time, as we need to release this feature on a tight deadline.


What Could We Do?


We could have made a manual test... But then we would lose the important regression capabilities we gain when making our tests (including integration tests) automatic.

Or, we could have done something a little bit in between: An integration test with no asserts.





Are You Crazy?

"No asserts?? What do you mean? What do you test when you do that? If a human needs to go over the results of the test run - it's not automatic, and it's of no use."

OK, I knew you guys would say that. But actually there are 2 things to notice here:

1. For the purpose of testing your code for the first time - it's better to write an automatic test with no asserts than to write a manual main(). This way you have the structure of the test that you can later add asserts to more easily. Also - when you run it for the first time and manually check the output, you can quickly find the problems, and write a more specific test for them - even go back to the unit tests and alter them for this purpose.

2. Even a test with no asserts automatically tests something. It tests that no exceptions are thrown. This is a VERY important thing to test for, and -- let me tell you -- it founds a decent amount of bugs!

So, here's the test we made (code almost untouched before uploading):



We simply logged on to a few hosts of different type, ran this test, and it found the most important bugs we had. May be surprising, but really cost-effective!

Now, we can add this test to all of our continuous integration slaves, and if for instance a new type of device is suddenly connected to one of the slaves -- we'll immediately know if out prettifier couldn't parse the inquiry responses it returned. Coolness!


In Conclusion


The next time you feel like writing a complicated integration test and give up because you just don't have time, think about the "assert free" approach. It might be easier, save you a lot of time, and still find most of the bugs!

Stay tuned for more posts on the "Testing: Why Bother?" series, and follow me on twitter :).

Excuse #2 - Writing tests is too hard

Introduction


So, after talking about the famous too-much-time excuse, I am going to address the next excuse in the Testing: Why Bother? series: We don't write tests because it's too hard!

I will try to make this post a little more practical than the previous one, but because it's still aimed more to the why and not the how, I'm not going to dive too much into the "how to write tests for beginners" techniques. If you feel you need some prior background for this, I can recommend the following great books:


So... I guess in your pretty world testing is a piece of cake...

Once again, you guess wrong my friends...

Testing isn't easy at all. Writing tests is a delicate art, and can be a very difficult task, especially for beginners with no experience in it.

However, if you consider the alternative, you'll eventually figure out that writing code that works without bugs, without a single line of test code, is even harder!

One might say: "Well... Still.. It's not cost-effective! You go through all the hard work to discover a few bugs you would find out about later anyhow?"

I have 2 answers to this claim:

1. Later? How later?

If you have read my previous post, you probably remember that finding out about a bug "later" can be actually really later, like when it's already at the customer's hands. In this case it can cost you WAY more. So cost-effectiveness isn't really a valid excuse here.

2. It's not THAT hard!

After you practice a little, read some books, become more comfortable with writing the tests, you'll see it's not that bad, and it will become a natural skill of yours. 

However, there still might be several cases that you really think "this is a piece of code I simply CAN'T test". In this case - you've come to the right place! I'll try to tackle down the common difficulties you might be having.

The Common Difficulties

Here's a list of some common difficulties taken from my personal experience and of the anti-testers around me:

• The code has too much dependencies: "I need to instantiate 50 different classes to write a measly test case."
• There's a lot of environment to initialize: "Before I can invoke this method, I need to acquire a DB connection, and run 4 different processes..."
• The code is behaving in an unpredictable way: "I have 20 threads running here, I should expect a different result in each run!"
• The code involves a lot of manual operations: "Most of this logic is invoked when a button is pressed in the GUI, I can't automate a test for it!"

Lets' try to find a solution for these difficulties:



Tightly coupled code is evil! Regardless of testing!

This is true for the first 2 bullets. If your UltraUberDoesEverythingManager is really hard to test because on its constructor it receives 20 different classes, or worse: it instantiates 20 different classes by itself, that's a real code smell.


Remember, if it's difficult to test, it is difficult to maintain and use!
If you find it hard to infer the class from its context and write a good unit test for it, having to pass real instances, having to initialize a real DB connection or insert the USB dongle for the test to run, it means that later when you'll want to extract the useful piece of algorithm or reuse the objects, you are going to have the exact same difficulties. That's just bad OOP.




Decouple! Use interfaces so you can replace them with mocks.

Consider the following code:

This code will be very hard to test. We simply want to test that serializeString() stores the string in DB using the dump() method, but in order to do so, we have to start a connection to DB using the horrible ConnectionManager singleton.


Instead, consider the following:


This way, we pass an implementation of ourselves to the Serializer interface. We can use one of the many mocking frameworks (like Mockito for Java, or Michael Foord's Mock for Python), and easily test that serializeString() called our mock's dump() method.


Notice that this was an example of taking existing code and making it more testable by decoupling the classes and introducing an interface in the middle. If we had tested our code first, whether by applying full TDD techniques or simply a test-first approach, we would have already seen this design defect earlier and it would have saved us some time... 
But I'm not saying you MUST use TDD in order to test your code. Even without it, and without tests whatsoever, your code will be better if it's testable.


Preparing for the unpredictable
Well, If you remember the list of excuses, you might remember the "It's hard to tell how the code behaves each time, so I can't write a test for it" excuse. We are going to talk more about this subject there.


But for now, I'll focus on a few useful tips about multithreaded testing:

• Add sleeps + polling to make your assertions more predictable. You can use the following delayedAssert idiom:

• Multithreaded testing is possible! I guarantee that every deadlock and race-condition can eventually be isolated and tested with the proper sleeping and locking.

• Use mocks instead of some of the threads that don't belong to the scenario you're testing.

• Don't forget to have some kind of ExceptionHandler that catches exceptions thrown on other threads. You don't want one of your thread to crash and your test to keep going without noticing it.

• Search for frameworks that may help you. I couldn't find any I really liked, but there are a few of them out there.

Testing UI
Well, that's a tough one, I must admit. I must also say I'm not an expert of testing GUI. However, as tough as it can be, there are a few points to remember:

• Think about whether you REALLY need to test the UI? Or should it be enough to use decoupling so that the UI part is trivial and you already tested all the logic behind it?

• There are frameworks like Selenium that can help you. Maybe even a cool AutoIt script. Don't give up on automation simply because it's hard... They might not help you with unit tests, but it's still something you can use for sanity and regression.

• Write your own in-house testing framework. For example, if your part of your system combines VUI (voice user interface) - write a framework that injects WAV files to your system, etc. It might be a lot of work, but eventually it can really pay off, and free some QA engineer's time for some harder-to-find bugs.

Remember: High level automation is better than no automation!
Unit tests are the most effective way to test your code: They are fast to run, if they fail you know exactly the place in the code that caused it, and they make your design better.


However, sometimes unit tests are not suitable for what you are testing. 
That doesn't mean you should give up on an automated regression suite! Write tests in a higher level!

For example, you are trying to test the behavior of a windows networking related driver in the case a network cable disconnected. Unit testing this can be practically impossible in conventional matters. Instead, you should write some kind of script that turns off the relevant port of the networking switch, and tests the behavior is still OK. This might not be perfect, but it's better than no automation for the bug at all.

Are you finally finished with your testing nonsense now?

Sorry, you'll have to put up some more excuses :) The next one is going to be : "It's QA's job, and they're going to test the code anyhow".

If for some reason you decided that after these last posts you still want to read the next ones, you might want to subscribe to this blog and follow me on twitter.