Welcome to the 5th excuse in the Testing: Why Bother? series:
"We refactor our code so frequently, that the time we invest in tests just isn't worth it - they are going to change and be irrelevant anyhow"
This excuse is one of my favorites, because it's so ironic, yet surprisingly quite common.
You are constantly refactoring your code? Without tests?? How can you be certain you didn't break anything???
As I mentioned in previous posts, the number one reason for automated tests as I see it, is REGRESSION. It gives the programmer the confidence of changing a piece of code without being afraid of possible side-effects, because if change introduces new bugs - good regression tests will immediately find them.
I totally agree that constantly improving your code quality and the design is a very important phase of developing dynamic software that responds well to changes in requirements. But it's that simple - you should NEVER refactor without tests to back you up.
Quoting from Martin Fowler & Kent Beck's great book Refactoring: Improving the Design of Existing Code: "If you want to refactor, the essential precondition is having solid tests."
Furthermore, for most refactoring operations, when you use refactor-friendly IDEs, performing a simple refactoring operations such as rename, change method signature, etc. will be almost painless, as it will make the necessary changes in your tests' code as well.
"Yeah but... Still... Every time I change my code I feel like I have to re-write all my tests, and then the ROI of testing vs. debugging seems to not be worthwhile any more".
Well... If you are talking from actual experience, and these are more than just fears, you may be "doing it wrong", and should ask yourself the following questions:
1) Are my tests too much low-level?
Often, tests that you feel you need to maintain too much and change for every little change in production code, present a code smell.
Is this test passing if your implementation was with a for loop, but when you replaced it with recursion it failed? This means you were testing too low.
If a refactoring step caused a test that shouldn't care about inner implementation but only about the final outcome to fail, try re-writing the test by using your code at a higher level (e.g. test the public method and not the "helper" that should be only used internally)
2) Do I have irrelevant data in my tests?
This does not refer only to refactoring, but to a very common cause for high-maintenance test code, are tests that fail because you changed the value of data that should be irrelevant to the functionality you are checking for.
I highly recommend the following post by @jbrains for further information about what the problem exactly is and how to deal with it.
3) Are your refactoring steps too big? (Is it actually a "rewrite"?)
I often got to hear the frequent refactoring excuse referred to as frequent "re-writes". Rewrites, as opposed to refactoring, is something I suffered a lot of pain from in the past.
Instead of working your way one small change at a time, you decide to "throw all this subsystem's code away and start over".
This usually turns out to be an adventure you regret every moment of getting into: all the old bugs (and many new ones) you encountered while developing the subsystem in the first place come back, and slowly but steady, the spaghetti situation in your code reoccurs, until you decide to have another rewrite - which starts the same cycle once again.
I don't believe on rewriting working software. I believe in refactoring in small steps, making sure nothing was broken after each step by running all tests. This way - the pain of maintaining the tests now becomes a piece of cake - as you know exactly what was changed in the latest step and you can deal with it.
If you don't have tests - well... I would start by adding them, making the minimal changes in production code to make it testable, and only then getting into this whole refactoring process (well.. that's my point in this whole post).
EDIT: actually, this is a bit harsh, in some cases rewrite may be the right thing to do. Read Nimi's comment and my response for it.
Tests should be a prerequisite for any refactoring adventure, so this excuse is full of irony, and if you really feel the pain of maintaining your tests while refactoring, not writing tests is definitely not the solution.
Another problem this excuse reveals, is why do you feel you are frequently changing everything in your code in the first place? Should you develop a better response-to-change-in-requirements mechanism? Or maybe you don't invest enough time to ensure quality of code in the first round of writing it? Maybe if you were using TDD it would help?
Think about it...
In the meanwhile, you should make sure you read all the previous excuses in the series, stay tuned for more, and follow me on twitter.