Debug Whack-a-mole

 General, Projects, Software Engineering  Comments Off on Debug Whack-a-mole
Aug 122017
 

Have you ever been in this situation?  You’ve been on a project for months, you’ve poured your sweat and tears into code, design, documentation, and now it’s finally time to see something work.  You flip the switch, you run your program, and it has a bug.  No big deal, bugs happen!  You fix it, it takes a few minutes, maybe an hour, and you are ready to go again.  You flip the switch, you run your program, and it has a bug.  Well, at least it’s in a different spot this time, you made it slightly farther in the application this time.  You fix it, this one takes a couple hours, it’s nested down a few layers.  Now though, it’s fixed, the time is here, time to use your glorious creation.  You flip the switch, you run  your program, and it has a bug.  Hmm, you didn’t get quite as far as you did last time.  Was this where your first bug showed up?  You start to dig in, but this looks eerily similar to the bug you first fixed.  Oh well, fix in, only took a few minutes, and it’s that time again.  You flip the switch, you run your program, and it has a bug.  Ok,@#%!, seriously.  It got farther this time, something new, something to start digging into.  Fast forward a little bit, the next morning let’s say, and you’ve knocked out a couple other bugs but you are finally actually using your program.  You flip it on once more to use it, and bam bug….does that bug look like our very first one again?  How, in the name of all that is good in universe does this keep happening?


If this has ever sounded like you, or maybe sounds like your right now, let’s talk about why that might be happening and what there is to be done about it.  I know it’s something I will run across on occasion, especially in my earlier years of programming. It is especially bad when someone would give me a stack trace or I would see an error log, and I immediately jump in fix it, deploy it, turn it back to a QA for testing.  It’s what I like to call “Debug Whack-a-mole”

.

Can we find some common times or reasons that this tends to happen?  Personally, if I open code and make a fix based on an error report or log, without then running the program and checking the result, it happens probably more than 50% of the time.  If I am continually iterating over a large project that I am build from the ground up, and am just fixing bugs as I come across them…well, I’m not sure.  Why am I not sure? Well, often times I won’t write these bugs up.  I will see them as I am stepping through code, or I see something look off in the app, so I just put a fix in and move on.  So, how can I remember if I’ve already whacked this particular mole already?

From this, you probably are coming to the same conclusions I am.  There seem to be two big things that stick out, documentation and testing.  How can these be improved?  Will these actually help? What if these things aren’t required by your company, or this is just a personal project you are working on and formal documentation isn’t required?

image

I don’t want us to fight, I know there are some that hear ‘testing’ and ‘documentation’ and are ready to hurl things around their office, cubicle, nook, etc, but hear me out.  I won’t give you the normal party line for why it’s important and that it’s best practice, I just want to say one thing.

By not testing, by not providing ample documentation, you are wasting your own time.  There, shots fired.  I know, I really do, I get it.  You want to write code, we all do.  You write to sit down, listen to some music, crank out code, be a rock star.  Whether it’s work or personal projects (perhaps even more so for personal work), you need something to help you remember how things work, what your original intentions were, and to save yourself from, well…yourself.  You don’t future you to make an innocuous bug fix, breaking your baby!  Maybe, we can have a better discussion now.

Friends again? Great! I want to share what I have found has helped me, the most, to combat this insidious plague.

1 – Commit often

In the new age of git (and probably other code repositories), the mantra is “commit often”.  Why? What does this do for us, what good do we get out of it.  Aside from having very small unit of work that we can roll backward or forward, it also leave behind a great paper trail in your repo history.  It becomes like a stream of consciousness, change log.  Any time you are wondering, did I see this bug before, go look at your history.  Is there an entry that mentions something similar?  If so, it’s a great place to look. 

Image result for rockstar meme

Now, we are at the same cross roads as before.  This advice sounds great, you probably hear it a lot, but how does this help me Mr/Mrs RockStar Engineer?

First, small commits with comments, improve your overall documentation.  Even without comments or documentation in your code itself, you now have a small change set delta with a very pointed commented as to what you were doing, what you were fixing, or why you were making this commit.  Multiply that by dozens of small commits a day across weeks and months, and you have a very verbose and complete

Second, it makes sure you never loose work.  More than once, back in the SubVersion (SVN) days, I would have a days worth of work checked out on my machine.  This doesn’t seem too bad, until one harrowing day when I was walking around with my laptop and it fell (buttered toast style), landing flatly on the screen and keyboard.  When I picked it up, all seemed fine, the machine still ran, nothing to worry about.  Until, about 5 minutes later, when WindowsXP let me know that its delayed writes to disk were failing. Oh god, oh god no.  The disk was ruined, it later turned out that I could not even get an external hd case to help me recover.  I lost a full days worth of work.  It was devastating for me, mostly because it was just code for a personal project.  I didn’t HAVE to work on it, and never did re-write any of what I lost.  That side project died that day.

Lastly, it also makes it easy for you back out changes that cause problems. I can’t be the only one to go through a day of refactoring or a few days worth of cranking on code only to find that things have are imploding?  Maybe something less dramatic? Maybe, You’ve made a few bug fixes and run some tests on it, but somehow something big has broken deep in your code.  All of these and so many more can be fixed by rolling back some of these super small changes you’ve been making.

2 – Unit Tests

I am not a TDD evangelist, I probably haven’t given TDD a fair enough shot, but I do believe whole-heartedly about unit testing your code.  Whenever possible, break you code down into small parts that you can write some test around.  It can be hard to start doing and sometimes feels like you’re doing too much or not enough.  If it makes you feel any better, I often get the same concerns when I’m writing them too.

The guidelines I usually give my students  and mentees (and the bare bones that TDD would espouse), is that every function should be tested (having a hard time knowing what your functions should really be, try writing/drawing out what your program should do) .  Write your classes so that data can be ‘mocked’ up, even if the algorithms/functions to do that are hidden away from you main consumer (interfaces are great way to hide many of these).  When you have classes and functions that can be tested individually, write down checks for all the things you would be checking for manually.  I have a notorious bad habit (bad may be a strong word) of stepping through parts of my code that I am unsure of as I am running test scenarios on it.  Wouldn’t that be better as a codified test that is reproducible and helps keep the quality of your code up to par?

There are so many places to go, to find information about unit testing and how to add tests to your specific language choice, I won’t get into them.  I will say that in the C#, MS unit test is more than sufficient, especially when coupled with something like Rhino Mocks for mocking out external dependencies and making sure you have written code that relies on dependency injection for passing most if not all classes (see: coding to an interface, not an implementation design principle).

3 – Bug Tracker

Now, I don’t mean you should be using bugzilla or TFS (though, to be fair, I’m not saying that you shouldn’t) but do something.  If you run into a bug, write it down, make an item for it, do something.  Don’t just see it, fix it, and move on with your life.  If you are working on anything other than some throw away code or PoC, then you will most likely see some tale-tell sign of this bug in the future.  So, write it down, even if it is in the code somewhere, write it down.

4 – Comments

That brings us to my last bit of advice, comment the code you are changing to reflect why you did it.  I can’t count how many times I’ve been reading/stepping through code and come across something that looks odd.  I will think on it, maybe run some test stuff through it, then say “Well that seems like a dumb change, I think it should be this other way”. BAM, bug is back. 

Now, that exact statement can be changed a bit to be more general.  Comment anything that is out of the ordinary or isn’t immediately recognizable.  Not just to you, but even to the people who will come after you and maintain or enhance your program.  It is easy, in the moment, to know exactly what your code is doing and why, it won’t be tomorrow…or next week…or next month.  I have written too much code that even the next day I end up saying, “why did I do that?”.  Sometimes the answer is, I was tired, I was dumb, some rare times it’s I had a clever epiphany on how to solve a problem elegantly.  A clever, elegant solution isn’t worth much if you forget it the next day and take out, right?

Let’s stop debug whack-a-mole.  Let’s help others stop debug whack-a-mole.