Software development cycle is a complex process with many stages full of underwater rocks and hidden problems. Writing a clear and elegant code is not sufficient these days. Even the best devs teams can’t write a perfect code. Bugs are inevitable, which makes testing and QA an essential part of the development team. Often, project and product managers underestimate importance of QA. Such wrong judgements then come back with huge issues and monetary losses. “I’ve got a fantastic development team, so why should I bother or a quality QA department?”, this is what I often hear from inexperienced PMs. Yes, development of a product is in the first place. Yet, no product should ever go into production, unless it is properly tested. What am I driving at? In this post I want to look at QA and testing metrics and explain how optimization of QA can improve the entire software development cycle.
Forecasting
How many testers do I need? Some managers hire 1 QA specialist for every 2-3 developers. Refusing to analyze data is not smart. By implementing probability of bugs metrics, you’ll be able to find out how many testers you lack. Or perhaps you have too many of them? This indicator is based on historic data. Say, testers have checked a new release that has 1k of new code line, and found 10 bugs or inconsistencies in it. So, it means that there is 1 bug per 100 lines of code (do not take these figures seriously, it’s just an example). By measuring this metrics, you will not only forecast future bugs, but also analyze improvement (or regress) in code writing. I am not trying to say that your devs are writing buggy code. However, it would be nice to implement such metrics.
Diversity of QA Methods
I am not teaching you here since you are sure to use diverse testing methods: manual testing, smoke tests, Selenium tests, JUnit test, Cucumber tests, GUI test automation tool box and what not. Do not be lazy and track them. The more test types you use, the more efficient the entire QA cycle is. At least it should be more efficient with a wide variety of QA methods. Know your ratio of manual vs automated test, and how many bugs both have discovered (in pieces or %). Look for new methods of functional testing. This is a kind of indicator that should not be updated too often. After all, you do not use new test types every day. However, your senior QA should regularly renew the toolbox.
How Your QA Work
Ideally, test coverage should be 100%. But, hey, let’s be honest and realistic. It is virtually impossible to achieve that number. Well, if you manage to cover 100% of your product with tests, you are a gorgeous head of QA! Anyway, implement this metrics to track the progress. There’s always some room for improvement.
Your dev team leader has made the final commit and QA are waiting for their turn. Great! How much time do they need to get the job done? 48 hrs? Not bad. But you’d better know the exact figure. Of course, turnaround time depends on the amount of work, since some commits are so huge.
How many bugs are reported by users, tester or devs after QA have finished their job? That’s a very interesting indicator that will keep your QA team in shape. After all, you pay money to find and fix bugs but not imitate testing. Huge number of reworks is an alarm bell. Your developers may end up fixing own bugs after QA checks, instead of developing new features.
Training
How often do your testers participate in trainings, seminars and workshops to grasp new technologies? Have you seen your QA department in the first place? Talk to people and invest in your personnel. Why not introduce such a KPI as training sessions per employee or % of testers having all necessary skills? You’ll see improvement in other indicators after figures in these ones start to increase. At least, this is the way it usually happens in most software development companies.
Communication
Success of perfect QA team is in efficiency of its communication with developers and PMs. How fast bugs are reported back? How many issues are created? How many comments are posted per issue? How many communication contacts are initiated? This stuff matters when it comes to mid sized and large teams.
Finance
Some simple metrics will help you allocate budgets in a more efficient way and optimize expenses related to QA. These metrics might include % of QA expenses as compared to annual dev budget, revenue acquired dues to timely product updates/releases due to efficient QA work etc. I know a few companies that spend more for QA than they pay their devs, simply because testing processes are not optimized.
In case you are interested in software development metrics and tools to implement the BSC in your dev teams, feel free to visit our site or ask questions in comments. And may you never have buggy code!
Source: http://server.dzone.com/articles/essentials-software-testing-qa-0