AVIO Consulting

Risky Business — How to Test With Priority

Jul 20, 2018 | Other

If you’ve ever had difficulty prioritizing testing activities, you’re not alone. Iterative frameworks like Agile mean testing is performed in miniature cycles that add up to what eventually becomes a product release. Unfortunately, though, testers don’t necessarily get to choose which tickets developers complete first. One test release may have several JIRA tickets the client considers high priority while another might have none. This begs a question: How does one prepare to handle such variability?

A risk-based testing approach is my favorite testing adaptation for evening out the peaks and valleys that sometimes accompany the Scrum-Agile style.  Although 1/3 (sometimes 17 or more) of the JIRA tickets in our sprints may be marked as high-priority items, not all tickets the business considers a high priority necessarily require the same level of testing.  Here’s my process for converting high-priority business items into risk categories that help me better manage the testing activities surrounding these VIPs of the development sprint.

Static Testing

For me, I like to start testing when the requirements are just that — the requirements.  Every time I read (and re-read) a new specification document and JIRA tickets, I notice areas of agreement and areas of misalignment.  Not only does this static testing approach allow me to start identifying bugs before they make it into the code, it allows me to determine how complex the development outcome(s) for a particular user story might be.

As the complexity grows, so too does my estimation of testing risk.  True enough, an un-skilled developer can make the re-labeling of a text field an unholy affair, but that just doesn’t happen very often.  Over time, the simple jobs start to become easy to pick out from the list.  This makes the more hairy tickets easier to identify as well.

Organize Data

Either simultaneously (or just after performing static testing), I like to begin organizing my data.  This is where ye good olde spreadsheet comes in handy.

Risk Priority Spreadsheet

Notice that the business risk column connotes a high level of risk (5) for every item listed.  The 5 indicates that these items are listed as high-priority business items in JIRA.  It’s worth noting that medium-priority items are given a business risk rating of 3, and low-priority items are given a business risk rating of 1.

In the adjacent column, the risk priority numbers (RP# = Testing Risk x Business Risk) assigned to each record appear to vary considerably.  When I first performed this exercise, I was shocked to find that the risk priority number for each of these high-priority business items rarely approached the apparent severity of the business need.

The reason for this is simple: the testing risk rating was relatively low for all but a couple of cases.

To be clear, this is not a case of quality control undervaluing what the business has assigned a high-value to.  Rather, it’s the realization that not every high-priority business item requires the same degree of testing.  When a test release is made available, provided your test cases, state and decision tables, and cyclonic complexity analyses are in order (You prepared them in advance, right?), you should be ready to test with risk priority helping steer the ship.

Whatever you do, though, don’t forget to …

Consult Your Roadmap!

In a hypothetical release containing 18 or more high and medium priority tickets, feeling overwhelmed is expected.  Note in the image below that there are a lot of tickets whose risk priority number is a 5/Report Bugs (the lowest level of testing priority). 

Several are medium-priority items while several more or high-priority items.  Which ones should be tackled first?  Yup, the ones with a business risk level of 5!

Risk Priority Spreadsheet 2

However, don’t forget that among those tickets marked as high-priority items, there’s a hierarchy.  So, unless circumstances prevent me from doing so, I like to hit the high-priority items with the highest risk priority numbers first.  Again, if I’ve been a boy scout and prepared all of my test cases and artifacts in advance, I should be able to make relatively quick work of these more demanding tickets.  The thing to remember here is that it’s easy to get distracted, so stick to the roadmap you made for yourself.


Testing tickets by business risk, then by risk priority number is an effective way of testing large numbers of tickets.  I like to think of this as a win-win for quality control.  The most gnarly of the items the business considers a high priority get tested first, which means they either get returned for bugs and defects earlier (win) or verified under test sooner (win).

By processing tickets in this way, I’m not only better organizing myself for efficient testing, I’m helping my team meet deadlines by keeping the testing activity contextualized and optimized.

What are your thoughts about this approach?  Let me know in the comments below.