Monthly Archives: November 2014

Report Verifications, Not ‘Tests’

In many testing shops, counts of passed/failed ‘tests’ are the main part of summary test reporting. But a ‘test’ result is just a collection of verification results, and its reporting is just a way to obscure the actual situation.

Suppose, just for a very simple example, that we have 100 tests, each with 10 verification points. Suppose further that the report says that 5 tests failed.

What, exactly, does that mean? Well, actually, it doesn’t mean anything very exact.

At one extreme, it could mean that in the 5 tests, all 50 verifications failed. At the other it could mean that just one verification failed in each of the 5 tests — 5 failures in all.

So we know that for failures we have somewhere in the range 5% (50/1000) down to 0.5% (5/1000). Pretty fuzzy, no?

That’s why I don’t report passed/failed ‘tests’; I report passed/failed verifications.

[Actually, I report passed/missed/failed verifications, where ‘missed’ means that the verification point in the test was not reached.]

Advertisements

A Kinder, Gentler Report

I really hate a report (whether Excel, HTML, or Word) that uses full-strength colors as visual aids. They’re too jarring!

passed
missed
failed

Instead, I’m now using the colors I cribbed from MS Excel’s Home tab, in Styles.

passed
missed
failed

[Here in WordPress I did not find out how to make narrow cells. Sorry.]

The new colors will look more true (and better) if you scroll down so that the old colors are no longer on your screen.

Excel

My History Report is an Excel spreadsheet. For that report, I’ve set the whole spreadsheet to conditional formatting. Here’s how:

  1. Select entire worksheet (by pressing Ctrl-A).
  2. Passed:
    1. Go to Home => Conditional Formatting => Highlight Cell Rules => Equal To....
    2. Type passed.
    3. Select Green Fill with Dark Green Text.
    4. Click button OK.
  3. Missed:
    1. Go to Home => Conditional Formatting => Highlight Cell Rules => Equal To....
    2. Type missed.
    3. Select Yellow Fill with Dark Yellow Text.
    4. Click button OK.
  4. Failed:
    1. Go to Home => Conditional Formatting => Highlight Cell Rules => Equal To....
    2. Type failed.
    3. Select Red Fill with Dark Red Text.
    4. Click button OK.

Then, any cell that has text passed, missed, or failed is automatically formatted appropriately.

HTML

My Changes Report is an HTML page. For that report, I’ve used the MS Excel colors as above, by adding this to the style section in the head section:

.good {color: rgb(0,97,0) ; background-color: rgb(198,239,206) }
.neutral { color: rgb(156,101,0) ; background-color: rgb(255,236,156) }
.bad { color: rgb(156,0,6); background-color: rgb(255,199,206) }

Then a cell can be, for example, defined in the HTML as:

<td class="good">passed</td>

The cell is then rendered with that same font color and background color as is seen in the Excel spreadsheet above.

MS Word

For a Word document, …

Well, actually I don’t generate reports in Word documents.