This is the report that matters most to actual testers, because it filters out all the noise of unchanged test results, focusing instead on the most interesting results — those that are different from the previous result.
See more at the link above.
For a long time now, I’ve been working on automated testing for legacy software (or legacy features of evolving software).
It’s been an education.
The thing is, a new feature should be developed with a test-first strategy that uses assertions. And a failed assertion should be repaired immediately.
But for a legacy feature, automated testing will reveal a number of low-severity bugs that will not be fixed anytime soon. That’s why detailed logging and a detailed changes report are paramount.
So now I’ve founded a new GitHub project, RubyTest, that will make available, as working software, much of what I’ve learned.
The beginning is modest — just a logger, a few helpers, and their tests. (Yes, the framework itself gets thorough testing.)
Next, I think, will be a simple REST client example that uses the endpoints at JSONPlaceholder.com. This will be the opportunity to show how the framework will handle endpoints, data objects, and verdicts.
Much more to come.
Here are some principles I try to keep in mind.
|Readability||Make code easy to read: code is read more often than
|DRYness: Don’t Repeat Yourself||Avoid redundant code and data.|
|YAGNIty: You Ain’t Gonna Need It||Don’t write code before it’s needed.|
|Sloth||Maximize use of existing packages and libraries. The line of code you don’t write is the line of code you never have to debug. — Steve Jobs|
|Explicitness||Explicate everything, even when “unnecessary.” If it goes without saying, it would go better by saying it. — Tallyrand|
|Cleanliness||Keep everything clean and consistent: run static code
analysis; resolve all issues before committing.
|Failed Verdict Diagnostics||Log data sufficient to diagnose a failed verdict.|
|Error Diagnostics||Detect test errors early, fail early, log useful information.|
|Monitoring||Trust, but verify. Monitor documentation for changes (programatically, of course).|