Saturday, June 07, 2008

Assessments has tools

I just finished the first set of Assessments tools, consisting of a checklist evaluator and a result viewer. The checklist evaluator is like the test runner, but it does not display results. Those are shown in a separate window, the result viewer.

What this allows is to bring up a result viewer that allows you to debug and run things to failure (Assessments' exception handling really shines here, you don't have to step through any of the evaluation machinery). It also remembers the assessment you ran, and so let's say you fix something. Then you don't need to go back to the checklist evaluator to start over --- you just click on a button that says [Reevaluate Assessment]. This updates the result viewer by evaluating the assessment again. Therefore, you can see your defect list going down without having to disturb any of your code browsers (either by clicking on things you want to test again, or even by having to move to a different package that has the root test class you need to run).

The beauty of all this is that, by design, remembering things like the assessment and the checks that ran cannot cause memory leaks. So all of this comes basically for free.

I also had the experience to fix several small bugs. I have to say that I am very happy about the amount of effort I put in so that the code is flexible, as well as with the results of this work. It is almost too easy to address seemingly problematic situations. For example, at one point I found that failures did not cause a debugger to pop up when rerunning failures to the first unhandled exception. Fixing this was a matter of adding a single message in the abstract class of failure notifications.

Also, at one point I noticed that assessment results were not counting the tests run properly. It turns out that, by design, some checks can produce more than one result, and this was throwing the count off. Fixing this was a simple matter of teaching the results about whether to count towards the total or not. And since there is one class per type of check result, it was resolved via polymorphism too. Piece of cake.

Here are the latest metrics.

Class count (including 4 class extensions): 94.

Method count (including 14 extension methods): 529.

Average methods per class: ~5.6.

What is interesting about the numbers above is that, before the introduction of UI classes, the average methods per class was about 5.1 instead. UIs cause this metric to rise quickly.

The next tasks in the to do list include a proper SUnit bridge that does not touch any of the existing SUnit code, and teaching Assessments to do SUnit Benchmarks and SUnit Validation.

Also in the queue is a task to streamline exception handling and removing as much code as possible from AbstractChecklist. While it is cleaner than TestCase even though it is doing more work (30 instance methods and no instance variables, versus 38 instance methods and one instance variable), it can be better still.

More to come...

No comments: