Hi All,
We’ve been trying out iTest 4 recently and although we’re getting on well with it, we think there are some additional features that would dramatically enhance our iTest experience and would really cement iTest as the core part of our automation infrastructure.
So I thought that I’d share some of the features we would like to see in iTest 4, and also try to get other peoples thoughts on these, along with any features they would like to see.
To kick things off, here are a few of our feature suggestions:
Selective logging
This would be the number 1 feature request for us. We have wrapped our automation primitives, variable type checks, range checks, pass, fail & abort mechanisms, message output, etc into a set of common procedures in order to minimise points of maintenance and simplify our end-user test scripts. All this works very well, apart from the fact we now have a substantial amount of extra procedure calls, variable validation and background activity going on in iTest. Which is really throttling our performance as each step executed by iTest is logged. Because of this, we are seeing test cases that used to take 1 minute to execute now taking around 3 – 4 minutes (and our MySQL database is very quickly gobbling up our available disk space)
Our request here would be to have logging levels which can be set on each procedure, and then within the general tab of each test case the user could select the log verbosity level for execution. This way we could disable logging on all our internal / framework procedures, and just log the procedures directly related to the test functionality when carrying out test runs. Or enable verbose logging when developing / debugging framework procedures.
Variable type validation / range checks
To prevent test script writers from passing invalid values into our procedures, we have to perform our own type validation / range checks on each argument. We think it would be a good feature if, when specifying each argument for a procedure, there is an option to specify the type, and range (for numeric values), or valid values for enumerations.
Variable instantiation
One issue that keeps on cropping up is variable name typo’s that aren’t picked up until run-time. For example:
eval set var1 “abcde”
.
.
.
.
.
.
eval get var_1
Will not cause an error within iTest when developing a test, but at runtime would cause a “variable var_1 does not exist” error. We have had cases where there have been typo’s in a rarely reached legs of widely used procedures, which sit there un-noticed for weeks until the leg was executed.
To avoid this type of problem cropping up, we think it would be a good feature to implement a Perl style “use strict” mechanism where each variable requires a “set” prior to being referenced, and an error is flagged on any steps that reference a variable that hasn’t been set.
This could be an optional flag that could be set on a test case or procedure level.
