Serious problem with using MySQL as reporting database: Wrong reports shown!
iTest 4.1.x testcase editor view shows results from a testcase that did not even run on the same test harness, is not from the same SVN tree, and can’t be opened with right click and “Open Test Report”. This happens even though I’ve set my “Search Reports” criteria to only find reports from tests run on this machine.
- reporting database engine: MySQL to database server
- reporting database: itest (common for all test harnesses)
What is the "testcase editor view" you are talking about? Do you mean the response view of the test case steps?
If you double click on a testcase in the itest explorer, it brings up an editor. If you do "open with", it is called "Test Case Editor". From that editor, you can see responses from the last time the test case was run; you can use wizards to add analysis rules, you can investigate queries.
But none of that works correctly if the results that it displays are from a completely different version of the test case!
We are matching response to test case step in the following steps:
1. matching step's GUID
2. matching step's action, session, command, and target
If you copied the test case file and changed some steps in it, the old steps will still be a match since the GUID of those steps aren't changed during file copying. If you only copy steps in test case editor, the steps will be assigned with new GUID.
So are you saying that you do this on a step-by-step basis? So then, if I have testcase run 1:
GUID=1, command=ls, output="bah" GUID=2, command=pwd, output="/"
And testcase run 2 with these:
GUID=1, command=ls, output="foo" GUID=3, command=pwd, output="/tmp"
Then I could conceivably see:
command=ls, output="foo" command=pwd, output="/"
On the machine I ran testcase run 1 on, because it finds the output to run 2?!?!
I hope that you can see that this would be a problem! If, for example, I've output messages from analysis rules, I'm going to see that file foo is in directory / in the Execution window!
If you have set your test report filter to only show test reports for that system, when you right click on the the line in the Execution window and try to open test report, nothing happens
This is a good feature request. We will look into this.
FEATURE REQUEST?!?!?! 
How about, "HUGE FLAMING BUG!" Particularly when support tells us to use MySQL instead of Derby.
Sorry if I sound cranky, but this is a really, really serious problem. The only current workarounds are to either go back to derby, or to use a separate database for each test harness. Seeing the results for release 3.3 when I'm on the test harness for release 3.4 is just unusable.
Doug,
That is current implementation in iTest. We completely understand your point. If you think about from an implementation or requirements perspective, the behavior is appropriate i.e show the latest response for this unique step. But to your point that when using central DB, lots of users could be using the same test case and running them can cause inconsistent behavior. You might see somebody elses response. And that is what was missed from our side and hence we would have to add a feature to allow user to pick responses only from your host or something along those lines.
We do understand the inconvenience you are going through and how it is affecting your daily job and rest assured we will try to get to this ASAP.
In the meanwhile, let us know if there is any other way we could help you.
-A
When you are looking at implementing this, maybe you can allow the user to specify any run, not just the latest. This is a feature request from long, long ago that seems to have been lost.
The use case for this feature would be that I have a common test case that I call; let's say for example, that it does an ls or a dir. The first time it is called, it finds file "foo". The second time (which is in the same run), it doesn't.
I'd like to create an analysis rule where, if it finds "foo", then the test fails. Currently, I can only do this manually, because the wizard will see the response from the second time. It would be great if you could choose which response the wizard sees.
Previously, this was somewhat possible from the report view, but that capability was lost when 4.0 came out (http://forums.fanfaresoftware.com/t5/General-iTest-Topics/Lost-quot-Add-Rule-quot-functionality-in-iTest-4-0/m-p/14543/highlight/true#M6255).
You are right, in pre 4.0 we used to be able to do that since you could just open that TR and add that rule.
We will look at it and try to get this sorted soon.
-A
