Low performance when reading/manupulating a text file

Question

question

trynitro asked Sep 21, '16 trynitro answered Sep 22, '16

Low performance when reading/manupulating a text file

I have a text file with ~150-200 lines in which I am searching for some strings, make a list of the strings found and then count how many times each string shows up. Based on this then I write in the same variable my findings.

This test case takes ~2 minutes to execute. For me, this is way too much. In a scripting language (tcl, perl, python, you name it) this would not take more than 2 seconds. I don't understand why this takes so long to execute, is this an iTest known issue or maybe it's cause by the way I coded (badly that is).

Can you please explain?

P.S. I tried to attach the test case as .fftc and as .txt but I get an error message This attachment is not permitted because the file type is invalid."

iTest performance issue

10 |950

Attachments: Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Answer 1 · 2016-09-22T17:30:08Z

JeffJ answered Sep 22, '16

I took the liberty of using some of the features embedded in iTest to rewrite your test case. This version runs extremely fast and is based on using response mapping to decode your input file. Take a look at this version (MyVersion.fftc) to see what I did. If you have questions, post them here.forum.zip

forum.zip (5.8 KiB)

10 |950

Attachments: Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Answer 2 · 2016-09-22T17:59:20Z

trynitro answered Sep 22, '16

I understand and I agree that I don't have the most effective code, or that my code uses all the internal functions of iTest in an efficient way - I said it from the start.

What I am understanding from your answer is that if I do that code my way then that is the expected run time. Is that correct?

Thank you

10 |950

Attachments: Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Answer 3 · 2016-09-22T14:59:32Z

trynitro answered Sep 22, '16

I re-run the file I gave you after starting iTest with "-clean" and it took 16 seconds. Still much, but a great improvement. But I can't close iTest each time I need to run a test and re-open it, that's not an option for anyone I think. Furthermore I see some memory usage from iTest: it starts with ~250MB, after a few runs it goes to ~500MB and there it stays. This tells me that you are not clearing the memory after each run, or you might have a memory leak somewhere.

The file I need to run is something like the new attached one. With this file, after starting iTest with "-clean" it took 1 min and 19 seconds for the test to run. That's too much, for such a simple text processing test.base.zip

base.zip (1.1 KiB)

10 |950

Attachments: Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Answer 4 · 2016-09-21T22:04:07Z

trynitro answered Sep 21, '16 bhavani.geepalem commented Sep 21, '16

sample.zip

Please find attached the file.

sample.zip (2.2 KiB)

1

10 |950

Attachments: Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

bhavani.geepalem commented · Sep 21, 2016 at 11:56 PM

Thanks for providing the requested files.

We have executed the provided test case in-house in both iTest 5.3 & 5.4 (latest version) and observed that, the test case is taking just of 7 or 8 seconds to execute it.

Can you please give a try to restar the itest with "-clean" option once and let us know the results. To do this, go to iTest installation directory from command prompt and type command "iTest.exe -clean"

0 ·

Answer 5 · 2016-09-21T21:42:16Z

bhavani.geepalem answered Sep 21, '16

@trynitro

Can you please provide us the test case and the .txt file in a zip format to assist you further on this.

10 |950

Attachments: Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Answer 6 · 2016-09-21T21:31:10Z

trynitro answered Sep 21, '16

I am reading the entire file in a variable and then working with that variable. I don't think I am using response filters, I don't know what those are :)

10 |950

Attachments: Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Answer 7 · 2016-09-21T21:10:39Z

JeffJ answered Sep 21, '16

Are you using Response Filters or reading the lines one at a time?

10 |950

Attachments: Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Answer 8 · 2016-09-21T20:53:29Z

trynitro answered Sep 21, '16

I forgot to add, I use iTest 5.3, on a Xeon E7 CPU (4 processors) with 32GB RAM.

10 |950

Attachments: Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

question