question

trynitro avatar image
trynitro asked trynitro answered

Low performance when reading/manupulating a text file

I have a text file with ~150-200 lines in which I am searching for some strings, make a list of the strings found and then count how many times each string shows up. Based on this then I write in the same variable my findings.

This test case takes ~2 minutes to execute. For me, this is way too much. In a scripting language (tcl, perl, python, you name it) this would not take more than 2 seconds. I don't understand why this takes so long to execute, is this an iTest known issue or maybe it's cause by the way I coded (badly that is).

Can you please explain?

P.S. I tried to attach the test case as .fftc and as .txt but I get an error message This attachment is not permitted because the file type is invalid."

iTestperformance issue
10 |950

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

JeffJ avatar image
JeffJ answered

I took the liberty of using some of the features embedded in iTest to rewrite your test case. This version runs extremely fast and is based on using response mapping to decode your input file. Take a look at this version (MyVersion.fftc) to see what I did. If you have questions, post them here.forum.zip


forum.zip (5.8 KiB)
10 |950

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

trynitro avatar image
trynitro answered

I understand and I agree that I don't have the most effective code, or that my code uses all the internal functions of iTest in an efficient way - I said it from the start.

What I am understanding from your answer is that if I do that code my way then that is the expected run time. Is that correct?

Thank you

10 |950

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

trynitro avatar image
trynitro answered

I re-run the file I gave you after starting iTest with "-clean" and it took 16 seconds. Still much, but a great improvement. But I can't close iTest each time I need to run a test and re-open it, that's not an option for anyone I think. Furthermore I see some memory usage from iTest: it starts with ~250MB, after a few runs it goes to ~500MB and there it stays. This tells me that you are not clearing the memory after each run, or you might have a memory leak somewhere.

The file I need to run is something like the new attached one. With this file, after starting iTest with "-clean" it took 1 min and 19 seconds for the test to run. That's too much, for such a simple text processing test.base.zip


base.zip (1.1 KiB)
10 |950

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

trynitro avatar image
trynitro answered bhavani.geepalem commented

sample.zip

Please find attached the file.


sample.zip (2.2 KiB)
1 comment
10 |950

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

bhavani.geepalem avatar image bhavani.geepalem commented ·

Thanks for providing the requested files.

We have executed the provided test case in-house in both iTest 5.3 & 5.4 (latest version) and observed that, the test case is taking just of 7 or 8 seconds to execute it.

Can you please give a try to restar the itest with "-clean" option once and let us know the results. To do this, go to iTest installation directory from command prompt and type command "iTest.exe -clean"

0 Likes 0 ·
bhavani.geepalem avatar image
bhavani.geepalem answered
@trynitro

Can you please provide us the test case and the .txt file in a zip format to assist you further on this.

10 |950

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

trynitro avatar image
trynitro answered

I am reading the entire file in a variable and then working with that variable. I don't think I am using response filters, I don't know what those are :)

10 |950

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

JeffJ avatar image
JeffJ answered

Are you using Response Filters or reading the lines one at a time?

10 |950

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

trynitro avatar image
trynitro answered

I forgot to add, I use iTest 5.3, on a Xeon E7 CPU (4 processors) with 32GB RAM.

10 |950

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.