question

dclaar avatar image
dclaar Suspended asked dclaar Suspended posted

HTML response map?

I have a variable with html in it. I do eval get $html, and the response shows (this is the first part...it is from bugzilla):

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
                      "http://www.w3.org/TR/html4/loose.dtd">
<html>
  <head>
    <title>Bug 5779 &ndash; is Disabled, not Healthy</title>

 

I created a response map, and pointed to it, and checked the box in the response map "HTML mapper: Maps any HTML response". Maybe it does, but it doesn't appear to generate any queries, and when I try to add a rule, neither XPATH nor queries has anything in it! So how exactly is this supposed to work?

iTestresponse map
10 |950
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

1 Answer

·
dclaar avatar image
dclaar Suspended answered dclaar Suspended posted

It appears that the DOCTYPE confuses it. Seems like a bug...

 

12 comments
10 |950
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

dclaar avatar image dclaar commented ·
It is also silly that you need an empty response map to get the HTML queries to work. There ought to be a default response map for that, since the HTML mapper matches all html.
0 Likes 0 ·
KumarS avatar image KumarS dclaar commented ·

HTML mapper simply converts HTML document to XML document. This is required because HTML is kind of an ugly protocol and is not a valid XML. We use some open source libraries to convert HTML to XML. Once it is converted to XML, we put this in the structured data. We do not generate any queries because HTML could possibly be anything. But since the data is in the structure part of the response, users can use [query ...] to create XPATH queries to extract data from HTML document.

 

Not all HTML can be converted to XML. There are a number of heuristics used to do that. We added a lot of our own heuristics to the open source code. May be we need another one to deal with this HTML document.

0 Likes 0 ·
dclaar avatar image dclaar KumarS commented ·
Yeah, seems like DOCTYPE would be fairly common...
0 Likes 0 ·
AdamB avatar image AdamB dclaar commented ·

We have a bug and are looking into it.  Can you provide either the page url that you were getting or the page source?

 

Thanks,

Adam

0 Likes 0 ·
dclaar avatar image dclaar AdamB commented ·
Do y'all use bugzilla, or have a login for something that does? This is just a standard bugzilla page. I can FTP it, I guess. I used the script I posted as a workaround for HTTP not doing cookies to grab the page.
0 Likes 0 ·
PaulD avatar image PaulD dclaar commented ·
By the way, iTest automatically detects XML contents (using automatic response mapping) and you don't need an empty response map for that case.  But valid HTML can be almost anything.  The text in this response, for example, is valid HTML.  We thought it was too dangerous to automatically try to infer that something is HTML.  Perhaps some heuristics could tell us in some of the common cases when something is HTML, but there is a worry about getting it right "most of the time".
0 Likes 0 ·
dclaar avatar image dclaar PaulD commented ·
Is that new? I thought that, in 3.3.1, I had to have an empty response map for XML.
0 Likes 0 ·
PaulD avatar image PaulD dclaar commented ·
Yes.  This is a new feature of automatic response mapping in 3.4.
0 Likes 0 ·
PreetS avatar image PreetS PaulD commented ·

I tested HTML mapping with the DOCTYPE tag as you mentioned, but it seens to be working for me (in 3.4).

HTML mapping itself does not generate any 'queries' as such. What it does is create XML structured data from the response. 

 

So when you do the mapping, do you see mapped html nodes in the structure view? (should be under structure->mapped->html )

You can evaluate any xpath you wish on this data and add a qurery for that. 

 

If you do not see mapped structured data, then something else may be going wrong, because the DOCTYPE tag seems to be properly handled

0 Likes 0 ·
dclaar avatar image dclaar PreetS commented ·

I tried to recreate this by taking out my

regsub {<!DOCTYPE[^>]+>} $bugzilla_html {} bugzilla_html

 

line, and now the darn thing works! I don't get it, but I guess don't mess with success!

0 Likes 0 ·
PreetS avatar image PreetS dclaar commented ·

Just for the sake of sanity, could you attach the zipped source of the html? 

I just want to make sure that there isn't something else on the page that may have caused this.  

0 Likes 0 ·
dclaar avatar image dclaar PreetS commented ·
I recreated it using the same bug as before, and can't reproduce it. And even going back and looking at the reports, I can't tell if it tried to apply the response map or not, as it doesn't say in the test report, either for the one where I know it did, or for the one that I'm not sure of . Even grabbing the text that didn't work, and using readFile, doesn't seem to do it. So I guess I just did something wrong before. Sorry!
0 Likes 0 ·

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.