_IEDocReadHTML does not fully process the webpage before sending back the HTML

Empika · March 29, 2019

Hello all,

I'm new to the forum, but not new to AutoIt. I currently work on the automation of the filling of a web form. Normally should be quite straightforward with the _IEFormGetCollection function and apparented.

However, in this case, this has revealed impossible. After investigations, I thing I got to the core of the issue ( but not to the solution). The webpage is containing statements to .aspx scripts and the HTML returned back by the _IE function seems to be the one *before* the execution of these .aspx

As a demo :

#include <IE.au3>
#include <MsgBoxConstants.au3>

Local $oIE = _IECreate("http://appliweb2.fr.mycompany.com/wfgen/") ; URL slightly modified for confidentiality
Local $sHTML = _IEDocReadHTML($oIE)
MsgBox($MB_SYSTEMMODAL, "Document Source", $sHTML)

This returns the following, only a few lines :

<html><head><META content="IE=11.0000" http-equiv="X-UA-Compatible">
                <meta http-equiv="X-UA-Compatible" content="IE=edge">
                <link href="./favicon.png" rel="icon" type="image/png">
        <title>WorkflowGen - Portail de workflow</title></head>
        <frameset rows="*" border="0" frameborder="no" framespacing="0">
                <frame name="WFGEN" src="show.aspx?QUERY=CONTEXT&amp;REQUEST_QUERY=WELCOME&amp;NO_REDIR=Y" marginwidth="0" marginheight="0" noresize="" scrolling="no">
        </frameset>
</html>

As you can see the frame item WFGEN points to show.aspx which seems not to have been executed, so impossible to work directly on the form items which should have been generated by the script. However, the page displayed in the IE launched by AutoIt is correct. On the displayed page, with an IE Viewsource, i get all all the form items. I don't copy all here, rather long, but as an extracted example, the submit button of the form is easy to find :

<div class="ButtonSet">
<input type="submit" name="SubmitButton" value="Envoyer" onclick="javascript:return ValidatePageProcessing();WebForm_DoPostBackWithOptions(new WebForm_PostBackOptions(&quot;SubmitButton&quot;, &quot;&quot;, true, &quot;WFGENPage&quot;, &quot;&quot;, false, false))" id="SubmitButton" class="SubmitButton" />
</div>

AU3Info shows the fully generated page is a unique control, so reachable via ControlClick. However, the positioning of the fields is way too variable to think to automation via ControlClick + Coordinates ( not even mentionning MouseClick here ... )

>>>> Control <<<<
Class:  Internet Explorer_Server
Instance:       1
ClassnameNN:    Internet Explorer_Server1
Name:   
Advanced (Class):       [CLASS:Internet Explorer_Server; INSTANCE:1]

I have tried unsuccesfully a couple of things:
- to give some time by inserting sleep functions
- to open the page by hand and then do _IEAttach

In all cases, the same HTML is returned by _IEDocReadHTML and the form items are unavailable to be adddred by _IEFormxxxxx functions.

On the server side, the form are managed with a 3rd party product : WorkFlowGen ( www.workflowgen.com ). I unfortunalty cannot give access to the actual webform, company intranet not rachable from the Internet.

On the client side :
Windows 7 Enterprise ; Service Pack 1
Internet Explorer 11 Version: 11.0.9600.19301 ; vupdates version: 11.0.115 (KB4489873)
AutoIt 3.3.14.5

Anyone would have an idea why the execution of the .aspx scripts of the page ( which are executed , since I can see the actual form in the end) is not reflected in the HTML provided back by the _IE function? And if there a way to modify this ?

thanks in advance

Patrick

Danp2 · March 29, 2019

Can you explain why you are using _IEDocReadHTML in your process? What's are you trying to solve by doing this?

Have you looked into using the _IEFrame* commands (like _IEFrameGetObjByName) to gain access to the frame's contents / elements?

Empika · March 29, 2019

Ultimatly what I'm looking for is to fill in a web form. I have done that in other occasions with _IEFormGetCollection and _IEFormElementSetValue ( or _IEFormElementOptionSelect etc ..). But in this case, _IEFormGetCollection returned nothing when I first started to work on this. Thats why I have looked to the raw HTML to understand what's going on.

Here is the output of a couple of _IEFrame function

#include <IE.au3>
#include <MsgBoxConstants.au3>

Local $oIE = _IEAttach("WorkflowGen")

Local $oFrames = _IEFrameGetCollection($oIE)
Local $iNumFrames = @extended
Local $sTxt = $iNumFrames & " frames found" & @CRLF & @CRLF
Local $oFrame = 0
For $i = 0 To ($iNumFrames - 1)
    $oFrame = _IEFrameGetCollection($oIE, $i)
    $sTxt &= @CRLF & "InnerText: " & @CRLF & _IEPropertyGet($oFrame, "innertext") & @CRLF & @CRLF
    $sTxt &= @CRLF & "Innerhtml: " & @CRLF & _IEPropertyGet($oFrame, "innerhtml") & @CRLF & @CRLF
    $sTxt &= @CRLF & "locationurl: " & @CRLF & _IEPropertyGet($oFrame, "locationurl") & @CRLF & @CRLF
    $sTxt &= @CRLF & "outerhtml: " & @CRLF & _IEPropertyGet($oFrame, "outerhtml") & @CRLF & @CRLF
Next

MsgBox($MB_SYSTEMMODAL, "Frames Info", $sTxt)
ConsoleWrite("Frame Info:  " &  _IEFormElementGetValue($oQuery))

Here is the output

Frame Info:  1 frames found


InnerText: 



Innerhtml: 

<frame name="WFGEN_CUSTOM" src="/wfgen/App_Themes/Default/portal/banner/banner.htm" marginwidth="0" marginheight="1" noresize="" style="-ms-overflow-x: hidden; -ms-overflow-y: hidden;">
<frame name="WFGEN_TOP" src="/wfgen/show.aspx?QUERY=HEADER&amp;ID_USER_DELEGATOR=-1&amp;FILTER_PROCESSID=" marginwidth="0" marginheight="1" noresize="">
<frame name="WFGEN_MAIN" src="/wfgen/show.aspx?QUERY=WELCOME&amp;NO_REDIR=Y" marginwidth="6" marginheight="6" noresize="" scrolling="auto">
<frame name="WFGEN_BOTTOM" src="/wfgen/show.aspx?QUERY=FOOTER&amp;ID_USER_DELEGATOR=-1&amp;FILTER_PROCESSID=" marginwidth="5" marginheight="2" noresize="" style="-ms-overflow-x: hidden; -ms-overflow-y: hidden;">



locationurl: 
http://appliweb2.fr.otis.com/wfgen/show.aspx?QUERY=CONTEXT&REQUEST_QUERY=WELCOME&NO_REDIR=Y


outerhtml: 
<frameset rows="50,68,*,0" border="0" frameborder="no" framespacing="0">
<frame name="WFGEN_CUSTOM" src="/wfgen/App_Themes/Default/portal/banner/banner.htm" marginwidth="0" marginheight="1" noresize="" style="-ms-overflow-x: hidden; -ms-overflow-y: hidden;">
<frame name="WFGEN_TOP" src="/wfgen/show.aspx?QUERY=HEADER&amp;ID_USER_DELEGATOR=-1&amp;FILTER_PROCESSID=" marginwidth="0" marginheight="1" noresize="">
<frame name="WFGEN_MAIN" src="/wfgen/show.aspx?QUERY=WELCOME&amp;NO_REDIR=Y" marginwidth="6" marginheight="6" noresize="" scrolling="auto">
<frame name="WFGEN_BOTTOM" src="/wfgen/show.aspx?QUERY=FOOTER&amp;ID_USER_DELEGATOR=-1&amp;FILTER_PROCESSID=" marginwidth="5" marginheight="2" noresize="" style="-ms-overflow-x: hidden; -ms-overflow-y: hidden;">
</frameset>

I have run _IEFormGetCollection in the first place, but the @extended return is 1 .. while the form is having 10th of inputs, radio button and so on ...

hence my look at the raw HTML to understand .. and still don 't understand why I dont get the result of these .aspx instead of a link to them.

But I m not familiar that much with frames and I may be missing something here

thanks for reading my post anyhow , much appreciated.

Patrick

Danp2 · March 30, 2019

You need to get a reference to the frame and then use that in place of the normal IE object, like this --

Local $oIE = _IEAttach("WorkflowGen")
Local $oFrame = _IEFrameGetObjByName($oIE, "WFGEN")
Local $sHTML = _IEDocReadHTML($oFrame)
MsgBox($MB_SYSTEMMODAL, "Frame Source", $sHTML)

Can't tell from your post above, but it appears that you may be dealing with frames within a frame. If so, you'll need to drill down into the desired frame before you can access it's contents.

Empika · April 8, 2019

Good guess. I have been confused by the Russian dolls 😉 . Drilling down the frames and subframes via _IEFrameGetCollection got me to the actual frame holding all the Forms elements. Very straight forward from there.

To add to the confusion, the number of embedded frames level was different depending on the way the form was accessed and several level of frames were having the same name. Writing a recursive fonction drilling down the frame structure until I got to the correct frame did the trick.

Thanks a lot for pointing me into the right direction.

Empika · April 8, 2019

In case it can help someone, the recursive exploration code

#include <IE.au3>
#include <MsgBoxConstants.au3>

Global $Recurs = 0                      ; put a limit in case the recursion got mad
Local $oIE = _IEAttach("Request Entry") ; replace with the Window you d like to explore

DOCExplorer($oIE, 0)

; end of program

; Function called recursivelly
Func DOCExplorer($oParam, $Level)

$Recurs += 1
If $Recurs > 10 Then
    consoleWrite("## recursion Loop ##" & @CRLF)
    Return
EndIf

    $sTxt = "========== Document at level " & $level & @CRLF

    ; links

    Local $oLinks = _IELinkGetCollection($oParam)
    Local $iNumLinks = @extended

    $sTxt &= $iNumLinks & " links found" & @CRLF
    For $oLink In $oLinks
        $sTxt &= "=> "  &$oLink.href & @CRLF
    Next

    ; forms

    Local $oForms = _IEFormGetCollection($oParam)
    $iNumForms = @extended
    $sTxt &= "Number of Forms: " & $iNumForms & @CRLF



    For $j = 1 to $iNumForms
        $oForms = _IEFormGetCollection($oParam,$j )
        $oElements = _IEFormElementGetCollection($oForms)
        $iNumElements =@extended
        ;$sTxt &= "Forms n° " & $j & "( " & $oForms.name & " ) has " & $iNumElements &  " inputs" & @CRLF
        $sTxt &= "Forms n° " & $j & " has " & $iNumElements &  " elements" & @CRLF
    Next


    ; frames

    Local $oFrames = _IEFrameGetCollection($oParam)
    Local $iNumFrames = @extended
    $sTxt &= $iNumFrames & " frames found at level "  & $level & @CRLF & @CRLF
    Local $oFrame = 0


    For $i = 0 To ($iNumFrames - 1)
        $oFrame = _IEFrameGetCollection($oParam, $i)
        $sTxt &= "frame n° " & $level & "." & $i & @CRLF
        ;$sTxt &= _IEPropertyGet($oFrame, "outerhtml") & @CRLF
        $sTxt &= "URL : " & _IEPropertyGet($oFrame, "locationurl") & @CRLF
    Next

    ConsoleWrite(@CRLF & $sTxt)

    For $i = 0 To ($iNumFrames - 1)
        $oFrame = _IEFrameGetCollection($oParam, $i)
        DOCExplorer($oFrame, $Level + 1)
    Next


EndFunc

Patrick

Sign In

_IEDocReadHTML does not fully process the webpage before sending back the HTML

Recommended Posts

Empika

Link to comment

Share on other sites

Danp2

Link to comment

Share on other sites

Empika

Link to comment

Share on other sites

Danp2

Link to comment

Share on other sites

Empika

Link to comment

Share on other sites

Empika

Link to comment

Share on other sites

Create an account or sign in to comment

Create an account

Sign in

Recently Browsing 0 members

Browse

AutoIt Resources

Release

Beta