script moving too fast?

[SueB]

I have a script that goes through links of fund reports and saves a copy of the different resulting frames. I tested it first with just a few of the departments so I know the code is correct as far as getting me the results I want. Now that I am trying to do it with all 34 departments I find it is not making it through all of them before the script stops. (Actually it had happened with the few files as well at times but I thought I had gotten it to the point where it would consistently go through all five departments.) The error is always

C:\Program Files\AutoIt3\Include\IE.au3 (2878) : ==> The requested action with this object has failed.:

$o_object.GoBack()

$o_object.GoBack()^ ERROR

but it is not consistent in when it fails. Sometimes it stops early, say at ART, and another time I've gotten it to go all the way to SCIENCE & TECHNOLOGY. It has stopped everywhere in between. It will either sit at the "Save As" pop-up without clicking the "Save" or occasionally it will have gotten past that point but then the "save web page" pop-up sits there with a piece of paper flying from Earth to a file folder. In the latter case I have to use task manager to get out of IE. I'm using IE 6.

It seems to me it is exiting the "Save Table" function too soon and getting to the "Go Back" line in the first While loop too soon, resulting in the error since there is no "go back" from a SaveAs" pop-up. I searched the forum and added some Sleep lines, which seemed to be a common suggestion. At times that helps and at others it works better if I comment at least one of them out. I also tried switching from ControlClick to Send {enter} after getting focus on the Save button but it didn't make a difference.

Should I make my Sleep's longer? Make more of them? Is it just a function of variable times to save the file so there isn't any thing else to do?

expandcollapse popup

#cs ----------------------------------------------------------------------------
    AutoIt Version: 3.2.12.1
    Script Function:
    Copy book/media fund reports for each department to c: drive then post to web site.
#ce ----------------------------------------------------------------------------

#include <IE.au3>
#include <File.au3>
#include <inet.au3>
#include <Array.au3>
#include <WindowsConstants.au3>
Opt("TrayIconDebug", 1)
Opt("WinTitleMatchMode", 2)
$oIE = _IEAttach("Books/Media 2007/2008")
$oFrame = _IEFrameGetObjByName($oIE, "data")
$oLinks = _IELinkGetCollection($oFrame)

$Links = FileOpen("departments.txt", 0)
; Check if file opened for reading OK
If $Links = -1 Then
    MsgBox(0, "Error", "Unable to open file.")
    Exit
EndIf

; Read in lines of text until the EOF is reached
While 1
    $oFrame = _IEFrameGetObjByName($oIE, "data") ;this works to make it not lose the $oFrame
    $dept = FileReadLine($Links) ;file listing text of each hyperlink (fund name with space after it)
    If @error = -1 Then ExitLoop
    _IELinkClickByText($oFrame, $dept, 0, 0)
    Sleep(3000)
    _SaveTable() ;Call function that performs "Save As"
    Sleep (800)
    _IEAction($oIE, "back") ; not sure why $oIE worked when $hIE didn't. Obviously have a lot to learn
WEnd
FileClose($Links)
;Save second level funds e.g. Foreign Language> Chinese
Dim $Departments[3] = ["FOREIGN LANGUAGE", "HISTORY", "POLITICS & GOVERNMENT"]
For $X = 0 To UBound($Departments) - 1
    $oFrame = _IEFrameGetObjByName($oIE, "data")
    _IELinkClickByText($oFrame, $Departments[$X] & " ", 0, 0)
    Sleep(1000)
    $dept = FileOpen(@ScriptDir & "\" & $Departments[$X] & ".txt", 0)
    ; Check if file opened for reading OK
    If $dept = -1 Then
        MsgBox(0, "Error", "Unable to open file.")
        Exit
    EndIf
    ; Read in lines of text until the EOF is reached
    While 1
        $oFrame = _IEFrameGetObjByName($oIE, "data") ;needed or won't work
        $prof = FileReadLine($dept) ;file listing text of each hyperlink (prof name with space after it)
        If @error = -1 Then ExitLoop
        _IELinkClickByText($oFrame, $prof, 0, 0)
        Sleep(1000)
        _SaveTable() ;Call function that performs "Save As"
        Sleep (200)
        _IEAction($oIE, "back")
    WEnd
    FileClose($dept)
    _IEAction($oIE, "back") ; go back to Books Media table
Next
Sleep(200)

MsgBox(0, "Success", "Reports downloaded. Now removing cash balance column.")
_IEQuit ($oIE)
;Function to save current data frame
Func _SaveTable()
    $hIE = _IEPropertyGet($oIE, "hwnd")
    WinActivate($hIE)
    WinWaitActive($hIE)
    ControlSend($hIE, "", "", "!f") ; File
    ControlSend($hIE, "", "", "a") ; SaveAs
    WinWait("Save Web Page", "", 10)
    $hSave = WinGetHandle("Save Web Page", "")
    If @error Then
    MsgBox(4096, "Error", "Could not find the correct window")
        Else
    WinActivate($hSave)
    EndIf
    ControlFocus($hSave, "", "[CLASS:ComboBox; INSTANCE:3]"); move focus to saveastype box
    ControlSend($hSave, "", "[CLASS:ComboBox; INSTANCE:3]", "{DOWN}{ENTER}") ;change file type to web page complete
    Sleep(700) ;see if this works to make it click save before "go back"
    ControlFocus($hSave, "", "[CLASS:Button; INSTANCE:2]") ; move focus to save button
    Sleep(300) ;see if this works to make it click save before "go back"
    ;ControlClick($hSave, "", "[CLASS:Button; TEXT:&Save; INSTANCE:2]") ; click Save button
    Send("{Enter}"); hit enter to select Save
    Sleep(4000) ; need this or it gets to "go back" too fast and errors out
    $hSave2 = WinGetHandle("Save Web Page", "")
    ControlClick($hSave2, "", "[CLASS:Button; TEXT:&Yes; INSTANCE:1]") ;for "file already exists" alert; works even if no file exists
    Sleep(4000)
EndFunc   ;==>_SaveTable
;
;*********part that saves table_1x1 as an html document*******
;
$Links = FileOpen("reports.txt", 0) ;open list of all reports WITHOUT space after name
If $Links = -1 Then
    MsgBox(0, "Error", "Unable to open reports.txt.")
    Exit
EndIf
; Read in lines of text until the EOF is reached
While 1
    $dept = FileReadLine($Links) ;file listing each fund name)
    If @error = -1 Then ExitLoop ; need this or get stuck in continuous loop
$path = @ScriptDir & "\FundReports\" & $dept & "_files\table_1x1"
$dest = @ScriptDir & "\FundReports\Reports\" & $dept & ".html"
    FileCopy($path, $dest, 9)   
    Sleep (200)
;
;*******************part that removes Cash Balance column from html files**********
;
Dim $newtable

$tablex1 = FileOpen($dest, 0) ; Opens the file and puts the handle of it in the variable
; Check if file opened for reading OK
If $tablex1 = -1 Then
    MsgBox(0, "Error", "Unable to open individual department html file.")
    Exit
EndIf
$table=FileRead($tablex1)
FileClose($tablex1) ; Closes the first file. It is read and is not needed anymore
$newtable = StringSplit($table, "<T", 1) ;Try splitting on <T instead to get line we want always on 13th. 
$Pos = _ArraySearch ($newtable, "BODY>", 0, 0, 1, True) ; search array for BODY>, all uppercase; 
;**Now that we've found TBODY line need to go 13 lines to get Cash Balance then add 12 after that because after it removes line there are now 12 lines to next rather than 13
For $deleteline = ($Pos +13) to $newtable[0] Step 12
        _ArrayDelete ($newtable, $deleteline)
Next
Dim $sEditedTable = _ArrayToString( $newtable,"<T") ;borrowed this from a forum example. Not sure why I have to Dim
$sEditedTable = StringTrimLeft ($sEditedTable,4)
$EditedFile=FileOpen($dest, 2) ; this mode deletes what is there
FileWrite ($EditedFile, $sEditedTable)  
        
    WEnd
FileClose($Links)
MsgBox(0, "Success", "HTML files loaded into Reports folder")

Exit

Edited June 16, 2008 by SueB

[nobbe]

im not shure what you are really doing-- but i think its related to the different loading times of the webpages so it could help to use

_IELoadWait

Wait for a browser page load to complete before returning.

instead of the fixed time sleep() Edited June 16, 2008 by nobbe

[SueB]

Thanks for the idea Nobbe. The page is loading fine but I built on that idea to add some WinWaitActive lines. Still no luck. I've searched the web and newsgroups for IE freezing and tried some of the suggestions. I've cleaned my registry, tried to update my video driver, and disabled all IE add-ons except Java Runtime environment, which is needed for the page I am concerned with which is full of Java applets. It is a proprietary page created by our Integrated Library System so unfortunately I can't show it to you. I also compiled the script and tried to run it on someone else's computer to make sure it wasn't something I had running on mine. I really think it is a Window/IE issue rather than an AutoIt one. I may have to bag this idea of getting fund reports up where faculty can see them.

[PsaltyDS]

I have a script that goes through links of fund reports and saves a copy of the different resulting frames. I tested it first with just a few of the departments so I know the code is correct as far as getting me the results I want. Now that I am trying to do it with all 34 departments I find it is not making it through all of them before the script stops. (Actually it had happened with the few files as well at times but I thought I had gotten it to the point where it would consistently go through all five departments.) The error is always

C:\Program Files\AutoIt3\Include\IE.au3 (2878) : ==> The requested action with this object has failed.:

$o_object.GoBack()

$o_object.GoBack()^ ERROR

but it is not consistent in when it fails. Sometimes it stops early, say at ART, and another time I've gotten it to go all the way to SCIENCE & TECHNOLOGY. It has stopped everywhere in between. It will either sit at the "Save As" pop-up without clicking the "Save" or occasionally it will have gotten past that point but then the "save web page" pop-up sits there with a piece of paper flying from Earth to a file folder. In the latter case I have to use task manager to get out of IE. I'm using IE 6.

It seems to me it is exiting the "Save Table" function too soon and getting to the "Go Back" line in the first While loop too soon, resulting in the error since there is no "go back" from a SaveAs" pop-up. I searched the forum and added some Sleep lines, which seemed to be a common suggestion. At times that helps and at others it works better if I comment at least one of them out. I also tried switching from ControlClick to Send {enter} after getting focus on the Save button but it didn't make a difference.

Should I make my Sleep's longer? Make more of them? Is it just a function of variable times to save the file so there isn't any thing else to do?

CODE

#cs ----------------------------------------------------------------------------

AutoIt Version: 3.2.12.1

Script Function:

Copy book/media fund reports for each department to c: drive then post to web site.

#ce ----------------------------------------------------------------------------

#include <IE.au3>

#include <File.au3>

#include <inet.au3>

#include <Array.au3>

#include <WindowsConstants.au3>

Opt("TrayIconDebug", 1)

Opt("WinTitleMatchMode", 2)

$oIE = _IEAttach("Books/Media 2007/2008")

$oFrame = _IEFrameGetObjByName($oIE, "data")

$oLinks = _IELinkGetCollection($oFrame)

$Links = FileOpen("departments.txt", 0)

; Check if file opened for reading OK

If $Links = -1 Then

MsgBox(0, "Error", "Unable to open file.")

Exit

EndIf

; Read in lines of text until the EOF is reached

While 1

$oFrame = _IEFrameGetObjByName($oIE, "data") ;this works to make it not lose the $oFrame

$dept = FileReadLine($Links) ;file listing text of each hyperlink (fund name with space after it)

If @error = -1 Then ExitLoop

_IELinkClickByText($oFrame, $dept, 0, 0)

Sleep(3000)

_SaveTable() ;Call function that performs "Save As"

Sleep (800)

_IEAction($oIE, "back") ; not sure why $oIE worked when $hIE didn't. Obviously have a lot to learn

WEnd

FileClose($Links)

;Save second level funds e.g. Foreign Language> Chinese

Dim $Departments[3] = ["FOREIGN LANGUAGE", "HISTORY", "POLITICS & GOVERNMENT"]

For $X = 0 To UBound($Departments) - 1

$oFrame = _IEFrameGetObjByName($oIE, "data")

_IELinkClickByText($oFrame, $Departments[$X] & " ", 0, 0)

Sleep(1000)

$dept = FileOpen(@ScriptDir & "\" & $Departments[$X] & ".txt", 0)

; Check if file opened for reading OK

If $dept = -1 Then

MsgBox(0, "Error", "Unable to open file.")

Exit

EndIf

; Read in lines of text until the EOF is reached

While 1

$oFrame = _IEFrameGetObjByName($oIE, "data") ;needed or won't work

$prof = FileReadLine($dept) ;file listing text of each hyperlink (prof name with space after it)

If @error = -1 Then ExitLoop

_IELinkClickByText($oFrame, $prof, 0, 0)

Sleep(1000)

_SaveTable() ;Call function that performs "Save As"

Sleep (200)

_IEAction($oIE, "back")

WEnd

FileClose($dept)

_IEAction($oIE, "back") ; go back to Books Media table

Next

Sleep(200)

MsgBox(0, "Success", "Reports downloaded. Now removing cash balance column.")

_IEQuit ($oIE)

;Function to save current data frame

Func _SaveTable()

$hIE = _IEPropertyGet($oIE, "hwnd")

WinActivate($hIE)

WinWaitActive($hIE)

ControlSend($hIE, "", "", "!f") ; File

ControlSend($hIE, "", "", "a") ; SaveAs

WinWait("Save Web Page", "", 10)

$hSave = WinGetHandle("Save Web Page", "")

If @error Then

MsgBox(4096, "Error", "Could not find the correct window")

Else

WinActivate($hSave)

EndIf

ControlFocus($hSave, "", "[CLASS:ComboBox; INSTANCE:3]"); move focus to saveastype box

ControlSend($hSave, "", "[CLASS:ComboBox; INSTANCE:3]", "{DOWN}{ENTER}") ;change file type to web page complete

Sleep(700) ;see if this works to make it click save before "go back"

ControlFocus($hSave, "", "[CLASS:Button; INSTANCE:2]") ; move focus to save button

Sleep(300) ;see if this works to make it click save before "go back"

;ControlClick($hSave, "", "[CLASS:Button; TEXT:&Save; INSTANCE:2]") ; click Save button

Send("{Enter}"); hit enter to select Save

Sleep(4000) ; need this or it gets to "go back" too fast and errors out

$hSave2 = WinGetHandle("Save Web Page", "")

ControlClick($hSave2, "", "[CLASS:Button; TEXT:&Yes; INSTANCE:1]") ;for "file already exists" alert; works even if no file exists

Sleep(4000)

EndFunc ;==>_SaveTable

;

;*********part that saves table_1x1 as an html document*******

;

$Links = FileOpen("reports.txt", 0) ;open list of all reports WITHOUT space after name

If $Links = -1 Then

MsgBox(0, "Error", "Unable to open reports.txt.")

Exit

EndIf

; Read in lines of text until the EOF is reached

While 1

$dept = FileReadLine($Links) ;file listing each fund name)

If @error = -1 Then ExitLoop ; need this or get stuck in continuous loop

$path = @ScriptDir & "\FundReports\" & $dept & "_files\table_1x1"

$dest = @ScriptDir & "\FundReports\Reports\" & $dept & ".html"

FileCopy($path, $dest, 9)

Sleep (200)

;

;*******************part that removes Cash Balance column from html files**********

;

Dim $newtable

$tablex1 = FileOpen($dest, 0) ; Opens the file and puts the handle of it in the variable

; Check if file opened for reading OK

If $tablex1 = -1 Then

MsgBox(0, "Error", "Unable to open individual department html file.")

Exit

EndIf

$table=FileRead($tablex1)

FileClose($tablex1) ; Closes the first file. It is read and is not needed anymore

$newtable = StringSplit($table, "<T", 1) ;Try splitting on <T instead to get line we want always on 13th.

$Pos = _ArraySearch ($newtable, "BODY>", 0, 0, 1, True) ; search array for BODY>, all uppercase;

;**Now that we've found TBODY line need to go 13 lines to get Cash Balance then add 12 after that because after it removes line there are now 12 lines to next rather than 13

For $deleteline = ($Pos +13) to $newtable[0] Step 12

_ArrayDelete ($newtable, $deleteline)

Next

Dim $sEditedTable = _ArrayToString( $newtable,"<T") ;borrowed this from a forum example. Not sure why I have to Dim

$sEditedTable = StringTrimLeft ($sEditedTable,4)

$EditedFile=FileOpen($dest, 2) ; this mode deletes what is there

FileWrite ($EditedFile, $sEditedTable)

WEnd

FileClose($Links)

MsgBox(0, "Success", "HTML files loaded into Reports folder")

Exit

First, you need to be beaten thoroughly about the head and shoulders with a copy of "Windows 98Me For Dummies" for putting function declarations in the middle of your main script body. Go find somebody willing to do that for you.

No, take your beating out of the room. I abhor the sight of violence...

Now that's out of the way, perhaps this would make your _SaveTable() function more reliable (less blind send and sleep):

expandcollapse popup

;Function to save current data frame
Func _SaveTable()
    $hIE = _IEPropertyGet($oIE, "hwnd")
    WinActivate($hIE)
    WinWaitActive($hIE)
    
    ControlSend($hIE, "", "", "!f"); File
    ControlSend($hIE, "", "", "a"); SaveAs
    
    WinWait("Save Web Page", "", 10)
    $hSave = WinGetHandle("Save Web Page", "")
    If @error Then
        MsgBox(4096, "Error", "Could not find the correct window")
    Else
        WinActivate($hSave)
    EndIf
    
    ControlCommand($hSave, "", "[CLASS:ComboBox; INSTANCE:3]", "SelectString", "Web Page, complete");change file type to web page complete
    If Not StringInStr(ControlCommand($hSave, "", "[CLASS:ComboBox; INSTANCE:3]", "GetCurrentSelection"), "Web Page, complete") Then
        MsgBox(16, "Error", "Failed to select save type.")
    EndIf
    ControlClick($hSave, "", "[CLASS:Button; INSTANCE:2]"); Save
    
    While 1
    ; If Main save page is closed, done
        If Not WinExists($hSave) Then ExitLoop
        
    ; If there is a second save page, clik Yes on it
        $avWinList = WinList("Save Web Page")
        If $avWinList[0][0] > 1 Then
            For $n = 1 To $avWinList[0][0]
                If $avWinList[$n][1] <> $hSave Then
                ; Found new save window
                    ControlClick($avWinList[$n][1], "", "[CLASS:Button; TEXT:&Yes; INSTANCE:1]"); Click Yes
                EndIf
            Next
        EndIf
    WEnd
EndFunc  ;==>_SaveTable

The bruises? They'll heal, and your scripting will be better for it.

[SueB]

Thanks for the more sophisticated function script Psalty. Unfortunately it doesn't solve the problem. I replaced mine with yours and tried running it three times. My web page is a list of university departments in alphabetical order Asian Studies, Art, Biology, Business etc. For a few departments I want to also save pages the next level down (what I call second level in my script comments) such as clicking on Foreign Language, then clicking the Chinese link on that page and saving the resulting page. Likewise there are a few where I burrow down to the third level of links to save the page.

The first time it froze at Art. The second time it made it through all 34 departments but froze on the second level Chinese. Interestingly, this time the "save web page" pop-up wasn't visible but the error was the same. The third time it made it to Economics but froze saving the page.

[DaleHohm]

The error you reported comes from _IEAction($oIE, "back")

IE maintains a history list of pages visited. Forward and Back allow you to navigate within the entries of that list. All is well unless you try to go Back when you are already at the beginning of the list or go Forward when you are at the end. In COM code these operations generate a COM exception. You must either keep your bookkepping straight so that you never do this or you need to trap the exception so that it is not fatal to your code. You can trap the expection by adding _IEErrorHandlerRegister(). See the simple webbrowser example in _IECreateEmbedded for an example that exactly matches your situation.

Dale

[luzif3r]

If i did understand dale right, it might already help to not use the "back" feature but rather navigate to the new page to avoid producing the error. you might also try the error handling, which i personaly do not understand

[SmOke_N]

If i did understand dale right, it might already help to not use the "back" feature but rather navigate to the new page to avoid producing the error. you might also try the error handling, which i personaly do not understand

That's not what he's saying.

1. Keep track of what you are doing.

2. If you don't then use proper error handling to trap fatal error messages that would otherwise end your script.

[SueB]

I really appreciate the comments I've been getting. I'm afraid I wasn't detailed enough in my reply to Nobbe's suggestion. Adding WinWaitActive ($hIE) as the last line in my SaveTable function caused the GoBack error to stop because it wasn't exiting the function before the save completed. But it didn't stop the freezing. What is happening now is IE just stays frozen with the "Save web page" at 0% and I have to manually stop the script. When I click on the AutoIt symbol in the task tray the "script paused" is checked.

I did try adding _IEErrorHandlerRegister() at the beginning of my script, after the includes. Please correct me if I am misunderstanding how to use that. I have one version of the script with my SaveTable function and one with Psalty's. Neither one has completed the entire program. Again, they are not consistent in where they fail each time though I am consistently farther along when it stops (minimum of the 16th link instead of the 2nd).

My version is still pausing at the "save as" window so there was no ComError. With Psalty's version (without the WinWaitActive) once it saved the same page over and over and then failed when it went to the "part that saves table_1x1 as an html document". Among the ComError's were

expandcollapse popup

--> IE.au3 V2.4-0 Warning from function _IELinkClickByText, $_IEStatus_NoMatch 
--> COM Error Encountered in FundHelp.au3 
----> $IEComErrorScriptline = 2878 
----> $IEComErrorNumberHex = 80020009
 ----> $IEComErrorNumber = -2147352567
 ----> $IEComErrorWinDescription = The requested resource is in use.
 ----> $IEComErrorDescription =  
----> $IEComErrorSource = 
 ----> $IEComErrorHelpFile =  
----> $IEComErrorHelpContext = 0 
----> $IEComErrorLastDllError = 0 

 --> COM Error Encountered in FundHelp.au3 
----> $IEComErrorScriptline = 2878 
----> $IEComErrorNumberHex = 80020009 
----> $IEComErrorNumber = -2147352567
 ----> $IEComErrorWinDescription = The requested resource is in use. 
----> $IEComErrorDescription = 
 ----> $IEComErrorSource =  
----> $IEComErrorHelpFile =  
----> $IEComErrorHelpContext = 0
 ----> $IEComErrorLastDllError = 0oÝ÷ Ù8^¢Ø^®Ø¦z+lj÷¶¢+bjZZêÚ¶«¶§rX)àjX§Øn±Æ®²)ඪº`¡¶«¨¶Új)Sç«®ìÁêÞjëh×6+>11:41:42 AU3Check ended.rc:0
>Running:(3.2.12.1):C:\Program Files\AutoIt3\autoit3.exe "C:\Sue\simon stuff\AutoIt\FundHelp.au3"    
--> IE.au3 V2.4-0 Warning from function _IELinkClickByText, $_IEStatus_NoMatch 
--> COM Error Encountered in FundHelp.au3 
----> $IEComErrorScriptline = 767 
----> $IEComErrorNumberHex = 80020009 
----> $IEComErrorNumber = -2147352567 
----> $IEComErrorWinDescription = Access is denied. -
---> $IEComErrorDescription = Access is denied. 
----> $IEComErrorSource =  ----> $IEComErrorHelpFile = C:\WINDOWS\system32\mshtml.hlp 
----> $IEComErrorHelpContext = 0 
----> $IEComErrorLastDllError = 0 
 
--> COM Error Encountered in FundHelp.au3
 ----> $IEComErrorScriptline = 2547 
----> $IEComErrorNumberHex = 80020009 
----> $IEComErrorNumber = -2147352567
 ----> $IEComErrorWinDescription = Access is denied. 
----> $IEComErrorDescription = Access is denied. 
----> $IEComErrorSource =  ----> $IEComErrorHelpFile = C:\WINDOWS\system32\mshtml.hlp
 ----> $IEComErrorHelpContext = 0 
----> $IEComErrorLastDllError = 0 

 C:\Program Files\AutoIt3\Include\IE.au3 (2597) : ==> Object referenced outside a "With" statement.:
SetExtended($oTemp.GetElementsByTagName($s_TagName).length)
SetExtended($oTemp.GetElementsByTagName($s_TagName)^ ERROR
->11:43:17 AutoIT3.exe ended.rc:1

[DaleHohm]

You'll need to check @error after your calls to _IE* functions and take appropriate actions.

You may also want to play with SciTe Debug mode to pinpoint the source of the errors.

Dale