morganlove Posted July 11, 2014 Share Posted July 11, 2014 (edited) '?do=embed' frameborder='0' data-embedContent>> Hi. I’m new to Autoit and have been using the script at the bottom of the above thread to save a list of thousands of websites. The script looks like this X5000. expandcollapse popup_INetGetMHT1( "http://www.site1.com, "C:test1.MHT" ) Func _INetGetMHT1( $url, $file ) Local $msg = ObjCreate("CDO.Message") If @error Then Return False Local $ado = ObjCreate("ADODB.Stream") If @error Then Return False With $ado .Type = 2 .Charset = "US-ASCII" .Open EndWith $msg.CreateMHTMLBody($url, 0) $msg.DataSource.SaveToObject($ado, "_Stream") FileDelete($file) $ado.SaveToFile($file, 1) $msg = "" $ado = "" Return True EndFunc _INetGetMHT2( "http://www.site2.com, "C:test2.MHT" ) Func _INetGetMHT2( $url, $file ) Local $msg = ObjCreate("CDO.Message") If @error Then Return False Local $ado = ObjCreate("ADODB.Stream") If @error Then Return False With $ado .Type = 2 .Charset = "US-ASCII" .Open EndWith $msg.CreateMHTMLBody($url, 0) $msg.DataSource.SaveToObject($ado, "_Stream") FileDelete($file) $ado.SaveToFile($file, 1) $msg = "" $ado = "" Return True EndFunc Anyways, there are some sites that are particularly image-heavy and the MHT script can’t save them. This causes Autoit to crash ("AutoIt v3 Script has stopped working. A problem caused the program to stop working correctly. Please close the program") and makes me have to restart it after editing it to start on the next Func after the bad page (done by deleting all the Funcs before the one I want to start on.) If the script can’t save _INetGetMHT1320, is there a way to make it simply skip to _INetGetMHT1321 without crashing? Edited July 11, 2014 by morganlove Link to comment Share on other sites More sharing options...
kylomas Posted July 11, 2014 Share Posted July 11, 2014 (edited) morganlove, Did you actually create 5000 functions? If so, a better technique would be to create one function with two parms, WEB Site and Output File Name. Then create a file of WEB site names and output file names, read the file and call the function for each line in the file. You can create another file of what has been successfully processed and anytime the script starts, reprocess the last file or skip to the next file and continue. Just spitballing... kylomas edit: I see that it is already a function. So that part is done. Edited July 11, 2014 by kylomas Forum Rules Procedure for posting code "I like pigs. Dogs look up to us. Cats look down on us. Pigs treat us as equals." - Sir Winston Churchill Link to comment Share on other sites More sharing options...
morganlove Posted July 11, 2014 Author Share Posted July 11, 2014 Thanks for the response. That sounds like a much better idea than what I’ve been doing. (Yes, thousands of Funcs. I had been wondering if there were a way to have it read a list, but I was hesitant to modify the script and this had been working for me so far.) Could you maybe point me to resources that explain how to do that, please? Link to comment Share on other sites More sharing options...
kylomas Posted July 11, 2014 Share Posted July 11, 2014 morgangrove, Am working up a rudimentary example. Have you read the Help file? kylomas Forum Rules Procedure for posting code "I like pigs. Dogs look up to us. Cats look down on us. Pigs treat us as equals." - Sir Winston Churchill Link to comment Share on other sites More sharing options...
kylomas Posted July 11, 2014 Share Posted July 11, 2014 morganlove, Some really basic code to get you started. I have not addressed restarting yet. Let's see where you go with this. You will find everything in the HELP file. If you run into problems look at the Help file then ask if you can't work it out. expandcollapse popup#include <array.au3> #include <MsgBoxConstants.au3> #include <FileConstants.au3> ;---------------------------------------------------------------------------------------------------------------------- ; create a test file ;---------------------------------------------------------------------------------------------------------------------- ; file structure: ; web site name,file name local $hfl = fileopen('c:\tmp\mytestfile.txt',$FO_OVERWRITE) if $hfl = -1 Then msgbox($MB_ICONERROR,'ERROR','File Open Error for test file') Exit endif local $sOut for $1 = 1 to 500 $sOut &= 'WebSite' & stringformat('%03i',$1) & ',File#' & stringformat('%03i',$1) & @crlf Next filewrite($hfl,$sOut) fileclose($hfl) ;---------------------------------------------------------------------------------------------------------------------- ; End create a test file ;---------------------------------------------------------------------------------------------------------------------- local $aScraperFile = filereadtoarray("c:\TMP\MyTestFile.txt") ; This is the file you have created in the above format _arraydisplay($aScraperFile) ; For Debugging purposes ; this loops through the array that was just created from your file (one row per line) local $aTMP for $1 = 0 to ubound($aScraperFile) - 1 $aTMP = stringsplit($aScraperFile[$1],',',3) _InetGetMHT( $aTMP[0], $aTMP[1] ) next Func _INetGetMHT($url, $file) ConsoleWrite('Writing ' & $url & ' to ' & $file & @CRLF) Local $msg = ObjCreate("CDO.Message") If @error Then Return False Local $ado = ObjCreate("ADODB.Stream") If @error Then Return False With $ado .Type = 2 .Charset = "US-ASCII" .Open EndWith $msg.CreateMHTMLBody($url, 0) $msg.DataSource.SaveToObject($ado, "_Stream") FileDelete($file) $ado.SaveToFile($file, 1) $msg = "" $ado = "" Return True EndFunc ;==>_INetGetMHT Good Luck, kylomas Forum Rules Procedure for posting code "I like pigs. Dogs look up to us. Cats look down on us. Pigs treat us as equals." - Sir Winston Churchill Link to comment Share on other sites More sharing options...
morganlove Posted July 11, 2014 Author Share Posted July 11, 2014 Thank you so much! I’m going to try it now. I know I need to read more of the documentation. I just recently got into Autoit for this project and have mostly been reading the threads and pages about saving files using IE. I still need to learn to write my own scripts because I’ve just been copy-pasting. Link to comment Share on other sites More sharing options...
kylomas Posted July 12, 2014 Share Posted July 12, 2014 Keep at it, we all start somewhere and this forum has outstanding technical people willing to contribute. Been mulling over how to handle the script crashing. My network knowledge is limited and I don't think that there is any kind of abend/termination hook that we can use. The best that I can come up with right now is to establish a timer at the start of the loop and retun a code when the timer is exceeded. Perhaps a network guru will chime in. kylomas Forum Rules Procedure for posting code "I like pigs. Dogs look up to us. Cats look down on us. Pigs treat us as equals." - Sir Winston Churchill Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now