Script to Open IE, Save URL and page content into flat file, then close

[xeshan]

Hello AutoIters

I am just looking for a push in the right direction.

I am wanting to have a script to:

1. check all open Internet Explorer pages (pages are already open, so no need to open them specifically)
2 save the URL and the content on the Internet Explorer page into a flat file or db
3. then close the IE page

Would this be possible?

Thank you so much for your response.

Edited July 16, 2013 by xeshan

[DW1]

1) Easy enough, just loop through _IEAttach()

2) Sure, save an MHT file from the current url.

3) Simple enough, just closing the instances from #1.

Something like this:

expandcollapse popup

#include <IE.au3>
#include <Date.au3>

;Directory to save to
Local $sDir = @DesktopDir & '\WebPages'

;Get all IE urls
Local $i = 1
Local $aURL[100][3]
While 1
    $aURL[$i - 1][0] = _IEAttach('', 'Instance', $i)
    If @error Then ExitLoop
    $aURL[$i - 1][1] = _IEPropertyGet($aURL[$i - 1][0], 'locationurl')
    $aURL[$i - 1][2] = _IEPropertyGet($aURL[$i - 1][0], 'title')
    $i += 1
WEnd
ReDim $aURL[$i - 1][3]

;Add trailing slash to $sDir if not there already
If StringRight($sDir, 1) <> '\' Then $sDir &= '\'

;Create $sDir if it doesn't exist
If Not FileExists($sDir) Then DirCreate($sDir)

;Create a date/time directory
Local $sDateTime = StringReplace(StringReplace(_Now(), '/', '-'), ':', '-')
If Not FileExists($sDir & $sDateTime) Then DirCreate($sDir & $sDateTime)

;Loop through the URLs
For $i = 0 To UBound($aURL) - 1
    ;Save URL to MHT
    _INetGetMHT($aURL[$i][1], $sDir & $sDateTime & '\' & $aURL[$i][2] & '.MHT')
    ;Close the IE instance
    _IEQuit($aURL[$i][0])
Next

;Function to create MHTML file from a URL
Func _INetGetMHT($url, $file)
    Local $msg = ObjCreate("CDO.Message")
    If @error Then Return False
    Local $ado = ObjCreate("ADODB.Stream")
    If @error Then Return False

    With $ado
        .Type = 2
        .Charset = "US-ASCII"
        .Open
    EndWith
    $msg.CreateMHTMLBody($url, 0)
    $msg.DataSource.SaveToObject($ado, "_Stream")
    FileDelete($file)
    $ado.SaveToFile($file, 1)
    $msg = ""
    $ado = ""
    Return True
EndFunc   ;==>_INetGetMHT

[xeshan]

Thank you so much Danwilli!

I wanted to add - if I needed to capture (rather than save) the content into a flat file would that be possible?

In other words, there are 2 fields that I will be capturing. The first is the URL of the page. The second is whatever text is on the page. 

Then write to a single text file. 

Would that be possible?

Many many thanks for your help.

[DW1]

That makes it even easier since you don't need the pages media content.

#include <IE.au3>
#include <Date.au3>

;Directory to save to
Local $sDir = @DesktopDir & '\WebPages'

;Add trailing slash to $sDir if not there already
If StringRight($sDir, 1) <> '\' Then $sDir &= '\'

;Create $sDir if it doesn't exist
If Not FileExists($sDir) Then DirCreate($sDir)

;Create a date/time directory
Local $sDateTime = StringReplace(StringReplace(_Now(), '/', '-'), ':', '-')
If Not FileExists($sDir & $sDateTime) Then DirCreate($sDir & $sDateTime)

;Loop through IE instances and record the data (URL / Text)
Local $oIE, $i = 1
While 1
    $oIE = _IEAttach('', 'Instance', $i)
    If @error Then ExitLoop
    FileWrite($sDir & $sDateTime & '\Instance ' & $i & '.txt', _IEPropertyGet($oIE, 'locationurl') & @CRLF & @CRLF & _IEBodyReadText($oIE))
    $i += 1
WEnd

[xeshan]

Danwilli,

You are brilliant! Thank you for responding and answering so quickly!

I know this is not what I asked originally - (since I asked about saving to a flat file), but would an insertion into a sql db be possible as well?

(Assuming this is in inside the loop)

$constrim="DRIVER={SQL Server};SERVER=sqlserver;DATABASE=mydatabase;uid=myusername;pwd=mypassword;"
$adCN = ObjCreate ("ADODB.Connection") ;<==Create SQL connection
$adCN.Open ($constrim) ;<==Connect with required credentials
MsgBox(0,"",$constrim )
if @error Then
  MsgBox(0, "Error", "No connection to database")
  Exit
Else
  MsgBox(0, "Success!", "Connection to database successful")
EndIf
$sQuery="update mytable set Body = & _IEBodyReadText($oIE) where LocationURL = locationurl)"

$result= $adCN.Execute($sQuery)
$adCN.Close ;==>Close the database

[DW1]

Yes that is possible, however your query is not written correctly.

I don't have a ton of experience with SQL, so I cannot guarantee that the query is correct, but the autoit portion has been fixed (You had the literal text "_IEBodyReadText($oIE)" instead of the text the function would return.

$sQuery="UPDATE mytable SET Body='" & _IEBodyReadText($oIE) & "' WHERE LocationURL=locationurl"

It may be best to store this data as binary since we do not know what it will contain.  Like I said though, I'm no SQL guru.

Edited July 19, 2013 by danwilli