Narrowing _INetGetSource results

[Bunn1]

Hello all,

I am looking for a way to narrow the results of _INetGetSource, or rather limit the amount of data that is searched. The current _INetGetSource pulls the entire page source top to bottom, and for most applications this is fine, however, I am looking to speed up the process and only need specific data. I have modified the _INetGetSource function to allow the user to specify a search term and have limited the number of characters returned, this still requires the function to iteratively step through the page source until the search term is found. So basically this only helps in the post processing of the data(smaller chunk to deal with) and cuts off the source results following the discovery of the search term. I need a way to narrow the scope of the search to a specific range, I have thought about using the wininet.dll InternetSetFilePointer function, however, my knowledge of windows dll scripting is laughable at best. I was able to tweek the _INetGetSource function, but all of my attempts to implement the InternetSetFilePointer function have all ended in failure. This either due to the fact that my skill at dll scripting is not up to snuff or possibly the server that I am trying to query does not support random access, however, I do not not how to verify this or not. If anyone has had experience with this or has any ideas on how I should proceed, your insight would be greatly appreciated.

My modified INetGetSource is below:

expandcollapse popup

Func _INetGetSource2($s_URL,$schtrm, $s_Header = '')
    If StringLeft($s_URL, 7) <> 'http://' And StringLeft($s_URL, 8) <> 'https://' Then $s_URL = 'http://' & $s_URL

    Local $h_DLL = DllOpen("wininet.dll")

    Local $ai_IRF, $s_Buf = ''

    Local $ai_IO = DllCall($h_DLL, 'int', 'InternetOpen', 'str', "AutoIt v3", 'int', 0, 'int', 0, 'int', 0, 'int', 0)
    If @error Or $ai_IO[0] = 0 Then
        DllClose($h_DLL)
        SetError(1)
        Return ""
    EndIf

    Local $ai_IOU = DllCall($h_DLL, 'int', 'InternetOpenUrl', 'int', $ai_IO[0], 'str', $s_URL, 'str', $s_Header, 'int', StringLen($s_Header), 'int', 0x80000000, 'int', 0)
    If @error Or $ai_IOU[0] = 0 Then
        DllCall($h_DLL, 'int', 'InternetCloseHandle', 'int', $ai_IO[0])
        DllClose($h_DLL)
        SetError(1)
        Return ""
    EndIf

    Local $v_Struct = DllStructCreate('udword')
    DllStructSetData($v_Struct, 1, 1)
    $y = 0
    While DllStructGetData($v_Struct, 1) <> 0
        $ai_IRF = DllCall($h_DLL, 'int', 'InternetReadFile', 'int', $ai_IOU[0], 'str', '', 'int', 356, 'ptr', DllStructGetPtr($v_Struct))
        If $y = 1 Then; adds one additional read to ensure that the data being looked for is included in the selection
            $s_Buf &= StringLeft($ai_IRF[2], DllStructGetData($v_Struct, 1))
            ExitLoop
        EndIf   
        if StringInStr($ai_IRF[2],$schtrm) and $y = 0 Then; only collects information that mathces the criteria passed to the function
            $s_Buf &= StringLeft($ai_IRF[2], DllStructGetData($v_Struct, 1))
            $y = $y + 1 
        EndIf
    WEnd

    DllCall($h_DLL, 'int', 'InternetCloseHandle', 'int', $ai_IOU[0])
    DllCall($h_DLL, 'int', 'InternetCloseHandle', 'int', $ai_IO[0])
    DllClose($h_DLL)
    Return $s_Buf
EndFunc

thanks in advance

[evilertoaster]

I think you'll mostly likely be getting the whole file back no matter how you request it... So i don't know if it's possible to speed it up by only getting part of the page's content.

Have a look here though for some ideas-

http://www.autoitscript.com/forum/index.php?showtopic=18135

and

http://www.autoitscript.com/forum/index.php?showtopic=30409 (which leads to http://www.autoitscript.com/forum/index.php?showtopic=77503 and http://www.tek-tips.com/faqs.cfm?fid=4579)

[Bunn1]

Thanks for the reply, I looked over the posts you suggested, however, the information I am looking for is not found in the header and that is what both of these posts cover. There was mention of this posting in your second suggestion http://www.autoitscript.com/forum/index.php?showtopic=77503 though, it contains a UDF called WinINET.au3 written by Ultima, It may contain some useful functionality, I'll post back when I've had time to review it.