Jump to content
trancexx

WinHTTP functions

Recommended Posts

_WinHttpSetOption($hRequest, $WINHTTP_OPTION_DISABLE_FEATURE, $WINHTTP_DISABLE_REDIRECTS)

Does this Code Enable or Disable Redirects..?

I was trying to get a cookie from a website and could only get it by using that code.

but when I tried to longin to a website using that same code in my script, I couldn't login..?

someone explain me whats going on with both of these Response Headers: HTTP/1.1 200 OK & HTTP/1.1 302 Found...?

Share this post


Link to post
Share on other sites

If using this WinHTTP Function within multiple .exe's simultaneously, will it flood the DLL call or something..??

Cause I have 200 .exe's all running simultaneously performing some database gathering on a website using my Win7 X64 i7 980X CPU with 12GB Ram PC and I'm noticing some stalling/lagging ever few seconds..

Not sure if its the website lagging or the multiple calls to use the WinHTTP Function from all the multiple processes..?

Im also using multiple fast proxies so I can overcome the max connections per IP to the server restriction..

What all can I check for to see what might be causing this Lag..?

If I use only 4 processes of my script, I notice no lag at all..?

so what do guize..?

Share this post


Link to post
Share on other sites

_WinHttpSetOption($hRequest, $WINHTTP_OPTION_DISABLE_FEATURE, $WINHTTP_DISABLE_REDIRECTS)

Does this Code Enable or Disable Redirects..?

This disables redirection.

I was trying to get a cookie from a website and could only get it by using that code.

but when I tried to longin to a website using that same code in my script, I couldn't login..?

someone explain me whats going on with both of these Response Headers: HTTP/1.1 200 OK & HTTP/1.1 302 Found...?

302 Found means: The current URL does not exists, but the document was found at another location. The browser should look there. Since you have disabled redirects, you have to do this manually. Read the loaction header, which should contain the new url.

200 OK means, that everything worked and you got the document you requested.


*GERMAN* [note: you are not allowed to remove author / modified info from my UDFs]My UDFs:[_SetImageBinaryToCtrl] [_TaskDialog] [AutoItObject] [Animated GIF (GDI+)] [ClipPut for Image] [FreeImage] [GDI32 UDFs] [GDIPlus Progressbar] [Hotkey-Selector] [Multiline Inputbox] [MySQL without ODBC] [RichEdit UDFs] [SpeechAPI Example] [WinHTTP]UDFs included in AutoIt: FTP_Ex (as FTPEx), _WinAPI_SetLayeredWindowAttributes

Share this post


Link to post
Share on other sites

I don't know why I receive ERROR_INTERNET_INCORRECT_HANDLE_STATE when try to get responds from 'webmail.stu.edu.cn' in either windows 7 x86 or x64(caused by _WinHttpQueryDataAvailable()), here is the simple code.

#include "WinHttp.au3"

Global $hOpen = _WinHttpOpen("Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.0.3; .NET CLR 2.0.50727; ffco7) Gecko/2008092417 Firefox/3.0.3")

Global $hConnect = _WinHttpConnect($hOpen, "webmail.stu.edu.cn")

$Var = _WinHttpSimpleRequest($hConnect)
If @error Then MsgBox(64,@error,$Var)
; Close handles
_WinHttpCloseHandle($hConnect)
_WinHttpCloseHandle($hOpen)

Thanks

Share this post


Link to post
Share on other sites

The error is raised by _WinHttp_ReceiveResponse, but I don't know the cause. Probably the server does not answer to our request, but I cannot understand why this would happen.


*GERMAN* [note: you are not allowed to remove author / modified info from my UDFs]My UDFs:[_SetImageBinaryToCtrl] [_TaskDialog] [AutoItObject] [Animated GIF (GDI+)] [ClipPut for Image] [FreeImage] [GDI32 UDFs] [GDIPlus Progressbar] [Hotkey-Selector] [Multiline Inputbox] [MySQL without ODBC] [RichEdit UDFs] [SpeechAPI Example] [WinHTTP]UDFs included in AutoIt: FTP_Ex (as FTPEx), _WinAPI_SetLayeredWindowAttributes

Share this post


Link to post
Share on other sites

The error is raised by _WinHttp_ReceiveResponse, but I don't know the cause. Probably the server does not answer to our request, but I cannot understand why this would happen.

Yet, this error is very strange. I try to use 'xmlhttp' but also gets an error.

I think is a bug of WinHttp.dll. What's your opinion?

Now my way to solve this problem is to use WinInet. It works well in windows 7.

Share this post


Link to post
Share on other sites

It's not a bug. WinHTTP refuses to process invalid responses by the server. This is valid reaction. If some other API allows it, use that.

What's invalid?

HTTP/1.1 200 OKDate: Fri, 11 Mar 2011 17:57:45 GMTServer: Apache/2.2.6 (Unix) mod_ssl/2.2.6 đˇ'    PHP/5.2.4X-Powered-By: PHP/5.2.4Set-Cookie: USER=deleted; expires=Thu, 11-Mar-2010 17:57:44 GMTTransfer-Encoding: chunkedContent-Type: text/htmlX-Pad: avoid browser bug

...That Server line.

edit:

#include <WinAPI.au3>
#include "WinHttp.au3"

Global $hOpen = _WinHttpOpen()
Global $hConnect = _WinHttpConnect($hOpen, "webmail.stu.edu.cn")
Global $hRequest = _WinHttpOpenRequest($hConnect)
_WinHttpSendRequest($hRequest)
_WinHttpReceiveResponse($hRequest)
If _WinAPI_GetLastError() = $ERROR_WINHTTP_INVALID_SERVER_RESPONSE Then MsgBox(48 + 262144, "Error 1", "ERROR_WINHTTP_INVALID_SERVER_RESPONSE")

; Get full header
Global $sHeader = _WinHttpQueryHeaders($hRequest)
; As the result of the previous error
If _WinAPI_GetLastError() = $ERROR_WINHTTP_INCORRECT_HANDLE_STATE Then MsgBox(48 + 262144, "Error 2", "ERROR_WINHTTP_INCORRECT_HANDLE_STATE")

_WinHttpCloseHandle($hRequest)
_WinHttpCloseHandle($hConnect)
_WinHttpCloseHandle($hOpen)

ConsoleWrite($sHeader & @CRLF)
Edited by trancexx

♡♡♡

.

eMyvnE

Share this post


Link to post
Share on other sites

Thank you so much trancexx and ProgAndy for this terrific UDF and great examples helping get started with it!

I was just wondering is it possible to "converse" somehow HTML document into an XML* on which it would be possible to operate with javascript to obtain some data from the page?

* I guess HTML is already an extended version of XML... So it's not the smartest question ever, but I hope you'll get my point :)

And wouldn't it be a game not worth a candle? I guess every .getElementsBy* operations can be successfully substituted with RegExp's functions, as in _WinHttpSimpleFormFill...

Thank you in advance!

Edited by 4ggr35510n

Share this post


Link to post
Share on other sites

@4ggr35510n: It depends on the format whether you can parse it as XML. If it is valid XHTML you can do it, but many websites are written in "normal" wich is not XML conform. And the websites in XHTML are often not valid, so a strict parser won't accept it.

Maybe someone is willing to give libxml2 a try, but currently I don't feel like creating a UDFs from scratch.


*GERMAN* [note: you are not allowed to remove author / modified info from my UDFs]My UDFs:[_SetImageBinaryToCtrl] [_TaskDialog] [AutoItObject] [Animated GIF (GDI+)] [ClipPut for Image] [FreeImage] [GDI32 UDFs] [GDIPlus Progressbar] [Hotkey-Selector] [Multiline Inputbox] [MySQL without ODBC] [RichEdit UDFs] [SpeechAPI Example] [WinHTTP]UDFs included in AutoIt: FTP_Ex (as FTPEx), _WinAPI_SetLayeredWindowAttributes

Share this post


Link to post
Share on other sites

I don't feel like needing UDF :]

But it would be very helpful if you could just point me a bit in a right direction :)

I can handle with all autoit DllCall/DllStruct functions.

I believe most functions I need can be found here:

http://xmlsoft.org/html/libxml-HTMLparser.html

But I have some language and knowledge issues to understanding the proper way to do it.

Should I just use htmlCreateMemoryParserCtxt() to create ParserCtxt from "char" Structure (received as in _WinHTTPReadData()) and then htmlCtxtReadDoc() to "parse an XML in-memory document and build a tree"?

I know I don't know a lot about that :)

I'll really appreciate you help!

Hoping for your help,

4gr

Share this post


Link to post
Share on other sites

Hi,

can someone tell me how I use WinHTTP at a proxy with authentication?

I have thie script:

#include <winhttp.au3>

$sProxyServer = "PROXY"
$hw_open = _WinHttpOpen("AutoIt v3 WinHTTP with Proxy ", $WINHTTP_ACCESS_TYPE_NAMED_PROXY, $sProxyServer)
_WinHttpSetOption($hw_open, $WINHTTP_OPTION_PROXY_PASSWORD, "PASSWORD")
_WinHttpSetOption($hw_open, $WINHTTP_OPTION_PROXY_USERNAME, "USERNAME")

$hw_connect = _WinHttpConnect($hw_open, "www.wieistmeineip.de")
$h_openRequest = _WinHttpOpenRequest($hw_connect)
_WinHttpSendRequest($h_openRequest)
_WinHttpReceiveResponse($h_openRequest)
MsgBox(0, '', _WinHttpQueryHeaders($h_openRequest))
If _WinHttpQueryDataAvailable($h_openRequest) Then
    Local $data=""
    Do
        $data&=_WinHttpReadData($h_openRequest)
    Until @error
    MsgBox(0, '', $data)
EndIf

_WinHttpCloseHandle($h_openRequest)
_WinHttpCloseHandle($hw_connect)
_WinHttpCloseHandle($hw_open)

But the MsgBox only deliver these errors#

HTTP/1.0 407 Proxy Authentication Required

Connection: close

Date: Mon, 14 Mar 2011 10:51:15 GMT

Via: 1.0 proxy.********.de:8888 (squid/2.7.STABLE9)

Content-Length: 1331

Content-Type: text/html

Proxy-Authenticate: Basic realm="Squid proxy-caching web server"

X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0

X-Cache: MISS from proxy.*********.de

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">

<HTML><HEAD><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">

<TITLE>ERROR: Cache Access Denied</TITLE>

<STYLE type="text/css"><!--BODY{background-color:#ffffff;font-family:verdana,sans-serif}PRE{font-family:sans-serif}--></STYLE>

</HEAD>

<BODY>

<H1>ERROR</H1>

<H2>Cache Access Denied</H2>

<HR noshade size="1px">

<P>

While trying to retrieve the URL:

<A HREF="http://www.wieistmeineip.de/">http://www.wieistmeineip.de/</A>

<P>

The following error was encountered:

<UL>

<LI>

<STRONG>

Cache Access Denied.

</STRONG>

</UL>

</P>

<P>Sorry, you are not currently allowed to request:

<PRE> http://www.wieistmeineip.de/</PRE>

from this cache until you have authenticated yourself.

</P>

<P>

You need to use Netscape version 2.0 or greater, or Microsoft Internet

Explorer 3.0, or an HTTP/1.1 compliant browser for this to work. Please

contact the <A HREF="mailto:root">cache administrator</a> if you have

difficulties authenticating yourself or

<A HREF="http://proxy.*******.de/cgi-bin/chpasswd.cgi">change</a> your default password.

</P>

<BR clear="all">

<HR noshade size="1px">

<ADDRESS>

Generated Mon, 14 Mar 2011 10:52:35 GMT by proxy.*******.de (squid/2.7.STABLE9)

</ADDRESS>

</BODY></HTML>

Share this post


Link to post
Share on other sites

The passwords must be set for each request handle and cannot be stored globally. This means, you cannot use _WinHttpSimpleRequest.

#include<WinHTTP.au3>
#include<WinApi.au3>
Global Const $tagWINHTTP_PROXY_INFO = "DWORD  dwAccessType;ptr lpszProxy;ptr lpszProxyBypass;"

Func _WinHttpProxyInfoCreate($dwAccessType, $sProxy, $sProxyBypass)
    Local $tWINHTTP_PROXY_INFO[2] = [DllStructCreate($tagWINHTTP_PROXY_INFO), DllStructCreate('wchar proxychars[' & StringLen($sProxy)+1 & ']; wchar proxybypasschars[' & StringLen($sProxyBypass)+1 & ']')]
    DllStructSetData($tWINHTTP_PROXY_INFO[0], "dwAccessType", $dwAccessType)
    If StringLen($sProxy) Then DllStructSetData($tWINHTTP_PROXY_INFO[0], "lpszProxy", DllStructGetPtr($tWINHTTP_PROXY_INFO[1], 'proxychars'))
    If StringLen($sProxyByPass) Then DllStructSetData($tWINHTTP_PROXY_INFO[0], "lpszProxyBypass", DllStructGetPtr($tWINHTTP_PROXY_INFO[1], 'proxybypasschars'))
    DllStructSetData($tWINHTTP_PROXY_INFO[1], "proxychars", $sProxy)
    DllStructSetData($tWINHTTP_PROXY_INFO[1], "proxybypasschars", $sProxyBypass)
    Return $tWINHTTP_PROXY_INFO
EndFunc

$hInternet = _WinHttpOpen()

$hConnect = _WinHttpConnect($hInternet, "api.hostip.info")
$standard = _WinHttpSimpleRequest($hConnect, "GET", "/get_html.php")
_WinHttpCloseHandle($hConnect)


$tProxyInfo = _WinHttpProxyInfoCreate($WINHTTP_ACCESS_TYPE_NAMED_PROXY, "127.0.0.1:8123", "localhost")
_WinHttpSetOption($hInternet, $WINHTTP_OPTION_PROXY, $tProxyInfo[0])

$hConnect = _WinHttpConnect($hInternet, "api.hostip.info")

_WinHttpSetOption($hConnect, $WINHTTP_OPTION_PROXY_USERNAME, "test")
_WinHttpSetOption($hConnect, $WINHTTP_OPTION_PROXY_PASSWORD, "pass")

$mit_proxy = ""
$hRequest = _WinHttpOpenRequest($hConnect, "GET", "/get_html.php")
If $hRequest Then
    ; Set password here
    _WinHttpSetCredentials($hRequest, $WINHTTP_AUTH_TARGET_PROXY, $WINHTTP_AUTH_SCHEME_BASIC, "test", "pass")
    If _WinHttpSendRequest($hRequest) Then
        _WinHttpReceiveResponse($hRequest)
        $mit_proxy = _WinHttpSimpleReadData($hRequest)
    EndIf
    _WinHttpCloseHandle($hRequest)
EndIf

_WinHttpCloseHandle($hConnect)

PS: I tested with polipo and this configuration file:

dnsNameServer = "8.8.8.8"
authCredentials = "test:pass"

*GERMAN* [note: you are not allowed to remove author / modified info from my UDFs]My UDFs:[_SetImageBinaryToCtrl] [_TaskDialog] [AutoItObject] [Animated GIF (GDI+)] [ClipPut for Image] [FreeImage] [GDI32 UDFs] [GDIPlus Progressbar] [Hotkey-Selector] [Multiline Inputbox] [MySQL without ODBC] [RichEdit UDFs] [SpeechAPI Example] [WinHTTP]UDFs included in AutoIt: FTP_Ex (as FTPEx), _WinAPI_SetLayeredWindowAttributes

Share this post


Link to post
Share on other sites

Is this going to be Fixed in the next release? >>

I was having a problem with Obfuscator and then remembered there was a discussion sometime go about it. I took your advice and edited the UDF and now it works :)


UDF List:

 
_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_ArrayFilter/_ArrayReduce_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 22/04/2018

Share this post


Link to post
Share on other sites

OK fair point then.


UDF List:

 
_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_ArrayFilter/_ArrayReduce_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 22/04/2018

Share this post


Link to post
Share on other sites

OK fair point then.

Yep. When we make other changes, we maybe could split this long string into parts and add some information to the helpfile that #Obfuscator_Ignore_Variables has to be added to the mainscript, but that's it since #Obfuscator_Ignore_Variables is only recognized in the mainscript. I wish options would be recognized in all UDFs and that there was something like #Obfuscator_Ignore_Parameters_Of_Function=_FunctionName

*GERMAN* [note: you are not allowed to remove author / modified info from my UDFs]My UDFs:[_SetImageBinaryToCtrl] [_TaskDialog] [AutoItObject] [Animated GIF (GDI+)] [ClipPut for Image] [FreeImage] [GDI32 UDFs] [GDIPlus Progressbar] [Hotkey-Selector] [Multiline Inputbox] [MySQL without ODBC] [RichEdit UDFs] [SpeechAPI Example] [WinHTTP]UDFs included in AutoIt: FTP_Ex (as FTPEx), _WinAPI_SetLayeredWindowAttributes

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...