Again WinHTTP and Proxy

[Simpel]

Hi there.

I'm testing this code:

expandcollapse popup

#include "WinHttp.au3"

; http://www.w3schools.com/php/demo_form_validation_escapechar.php
$sUserName = "SomeUserName"
$sEmail = "some.email@something.com"
$sDomain = "www.w3schools.com"
$sPage = "/php/demo_form_validation_escapechar.php"

; Data to send
$sAdditionalData = "name=" & $sUserName & "&email=" & $sEmail

; Initialize and get session handle
$hOpen = _WinHttpOpen("Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.0")
ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $hOpen = ' & $hOpen & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console

; Get connection handle
$hConnect = _WinHttpConnect($hOpen, $sDomain)
ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $hConnect = ' & $hConnect & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console

; Make a request
$hRequest = _WinHttpOpenRequest($hConnect, "POST", $sPage)
ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $hRequest = ' & $hRequest & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console

; Send it. Specify additional data to send too. This is required by the Google API:
$hSendRequest = _WinHttpSendRequest($hRequest, "Content-Type: application/x-www-form-urlencoded", $sAdditionalData)
ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $hSendRequest = ' & $hSendRequest & @CRLF & '>Error code: ' & @error & @TAB & "@extended: " & @extended & @CRLF) ;### Debug Console

; Wait for the response
$hReceiveResponse = _WinHttpReceiveResponse($hRequest)

; See what's returned
Dim $sReturned
If _WinHttpQueryDataAvailable($hRequest) Then ; if there is data
    Do
        $sReturned &= _WinHttpReadData($hRequest)
    Until @error
EndIf

; Close handles
_WinHttpCloseHandle($hRequest)
_WinHttpCloseHandle($hConnect)
_WinHttpCloseHandle($hOpen)

; See what's returned
ConsoleWrite("Returned: " & $sReturned & @CRLF)

But sending a request fails.

Console outputs this:

>Running:(3.3.12.0) ...
--> Press Ctrl+Alt+F5 to Restart or Ctrl+Break to Stop
@@ Debug(14) : $hOpen = 0x00000000007DF540
>Error code: 0
@@ Debug(18) : $hConnect = 0x00000000008271C0
>Error code: 0
@@ Debug(22) : $hRequest = 0x00000000008B88E0
>Error code: 0
@@ Debug(26) : $hSendRequest = 0
>Error code: 1 @extended: 0
Returned:
+>17:00:18 AutoIt3.exe ended.rc:0
+>17:00:18 AutoIt3Wrapper Finished.
>Exit code: 0    Time: 0.6003

I'm testing it at work. There, we always have to identify at a proxy to be allowed to go to internet. But requests via inet.au3 will work. Maybe our firewall will block the request?

Any ideas? I don't have any possibilities to test at home, because of OSX.

 

Regards, Conrad

Edited May 9, 2016 by Simpel
formating code + [Solved] + Problem again

[trancexx]

If you need to identify proxy IE uses then you can use _WinHttpGetIEProxyConfigForCurrentUser() function.

;...
$aIEproxy = _WinHttpGetIEProxyConfigForCurrentUser()
$sProxy = $aIEproxy[2]

$hOpen = _WinHttpOpen("Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.0", $WINHTTP_ACCESS_TYPE_NAMED_PROXY, $sProxy)
;...

 

[Simpel]

_ArrayDisplay($aIEproxy)

says:

[0]         1
[1]
[2]
[3]

and:

ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $sProxy = ' & $sProxy & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console

says:

@@ Debug(15) : $sProxy = 
@@ Debug(21) : $hOpen = 0
>Error code: 0

$aIEproxy[0] = 1 means autodetect, correct? If I now try your modified "$hOpen" it's not working.

[trancexx]

AutoDetect 1 means IE doesn't use PAC.
If you get no-proxy, like you do then that's not the way to go.

How do yo identify yourself for proxy?

 

[Simpel]

ipconfig /all replies

WINS-Proxy activated - NO

If I use intranet I'm always fine. When I want to use internet connection I first have to type in username and password.

[trancexx]

Type in where?

Do you open some site in IE or what?

[Simpel]

Yes.

If I open a site in IE or FF. FF says something like: "Proxy moz-proxy://xxxprx01.xxx-xxx.de:8080 Needs username and password".

[Simpel]

$sProxy = "xxxprx01.xxx-xxx.de:8080"

Now I'm a bit forward. $hSendRequest includes now "credentials are missing - username and password". Fine.

How to put username and password into the proxy variable? "USERNAME:PASSWORD@xxxprx01.xxx-xxx.de:8080" is not working.

[trancexx]

My guess you are dealing with proxy that supports session based authentication. That means that for every session you have to authenticate.
If that's true you need to extend _WinHttpOpen() and add authentication routine. Like this maybe:

expandcollapse popup

#include "WinHttp.au3"

Global Const $sAddress = "http://xxxprx01.xxx-xxx.de:8080" ; address of your proxy that requires authorization

; Credentials
Global Const $sUserName = "user", $sPassword = "password"
; Open session (with credentials from above)
Global $hOpen = OpenProxySession($sAddress)


; Try some site now
$hConnect = _WinHttpConnect($hOpen, "http://google.com")
; Simple-request it...
MsgBox(64, "Returned (first 1400 characters of google.com)", StringLeft(_WinHttpSimpleRequest($hConnect), 1400) & "...")
; Close handle
_WinHttpCloseHandle($hConnect)


; Finally close session
_WinHttpCloseHandle($hOpen)



Func OpenProxySession($sAddress)
    ; Initialize and get session handle
    Local $hOpen = _WinHttpOpen()
    If $hOpen Then
        ; Get connection handle
        Local $hConnect = _WinHttpConnect($hOpen, $sAddress)

        If $hConnect Then
            ; Specify the reguest
            Local $hRequest = _WinHttpOpenRequest($hConnect)
            If $hRequest Then
                ; Send the request
                _WinHttpSendRequest($hRequest)

                ; Wait for the response
                _WinHttpReceiveResponse($hRequest)

                ; Query status code
                Local $iStatusCode = _WinHttpQueryHeaders($hRequest, $WINHTTP_QUERY_STATUS_CODE)
                ; Check status code
                If $iStatusCode = $HTTP_STATUS_DENIED Or $iStatusCode = $HTTP_STATUS_PROXY_AUTH_REQ Then
                    ; Query Authorization scheme
                    Local $iSupportedSchemes, $iFirstScheme, $iAuthTarget
                    If _WinHttpQueryAuthSchemes($hRequest, $iSupportedSchemes, $iFirstScheme, $iAuthTarget) Then
                        ; Set passed credentials
                        _WinHttpSetCredentials($hRequest, $iAuthTarget, $iFirstScheme, $sUserName, $sPassword)
                        ; Send request again now
                        _WinHttpSendRequest($hRequest)
                        ; And wait for the response again
                        _WinHttpReceiveResponse($hRequest)
                        ; Excellent. You should be in if credentials were ok!!!
                    EndIf
                EndIf
                _WinHttpCloseHandle($hRequest)
            EndIf
            _WinHttpCloseHandle($hConnect)
        EndIf
    EndIf
    ; Return session handle
    Return $hOpen
EndFunc

But that's really only my guess.

[trancexx]

Scratch that, this should work:

expandcollapse popup

#include "WinHttp.au3"

Global Const $sProxy = "http://xxxprx01.xxx-xxx.de:8080" ; address of your proxy that requires authorization
; Credentials
Global Const $sUserName = "username", $sPassword = "password" ; set correct 

; Open session
Global $hOpen = _WinHttpOpen("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.10240", $WINHTTP_ACCESS_TYPE_NAMED_PROXY, $sProxy)
;~ _WinHttpSetOption($hOpen, $WINHTTP_OPTION_REDIRECT_POLICY, $WINHTTP_OPTION_REDIRECT_POLICY_NEVER)

; Try some site now
$hConnect = _WinHttpConnect($hOpen, "http://www.w3.org/")

$hRequest = _WinHttpSimpleSendRequest_ProxyAuth($hConnect)
; Simple-read...
MsgBox(4096, "http://www.w3.org/ says", _WinHttpSimpleReadData($hRequest))


; Close handles
_WinHttpCloseHandle($hRequest)
_WinHttpCloseHandle($hConnect)
_WinHttpCloseHandle($hOpen)



Func _WinHttpSimpleSendRequest_ProxyAuth($hConnect, $sType = Default, $sPath = Default, $sReferrer = Default, $sDta = Default, $sHeader = Default)
    ; Specify the reguest
    Local $hRequest = _WinHttpSimpleSendRequest($hConnect, $sType, $sPath, $sReferrer, $sDta, $sHeader)
    If $hRequest Then
        ConsoleWrite(_WinHttpQueryHeaders($hRequest) & @CRLF) ; let's print what the proxy says
        ; Query status code
        Local $iStatusCode = _WinHttpQueryHeaders($hRequest, $WINHTTP_QUERY_STATUS_CODE)
        ; Check status code
        If $iStatusCode = $HTTP_STATUS_PROXY_AUTH_REQ Then
            ; Query Authorization scheme
            Local $iSupportedSchemes, $iFirstScheme, $iAuthTarget
            If _WinHttpQueryAuthSchemes($hRequest, $iSupportedSchemes, $iFirstScheme, $iAuthTarget) Then
                ; Set passed credentials
                _WinHttpSetCredentials($hRequest, $iAuthTarget, $iFirstScheme, $sUserName, $sPassword)
                ; Send request again now
                _WinHttpSendRequest($hRequest)
                ; And wait for the response again
                _WinHttpReceiveResponse($hRequest)
                ; Excellent. You should be in if credentials were ok!!!
            EndIf
        EndIf
    EndIf
    Return $hRequest
EndFunc

However I'm not sure what will happen if you "open" address that redirects to another address. Proxy I tested this with doesn't cache credentials, and therefore redirects fail. If that would happen to you then uncomment the line below _WinHttpOpen() and test it again.

[Simpel]

Thank you. It's working.

Sorry for delay. I had to test it under various conditions. And it took me a long time to understand how WinHttp.au3 and your snippet works.

Sometimes my code was running and then with little changes not.

Thanks and regards, Conrad

[Simpel]

Hi,

now it's broken. They changed something. This is your code modified by me:

expandcollapse popup

#include "WinHttp.au3"

Global Const $sProxy = "http://PROXY:8080" ; address of your proxy that requires authorization
; Credentials
Global Const $sUserName = "USER", $sPassword = "PASSWORD" ; set correct

; Open session
Global $hOpen = _WinHttpOpen("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.10240", $WINHTTP_ACCESS_TYPE_NAMED_PROXY, $sProxy)
;~ _WinHttpSetOption($hOpen, $WINHTTP_OPTION_REDIRECT_POLICY, $WINHTTP_OPTION_REDIRECT_POLICY_NEVER)

; Try some site now
$hConnect = _WinHttpConnect($hOpen, "http://posttestserver.com")

$hRequest = _WinHttpSimpleSendRequest_ProxyAuth($hConnect)
; Simple-read...
MsgBox(4096, "http://posttestserver.com/ says", _WinHttpSimpleReadData($hRequest))

$msg = _POST()
MsgBox(0, '', $msg)
ClipPut($msg)
Exit

; Close handles
_WinHttpCloseHandle($hRequest)
_WinHttpCloseHandle($hConnect)
_WinHttpCloseHandle($hOpen)

Func _POST()
   Local $sData = ("Something to test and post.")
   Local $vRequest = _WinHttpSimpleRequest($hConnect, "POST", "http://posttestserver.com/post.php", $WINHTTP_NO_REFERER, $sData, "Content-Type: application/x-www-form-urlencoded")
   If @error Then Return "Unable To Send Request"
   Return $vRequest
EndFunc

Func _WinHttpSimpleSendRequest_ProxyAuth($hConnect, $sType = Default, $sPath = Default, $sReferrer = Default, $sDta = Default, $sHeader = Default)
    ; Specify the reguest
    Local $hRequest = _WinHttpSimpleSendRequest($hConnect, $sType, $sPath, $sReferrer, $sDta, $sHeader)
    If $hRequest Then
;~         ConsoleWrite(_WinHttpQueryHeaders($hRequest) & @CRLF) ; let's print what the proxy says
        ; Query status code
        Local $iStatusCode = _WinHttpQueryHeaders($hRequest, $WINHTTP_QUERY_STATUS_CODE)
        ; Check status code
        If $iStatusCode = $HTTP_STATUS_PROXY_AUTH_REQ Then
            ; Query Authorization scheme
            Local $iSupportedSchemes, $iFirstScheme, $iAuthTarget
            If _WinHttpQueryAuthSchemes($hRequest, $iSupportedSchemes, $iFirstScheme, $iAuthTarget) Then
                ; Set passed credentials
                _WinHttpSetCredentials($hRequest, $iAuthTarget, $iFirstScheme, $sUserName, $sPassword)
                ; Send request again now
                _WinHttpSendRequest($hRequest)
                ; And wait for the response again
                _WinHttpReceiveResponse($hRequest)
                ; Excellent. You should be in if credentials were ok!!!
            EndIf
        EndIf
    EndIf
    Return $hRequest
EndFunc

The output my clipboard contains:

expandcollapse popup

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html>
<!-- FileName: index.html
     Language: [en]
-->
<!--Head-->
<head>
  <meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
  <meta http-equiv="X-UA-Compatible" content="IE=7" />
  <title>Proxy Message</title>
  <script src="/mwg-internal/de5fs23hu73ds/files/javascript/sw.js" type="text/javascript" ></script>
  <link rel="stylesheet" href="/mwg-internal/de5fs23hu73ds/files/default/stylesheet.css" />
</head>
<!--/Head-->
<!--Body-->
<body onload="swOnLoad();">
  <table class='bodyTable'>
    <tr>
      <td class='bodyData' background='/mwg-internal/de5fs23hu73ds/files/default/img/bg_body.gif'>
<!--Logo-->
<table class='logoTable'>
  <tr>
    <td class='logoData'>
      <a href='http://www.xxxxxxxxxxxxxxx.de'>
        <img src='/mwg-internal/de5fs23hu73ds/files/default/img/logo_mwg.png'></a>
    </td>
  </tr>
</table>
<!--/Logo-->
<!--Contents-->
<!-- FileName: authenticationrequired.html
     Language: [en]
-->
<!--Title-->
<table class='titleTable' background='/mwg-internal/de5fs23hu73ds/files/default/img/bg_navbar.jpg'>
  <tr>
    <td class='titleData'>
      Authentication Required
    </td>
  </tr>
</table>
<!--/Title-->

<!--Content-->
<table class="contentTable">
  <tr>
    <td class="contentData">
      You must be authenticated to access this URL.
    </td>
  </tr>
</table>
    
<script language="javascript" type="text/javascript">
   urlprotocol = "http";
   statuscode=407;

   if(statuscode==401 && urlprotocol == "ftp"){
      document.write("<form name=\"ftpform\" method=\"get\" action=\"\">");
      document.write("<table class=\"contentData\">");
      document.write("<tr><td class=\"contentData\" colspan=2>Please enter your credentials in the form below and click \"Access FTP\" button if your browser doesn't present authentication promt for FTP sites.</td></tr>");
      document.write("<tr><td class=\"contentData\">Username:</td><td><input type=\"text\" id=\"ftpUsername\" name=\"ftpUsername\" size=40 /></td></tr>");
      document.write("<tr><td class=\"contentData\">Password:</td><td><input type=\"password\" id=\"ftpPassword\" name=\"ftpPassword\" size=40 /></td></tr>");
      document.write("<tr><td class=\"contentData\" colspan=2 align=center><input type=\"button\" onclick=\"redirectToFTP();\" value=\"Access FTP\" /></td></tr>");
      document.write("</table>");
      document.write("</form>");
   }
   
   function redirectToFTP(){
      var username=escape(document.getElementById("ftpUsername").value);
      var password=escape(document.getElementById("ftpPassword").value);
      location.href = "ftp://"+username+":"+password+"@posttestserver.com:80/http://posttestserver.com/post.php"
   }
</script>
<!--/Content-->

<!--Info-->
<table class="infoTable">
  <tr>
    <td class="infoData">
      <b>URL: </b><script type="text/javascript">break_line("http://posttestserver.com/http://posttestserver.com/post.php");</script><br />
    </td>
  </tr>
</table>
<!--/Info-->

<!--/Contents-->
<!--Policy-->
<table class='policyTable'>
  <tr>
    <td class='policyHeading'>
      <hr>
      Company Acceptable Use Policy
    </td>
  </tr>
  <tr>
    <td class='policyData'>
      This is an optional acceptable use disclaimer that appears on every page. You may change the wording or remove this section entirely in index.html.
    </td>
  </tr>
</table>
<!--/Policy-->
<!--Foot-->
<table class='footTable'>
  <tr>
    <td class='helpDeskData' background='/mwg-internal/de5fs23hu73ds/files/default/img/bg_navbar.jpg'>
      For assistance, please contact your system administrator.
    </td>
  </tr>
  <tr>
    <td class='footData'>
      generated <span id="time">2016-05-09 21:46:25</span> by McAfee Web Gateway
      <br />
<br />
      Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.10240
    </td>
  </tr>
</table>
<!--/Foot-->
      </td>
    </tr>
  </table>
</body>
<!--/Body-->
</html>

_WinHttpSimpleReadData is working _WinHttpSimpleRequest not. What's wrong?

Regards, Conrad

Edited May 9, 2016 by Simpel