Jump to content
Sign in to follow this  
Xenobiologist

InetGet

Recommended Posts

Xenobiologist

Hi,

I'm trying to use InetGet through our companies squid proxy server.

The function

HttpSetProxy(2, 'http://proxy.xxx.de:8080', @UserName, 'xxx')oÝ÷ ÚÇËhÂäN«z«²ÚÞiÈ^²Ø^¦º1ÊÇ«½êËk'«ÝÊ&éâq«b¢pn¶Ø^©e¶ëFzÚ+ ×­ëR¢êÜyû§rبƥÇhp¢¹ÊènW¦ËajÛaz·§¢·©¢Ë"n)b·'è®g­¡×º!jج§
è
C¡ë-çîËb¢{¬zz+©GM3ö¬Û¬zØ^{^®w­{e¢·«­ë(º·³b,Úh²ÈëhëmáÓLúÞ¶êçr^&"w­ëS©ä²Ú&èw'讥¦)න¦V¢Z´ÞyÛhz0!©l¢ÚâyÛ(ëax%G­+ºÚ"µÍ[ÈÜÙ][ÞJ ÌÍÜÞK   ÌÍÜÜ
B[Ù]
    ÌÎNÚÜÞIÌÎNË ÌÍÜÞH   [È ÌÎNÎÌÎNÈ  [È ÌÍÜÜ
B[]J
B[[ÈÏOIÝ×ÜÙ][Þ

with the same result.

Mega

P.s.: Till now, I'm using perl or Autoit in combination with wget.exe to download files through proxy (with authentication), but I still would like to switch to pure Autoit.

@Devs/Mods : No problem moving this to support, when you give me a little answer, too. :)


Scripts & functions Organize Includes Let Scite organize the include files

Yahtzee The game "Yahtzee" (Kniffel, DiceLion)

LoginWrapper Secure scripts by adding a query (authentication)

_RunOnlyOnThis UDF Make sure that a script can only be executed on ... (Windows / HD / ...)

Internet-Café Server/Client Application Open CD, Start Browser, Lock remote client, etc.

MultipleFuncsWithOneHotkey Start different funcs by hitting one hotkey different times

Share this post


Link to post
Share on other sites
Xenobiologist

Any problems understanding what I want to do?

Can you send me the InetGet function, so that I can have a look myself?

Sorry for being that impatient.

Mega

Edited by Xenobiologist

Scripts & functions Organize Includes Let Scite organize the include files

Yahtzee The game "Yahtzee" (Kniffel, DiceLion)

LoginWrapper Secure scripts by adding a query (authentication)

_RunOnlyOnThis UDF Make sure that a script can only be executed on ... (Windows / HD / ...)

Internet-Café Server/Client Application Open CD, Start Browser, Lock remote client, etc.

MultipleFuncsWithOneHotkey Start different funcs by hitting one hotkey different times

Share this post


Link to post
Share on other sites
ProgAndy

I think, you could use the WinHTTP udfs and setcredentials (included in the udfs). Then you can fetch all information you want.

//Edit: to set the proxy, you have to use WinHTTPSetOption: http://msdn.microsoft.com/en-us/library/aa383912(VS.85).aspx

Edited by ProgAndy

*GERMAN* [note: you are not allowed to remove author / modified info from my UDFs]My UDFs:[_SetImageBinaryToCtrl] [_TaskDialog] [AutoItObject] [Animated GIF (GDI+)] [ClipPut for Image] [FreeImage] [GDI32 UDFs] [GDIPlus Progressbar] [Hotkey-Selector] [Multiline Inputbox] [MySQL without ODBC] [RichEdit UDFs] [SpeechAPI Example] [WinHTTP]UDFs included in AutoIt: FTP_Ex (as FTPEx), _WinAPI_SetLayeredWindowAttributes

Share this post


Link to post
Share on other sites
Valik

It uses the WinInet functions.

Make sure the username and password are correct. Specifically I'm talking about case sensitivity. Since you are using an AutoIt macro you may be passing "Xeno" instead of "xeno" which would obviously fail if the proxy is case sensitive. Try setting up the proxy in IE and make sure that IE works. Then use HTTPSetProxy(0) to use IE's settings and try InetGet() again.

Share this post


Link to post
Share on other sites
Xenobiologist

Thanks to both of you.

I'm one step further, but I've still not reached the goal. :) All I want to test, is a simple download script for files (binary) with pure Autoit through a squid proxy which needs username and password.

I tested this: (Setting the proxy information in the IE settings) Of course without user and password, cause it is not possible. Then ...

HTTPSetProxy(0)
Global $s_URL = 'myUsername:myPassword@www.autoit.de'
Global $source = _INetGetSource ( $s_URL )
ConsoleWrite($source & @CRLF)oÝ÷ Ù8Z·
+çwÜ­ºÈ§Þ´g­×hzÉ÷öÐ'thpj{m¢Ç­¶©®rwè®f­âÁ©í¶¬zØ^wè®f­ìzا+®*m|¢u欶^²Ü!jج§
èáÇhzÉ÷öÜ(®I覺1ÊÇ­rjëh×6HttpSetProxy(2, 'http://proxy.xxx.de:8080', 'xxx', 'xxx')
Global $s_URL = 'http://www.autoit.de'
Global $source = _INetGetSource ( $s_URL )
ConsoleWrite($source & @CRLF)

Same code with proxy set in IE leads to <TITLE>FEHLER: Cache-Zugriff Verweigert</TITLE>

which means :ERROR cache access denied!!!

@ProgAndy : How to use your nice UDF to get it done? I need to set proxy with credentials and the url.

Mega

P.S: Proxy is not case sensitive.


Scripts & functions Organize Includes Let Scite organize the include files

Yahtzee The game "Yahtzee" (Kniffel, DiceLion)

LoginWrapper Secure scripts by adding a query (authentication)

_RunOnlyOnThis UDF Make sure that a script can only be executed on ... (Windows / HD / ...)

Internet-Café Server/Client Application Open CD, Start Browser, Lock remote client, etc.

MultipleFuncsWithOneHotkey Start different funcs by hitting one hotkey different times

Share this post


Link to post
Share on other sites
weaponx

I'm wondering if your proxy is looking at the user agent and blocking it. Take a look at this topic about using winhttprequest and changing the user agent:

http://www.autoitscript.com/forum/index.ph...=winhttprequest

You will also need to look into the setcredentials function on MSDN:

http://msdn.microsoft.com/en-us/library/aa384106(VS.85).aspx

Edited by weaponx

Share this post


Link to post
Share on other sites
ProgAndy

I think, you could use the WinHTTP udfs and setcredentials (included in the udfs). Then you can fetch all information you want.

//Edit: to set the proxy, you have to use WinHTTPSetOption: http://msdn.microsoft.com/en-us/library/aa383912(VS.85).aspx

@ProgAndy : How to use your nice UDF to get it done? I need to set proxy with credentials and the url.

I think, you have to insert the credential-thing between those lines: (from example in WinHTTP thread)

$h_openRequest = _WinHttpOpenRequest($hw_connect, "GET", "en-us/library/aa384101(VS.85).aspx", "HTTP/1.1")

_WinHttpSendRequest($h_openRequest)

_WinHttpSetCredentials($h_openRequest,$WINHTTP_AUTH_TARGET_PROXY, $WINHTTP_AUTH_SCHEME_BASIC, "USER", "PASS")

//Edit: @weaponx: Xeno said, _InetGetSource works. This func uses "AutoIt v3" as user agent. So this shouldN#t be a problem.

Edited by ProgAndy

*GERMAN* [note: you are not allowed to remove author / modified info from my UDFs]My UDFs:[_SetImageBinaryToCtrl] [_TaskDialog] [AutoItObject] [Animated GIF (GDI+)] [ClipPut for Image] [FreeImage] [GDI32 UDFs] [GDIPlus Progressbar] [Hotkey-Selector] [Multiline Inputbox] [MySQL without ODBC] [RichEdit UDFs] [SpeechAPI Example] [WinHTTP]UDFs included in AutoIt: FTP_Ex (as FTPEx), _WinAPI_SetLayeredWindowAttributes

Share this post


Link to post
Share on other sites
Xenobiologist

Thanks again!

I tried my luck with the WinHTTP.au3 and I was able to get something to work.

#include <WinHTTP.au3>

Global $hw_open = _WinHttpOpen("WinHTTP Example", $WINHTTP_ACCESS_TYPE_NAMED_PROXY, 'proxy.xxx.de:8080')
;~ $hw_connect = _WinHttpConnect($hw_open, "msdn.microsoft.com")
Global $hw_connect = _WinHttpConnect($hw_open, "autoit.de")
If @error = 1 Then ConsoleWrite(1 & @CRLF)
;~ $h_openRequest = _WinHttpOpenRequest($hw_connect, "GET", "en-us/library/aa384101(VS.85).aspx", "HTTP/1.1")
Global $h_openRequest = _WinHttpOpenRequest($hw_connect, "GET", "index.php?page=MembersList", "HTTP/1.1")
If @error = 1 Then ConsoleWrite(2 & @CRLF)
_WinHttpSetCredentials($h_openRequest, $WINHTTP_AUTH_TARGET_PROXY, $WINHTTP_AUTH_SCHEME_BASIC, "xf01145", "sony1234")
If @error = 1 Then ConsoleWrite(3 & @CRLF)
_WinHttpSendRequest($h_openRequest)
_WinHttpReceiveResponse($h_openRequest)

If _WinHttpQueryDataAvailable($h_openRequest) Then
;~     MsgBox(0, "Header", _WinHttpQueryHeaders($h_openRequest))
    ConsoleWrite(_WinHttpQueryHeaders($h_openRequest) & @CRLF)
    ConsoleWrite('! --- ' & @CRLF)
;~     MsgBox(0, 'Text', _WinHttpReadData($h_openRequest, 1, 1000000))    
    ConsoleWrite(_WinHttpReadData($h_openRequest, 1, 1000000) & @CRLF)
EndIf

_WinHttpCloseHandle($h_openRequest)
_WinHttpCloseHandle($hw_connect)
_WinHttpCloseHandle($hw_open)

I received data from both functions.

HEADER

HTTP/1.0 200 OK
Date: Wed, 11 Feb 2009 21:37:51 GMT
Server: Apache/2.2.4 (Linux/SUSE)
X-Powered-By: PHP/5.2.6
Set-Cookie: wcf_cookieHash=2116a2c50364a761116702bd287d5d4d50547c27; HttpOnly
Set-Cookie: wcf_boardLastActivityTime=1234388271; expires=Thu, 11-Feb-2010 21:37:51 GMT; HttpOnly
Content-Type: text/html; charset=UTF-8
X-Cache: MISS from wspprx03.xxx.de
X-Cache: MISS from wspprx07.xxx.de
Via: 1.0 wspprx07.xxx.de:8081 (squid/2.6.STABLE6)
Proxy-Connection: close

Question : How to use it with binary files? Using the func with a download URL and WinHttpReadData($h_openRequest, 2, 10 * 1024 * 1024) didn't work.

And one question back to topic. Does HttpSetProxy really work? Why does it not work like I used it in post one?

Mega


Scripts & functions Organize Includes Let Scite organize the include files

Yahtzee The game "Yahtzee" (Kniffel, DiceLion)

LoginWrapper Secure scripts by adding a query (authentication)

_RunOnlyOnThis UDF Make sure that a script can only be executed on ... (Windows / HD / ...)

Internet-Café Server/Client Application Open CD, Start Browser, Lock remote client, etc.

MultipleFuncsWithOneHotkey Start different funcs by hitting one hotkey different times

Share this post


Link to post
Share on other sites
LarryDalooza

Question : How to use it with binary files?

I think this has been explored before and runs into a limitation of AutoIt COM and ByRef parameters... but that is just off the top of my head. You can try searching the forums for more.

Lar.


f_mrcleansmalm_77ce002.jpgAutoIt has helped make me wealthy

Share this post


Link to post
Share on other sites
Xenobiologist

Hi Larry,

thanks. But using the WinHTTP.au3 was just a try to get it done. Before I start to use COM in Autoit to download stuff through proxy I will use Perl or Java, cause that way I know what happens. :)

But I have some other stuff around that piece of code, which is much easier to achieve with Autoit and I do not want to combine the languages.

The initial question was, I want to set a proxy with username and password (with Autoit) and then I want to use the "normal" Autoit functions like InetGet und _INetGetSource. Till now I wasn't able to do so.

So, can you help me to investigate why it doesn't work that way?

From my point of view the HttpSetProxy and InetGet functions do not work together.

That is why I asked in developer forum. I do not need any solution for a download through proxy script. I want to use InetGet through proxy, cause there is a SetProxy function available in Autoit.

Mega


Scripts & functions Organize Includes Let Scite organize the include files

Yahtzee The game "Yahtzee" (Kniffel, DiceLion)

LoginWrapper Secure scripts by adding a query (authentication)

_RunOnlyOnThis UDF Make sure that a script can only be executed on ... (Windows / HD / ...)

Internet-Café Server/Client Application Open CD, Start Browser, Lock remote client, etc.

MultipleFuncsWithOneHotkey Start different funcs by hitting one hotkey different times

Share this post


Link to post
Share on other sites
weaponx

The code I posted above shows how to get binary data.

#551920

Share this post


Link to post
Share on other sites
Xenobiologist

The code I posted above shows how to get binary data.

#551920

Hi,

yes thanks. I will have a look whether I can use your code with proxy configuration.

Anyhow, this was not what I wanted to get by opening this thread.

Mega


Scripts & functions Organize Includes Let Scite organize the include files

Yahtzee The game "Yahtzee" (Kniffel, DiceLion)

LoginWrapper Secure scripts by adding a query (authentication)

_RunOnlyOnThis UDF Make sure that a script can only be executed on ... (Windows / HD / ...)

Internet-Café Server/Client Application Open CD, Start Browser, Lock remote client, etc.

MultipleFuncsWithOneHotkey Start different funcs by hitting one hotkey different times

Share this post


Link to post
Share on other sites
ProgAndy

To download binary data with WinHTTP, you should use a loop and mode 2:

;~     ConsoleWrite(_WinHttpReadData($h_openRequest, 1, 1000000) & @CRLF)
    Local $data, $hFile = FileOpen(@DesktopDir & "\ReadData.bin",18) ;binary write mode
    While 1
        $data = _WinHttpReadData($h_openRequest, 2) ; read some bytes
        If @error <> 0 Then ExitLoop ; error -1 for EOF or > 0 for other error then ExitLoop
        FileWrite($hFile,$data) ; Write bytes to file
    WEnd
    FileClose($hFile) ;close file

//Edit: _InetGetSource uses the functions from WinInet, but not in UDF-format, just the DLLCalls directly.

WinHTTP is just a simpler Interface for HTTP than WinInet and is provided from Windows, too.

Edited by ProgAndy

*GERMAN* [note: you are not allowed to remove author / modified info from my UDFs]My UDFs:[_SetImageBinaryToCtrl] [_TaskDialog] [AutoItObject] [Animated GIF (GDI+)] [ClipPut for Image] [FreeImage] [GDI32 UDFs] [GDIPlus Progressbar] [Hotkey-Selector] [Multiline Inputbox] [MySQL without ODBC] [RichEdit UDFs] [SpeechAPI Example] [WinHTTP]UDFs included in AutoIt: FTP_Ex (as FTPEx), _WinAPI_SetLayeredWindowAttributes

Share this post


Link to post
Share on other sites
Xenobiologist

Hi ProgAndy,

danke! I've tested your piece of code and it worked perfectly. So, now I'm able to use the WinHTTP.au3 through proxy to download even binary files.

I need to read up the differences between your Http wrapper and the WinInet Wrapper to microsoft API. Maybe this leads me to the solution why HttpSetProxy and InetGet don't work together.

If I understand you correctly, then both "functions" are wrappers to enable the functionality to access internet resources via different kinds of protocols like HTTP.

http://msdn.microsoft.com/en-us/library/aa383630(VS.85).aspx

http://msdn.microsoft.com/en-us/library/aa382925.aspx

WinHTTP is explained to be the better, newer way. Maybe, the Autoit functions should be migrated to WinHTTP.

Thanks again, I learnt a lot!

Mega


Scripts & functions Organize Includes Let Scite organize the include files

Yahtzee The game "Yahtzee" (Kniffel, DiceLion)

LoginWrapper Secure scripts by adding a query (authentication)

_RunOnlyOnThis UDF Make sure that a script can only be executed on ... (Windows / HD / ...)

Internet-Café Server/Client Application Open CD, Start Browser, Lock remote client, etc.

MultipleFuncsWithOneHotkey Start different funcs by hitting one hotkey different times

Share this post


Link to post
Share on other sites
Valik

This page pretty much sums up why we are using the right thing in AutoIt. In short, we are a client so we should use WinInet. In addition WinInet is available on all systems AutoIt runs on. WinHTTP requires specific minimum service pack versions for Windows 2000 and Windows XP.

Create a new bug report about the proxy issue you are having so I don't forget. I'll see if I can take a look at it sometime in the future.

Share this post


Link to post
Share on other sites
Valik

Mega, is your proxy server HTTP? HttpSetProxy() works *only* with HTTP proxies. With Internet Explore it's possible to get it to use a SOCKS proxy so maybe that's why your proxy works in IE but not in AutoIt?

I'm not sure this is a bug in AutoIt. It's a clear limitation in the WinInet API.

Share this post


Link to post
Share on other sites
Xenobiologist

We are using this one : Wiki EN Squid

Official homepage : Squid Homepage

I'm using this Perl code without any problems.

#!/usr/bin/perl
use warnings;
use strict;
use diagnostics;

use LWP::Simple;
use LWP::UserAgent;

# Variablendeklaration
my $URL                 = "http://www.url.de";
my $fileDestinationPath = "c:\\Temp\\test.txt";
my $user                = "user";
my $password            = "passwort";
my $proxy               = "proxy.xxx.de";
my $port                = "8080";

# Proxy
my $proxyString =
  "http://" . $user . ":" . $password . "\@" . $proxy . ":" . $port;
my $ua = LWP::UserAgent->new();
$ua->proxy( [ 'http', 'ftp' ], $proxyString );

# Datei download
print _getFile( $URL, $fileDestinationPath );

sub _getFile {
    my $url      = shift;
    my $filePath = shift;
    my $request  = HTTP::Request->new( 'GET', $url );
    my $response = $ua->request($request);
    if ( $response->is_success() ) {
        my $content = $response->content();

        # print $content;                    # Test print in die Console
        open( my $DATEI, " > ", $filePath );    # FileHandle zum Schreiben
        binmode($DATEI);                        # Binary Mode aktivieren
        print $DATEI "$content ";               # Datei schreiben
        close($DATEI);                          # FileHandle schliessen
    }
    else {
        return (
            -1, " Error-Code : ",
            $response->code(), " Fehlermeldung : ",
            $response->message()
        );
    }
}

I guess there must be a way to get it done with Windows/Autoit stuff. There are so many tools which do have no problems to go through the proxy when given the credentials like Pidgin, Teamviewer and so on.

Any ideas what I can test or what data I should collect to help you?

Mega


Scripts & functions Organize Includes Let Scite organize the include files

Yahtzee The game "Yahtzee" (Kniffel, DiceLion)

LoginWrapper Secure scripts by adding a query (authentication)

_RunOnlyOnThis UDF Make sure that a script can only be executed on ... (Windows / HD / ...)

Internet-Café Server/Client Application Open CD, Start Browser, Lock remote client, etc.

MultipleFuncsWithOneHotkey Start different funcs by hitting one hotkey different times

Share this post


Link to post
Share on other sites
Valik

Knowing that it's Squid is enough. I should be able to set it up sometime and test. Or read the documentation and see if I can spot any reason it might not work.

Share this post


Link to post
Share on other sites
Xenobiologist

OKay fine. If you need something I can try, please let me know.


Scripts & functions Organize Includes Let Scite organize the include files

Yahtzee The game "Yahtzee" (Kniffel, DiceLion)

LoginWrapper Secure scripts by adding a query (authentication)

_RunOnlyOnThis UDF Make sure that a script can only be executed on ... (Windows / HD / ...)

Internet-Café Server/Client Application Open CD, Start Browser, Lock remote client, etc.

MultipleFuncsWithOneHotkey Start different funcs by hitting one hotkey different times

Share this post


Link to post
Share on other sites
Valik

Wow. First, the username and password feature has been broken since day one. Second, how the hell does Microsoft expect those to be set...?

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×