This topic is now closed to further replies.
I'm trying to get data from http://poe.trade/ - disclaimer, although this site is about a game, my script will not in any way interact directly with the game in any way. The script is just to get data from the site.
To explain how it works - you submit a POST request and a custom URL is returned, then you do a GET request on that URL and you get the final URL you want.
Now, I've tried doing so by using https://apitester.com/ and the first phase works. Here's how it looks like at APITester:
Request Headers POST /search HTTP/1.1 Host: poe.trade Accept: */* User-Agent: Rigor API Tester Content-Length: 43 Content-Type: application/x-www-form-urlencoded Request Body online=x&name=kaom%27s%20heart&league=incursion When I submit this, the response I get is this:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN"> <title>Redirecting...</title> <h1>Redirecting...</h1> <p>You should be redirected automatically to target URL: <a href="http://poe.trade/search/ioritewoteteme">http://poe.trade/search/ioritewoteteme</a>. If not click the link. So I then do a GET request for 'http://poe.trade/search/ioritewoteteme', which results in this response:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN"> <title>Redirecting...</title> <h1>Redirecting...</h1> <p>You should be redirected automatically to target URL: <a href="http://poe.trade/search/inamotezuakito">http://poe.trade/search/inamotezuakito</a>. If not click the link. Great, this link (http://poe.trade/search/inamotezuakito) is exactly what we want.
However, when I try to do the same in autoit, the result is quite different:
Global Const $HTTP_STATUS_OK = 200 $test = HttpPost("http://poe.trade/search", "/online=x&name=kaom%27s%20heart&league=incursion") ClipPut($test) MsgBox(1, "", $test) Func HttpPost($sURL, $sData = "") Local $oHTTP = ObjCreate("WinHttp.WinHttpRequest.5.1") $oHTTP.Open("POST", $sURL, False) If (@error) Then Return SetError(1, 0, 0) $oHTTP.SetRequestHeader("Host", "poe.trade") $oHTTP.SetRequestHeader("User-Agent", "Rigor API Tester") $oHTTP.SetRequestHeader("Accept", "*/*") $oHTTP.SetRequestHeader("Content-Type", "application/x-www-form-urlencoded") $oHTTP.Send($sData) If (@error) Then Return SetError(2, 0, 0) If ($oHTTP.Status <> $HTTP_STATUS_OK) Then Return SetError(3, 0, 0) Return SetError(0, 0, $oHTTP.ResponseText) EndFunc The code above returns: ' 謟 '
Any ideas as to what I am doing incorrectly?
Once I get the final link using APITester and do a GET on that i get a bunch of hieroglyphs. A friend of mine advised that the data is GZiped, which is a pain in the butt to be honest. However, apparently curl can uncompres that.
How would I go about it?
Some time ago, some members asked how to see the Http Requests.
There are quite a few external applications but not in AutoIt!
HttpHeaderWatcher in association with WinPcap, very modestly solves this lack.
Once done this http watcher, i asked me : why not Re-build a WinHttp Request in AutoIt from a selected Request in one Click ?
So i have added a "Create au3" button who open the WinHttp Request of your choice in AutoIt format in SciTE Editor.
Voila voila, hope it can help !
Buttons were made online with chimply.com the easy and free buttons generator !
See Help for more infos.
previous downloads : 253
source and executable are available in the Download Section
Hope you like it !
I'm trying to get an array from a website so that I can just get the url, but I am not sure how. I read a bit of arrays but I have a feeling I'd have to be writing a lot more than what I should be. I will include the script I have so far and the API url for what I want.
API: https://api.fast.com/netflix/speedtest?https=true&token=YXNkZmFzZGxmbnNkYWZoYXNkZmhrYWxm&urlCount=1 (I want the 'url' array that contains the url)
#include <MsgBoxConstants.au3> #include <Inet.au3> #include <Array.au3> $site = _INetGetSource('http://api.fast.com/netflix/speedtest?https=true&token=YXNkZmFzZGxmbnNkYWZoYXNkZmhrYWxm&urlCount=1') MsgBox($MB_SYSTEMMODAL, "Title", $site)
I've ported these two functions from PHP to AU3 to work with URLs.
Made them for those who work with libraries like HTTP.au3 (not the one I coded), that needs passing the server domain, path, etc., instead of the full URL.
Grab the lib here.
ParseURL( $sURL )
Parses the URL and splits it into defined parts. Returns an array:
 = Full URL (same as $sURL)  = Protocol (i.e.: http, https, ftp, ws...)  = Domain  = Port (or null if not specified)  = Path (or null if not specified)  = Query string (everything after the ? - or null if not specified) Example:
$aExample = ParseURL("https://google.com:8080/?name=doe") MsgBox(0, "Test", "URL: " & $aExample & @CRLF & _ "Protocol: " & $aExample & @CRLF & _ "Domain: " & $aExample & @CRLF & _ "Port: " & $aExample & @CRLF & _ "Path: " & $aExample & @CRLF & _ "Query string: " & $aExample)
ParseStr( $sStr )
Parses a query string (similar to the  of the previous function) and returns a multidimensional array, where:
 = number of variables found  = ununsed  = key name of the first variable  = first variable value (already URL decoded) [n] = key name of the nth variable [n] = nth variable value (already URL decoded) Example:
include <Array.au3> ; need only to do _ArrayDisplay, not needed by the lib _ArrayDisplay(ParseStr("foo=bar&test=lol%20123")) #cs Result is:  = 2  = ununsed  = foo  = bar  = test  = lol 123 #ce Feel free to fork!
I have a potential problem of memory leakage in the script I wrote and wonder if anyone will be able to suggest a potential source of the problem?
The script is desgined to log into secure website and constantly send HTTP GET request to obtain a list of tasks the client has assigned to us (the tasks are auctions, so swift actions are neccessary, hence the script) and then occasionally send HTTP POST to accept certain tasks based on certain criteria. It works 24/7 and works alright, but I noticed that amount of RAM it uses will increase significantly over time (say, it jumps from 20 000 K to 100 000 K in a few days).
I cannot post the entire script: it modestly big, also might contatin some sensetive information. But here's the main loop (like, exactly):
While True If Script_SignIn() And Script_GetTasks() Then Script_AcceptTasks() Sleep(10) WEnd The Script_ functions are here just to simplify the structure, they don't pass any parameters to each other, just return True \ False when finished and change some global variables when executed. Entire GUI is in OnEvent mode.
TMS_ functions are where the actual work is done, names are self-explanatory:
TMS_SignIn($sURL, $sUsername, $sPassword) -- returns $sAuthKeys (used subsequently in other functions to authenticate yourself, while session is active). TMS_GetTasks(ByRef $oHTTP, $sURL, $sAuthKeys) -- this function is running constantly returning $avTasks (basically a table with all information I need to accept them), I'm passing a global variable $g_oHTTP to it, it creates HTTP object only once per session (hence ByRef, I don't want to locally create HTTP object at every pass, just create once and reuse) and in next loop I pass the object the function created to it again, $g_oHTTP is destroyed when session has ended and recreated in next loop after re-login. TMS_AcceptTasks(ByRef $avTasks, $sURL, $sAuthKeys) -- returns True\False (changes $avTasks table slightly -- the status of acceptance if it's successful). TMS_SignOut($sURL, $sAuthKeys) -- returns True\False. These functions create local HTTP and HTML objects inside as well, I even "= 0" them at the end, even though I don't think I need to do this, since they are local variables. Still the problem persists.
Any ideas where to look?