Using WinHTTP to POST and Receive File

[big_daddy]

I was working on a solution for adamchapman on his topic here. I was able to make the option change quite easily, but downloading the file is a different story.

AutoIt Code to change select option

#include <IE.au3>

$sSymbol = "AA"
$sURL = "http://www.nasdaq.com/symbol/" & $sSymbol & "/historical"
$sFormId = "getFile"
$sSelectId = "ddlTimeFrame"
$sOptionValue = "5d"
$sInputId = "quotes_content_left_submitString"
$sLinkText = "Download this file in Excel Format"
$bDownload = False

_IEErrorHandlerRegister()
$oIE = _IECreate($sURL)
$oSelect = _IEGetObjById($oIE, $sSelectId)
_IEFormElementOptionSelect($oSelect, $sOptionValue)
If $bDownload Then _IELinkClickByText($oIE, $sLinkText)

I reviewed the function tied to the change event. This same function is called when you click the download link. The only difference is the argument is false and true respectively. I removed the code that is executed when false as it is not relevant for the issue. It basically builds a string with the selected option value + true + symbol (Example: 5d|true|AA), then sets the value of the input that has an id ending in submitString and submits the form.

function getQuotes(download) 
        {
            if(!download)
                showLoadingSpinner();
            
            var data = $("[id$='ddlTimeFrame']").val();
            var submitString = data + '|' + download + "|" + quoteBoxSelectedSymbol;
            
            if (!download) {
                <!-- Removed -->
            }
            else {
                $("[id$='submitString']").val(submitString);
                $("#getFile").submit();
            }
            
        }

-

---

Now we move on to the form that is submitted by the getQuotes function. The input with id "quotes_content_left_submitString" is the input that will have the value set from the previous function. 

<form id="getFile" method="post" action="http://www.nasdaq.com/symbol/aa/historical" lpsubdone="1">
    <div class="aspNetHidden">
        <input id="__VIEWSTATE" name="__VIEWSTATE" value="TRIMMED" type="hidden">
    </div>

    <div class="aspNetHidden">
        <input id="__VIEWSTATEENCRYPTED" name="__VIEWSTATEENCRYPTED" value="" type="hidden">
        <input id="__EVENTVALIDATION" name="__EVENTVALIDATION" value="FslL5mK8EfHMdRmVPirjPRHx0rzoR4ZblfHZcyVf/H6sJ58PbPdR2nXwJige52KEPcse4Wp37klD+PDrLXkhysNoAbltd+CEe5ntj67kFmk=" type="hidden">
    </div>
    <input id="quotes_content_left_submitString" name="ctl00$quotes_content_left$submitString" value="" type="hidden">
</form>

-

---

I believe this can be easily solved using WinHTTP, however I don't have a good understanding of that library currently. I did use Fiddler to capture the request and response that are generated when downloading the file.

Request

POST /symbol/aa/historical HTTP/1.1
Accept: text/html, application/xhtml+xml, */*
X-SFS-Content-Type: text/html; charset=utf-8
X-SFS-Top: http://www.nasdaq.com/symbol/aa/historical
Referer: http://www.nasdaq.com/symbol/aa/historical
Accept-Language: en-US
User-Agent: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)
Content-Type: application/x-www-form-urlencoded
Accept-Encoding: gzip, deflate
Host: www.nasdaq.com
Content-Length: 5687
Connection: Keep-Alive
Pragma: no-cache
Cookie: c_enabled$=true; clientPrefs=||||black; asi_segs=C10001|C10003|C10023|C0; s_pers=%20bc%3D1%7C1372648582375%3B%20s_nr%3D1372562217412-New%7C1380338217412%3B; userSymbolList=AA+; userCookiePref=true; s_sess=%20s_cc%3Dtrue%3B%20s_sq%3Dnasdaqprod%253D%252526pid%25253D%25252528AA%25252529%25252520Historical%25252520Prices%25252520%25252526%25252520Data%25252520-%25252520NASDAQ.com%252526pidt%25253D1%252526oid%25253Djavascript%2525253AgetQuotes%25252528true%25252529%2525253B%252526ot%25253DA%252526oi%25253D898%3B; NSC_W.TJUFEFGFOEFS.OBTEBR.80=ffffffffc3a08e3745525d5f4f58455e445a4a423660; __atuvc=5%7C26

__VIEWSTATE=vrdXsfBP2CJNWK4qdpzeDQ%2B85IYYgny7JkJYNc%2F3LaENWzOHiJW05WwMwTsOS7lz3jxV2h%2FQWE0JiUa2EbHDcTMVVTPixtTnlCXdTmZJglCj8ENYrGrMRlQJ3ZiX7Bp2%2Fgos2pw6fVLkRKq5QwDuqfzPd9N7T%2FqqIkg0pMJWXi6XFLwU8hgmOMiK7TCpvWg0tlE9Qk0Syk26AlbkmAcTb4Js%2FcE2G1L0j%2FrnbficD2aJCYrLVtUp5zkns7QztWfB%2BA87i3XcO84%2BZBhYeYU%2FABLX1Cht9Puez7nCPMmDlr5h4byKeUIP6RvTSYdKMwD0dFXXFGHoGKF7cFZ2Yt%2B96rUU%2FJ23Pt9wJ8m9rgDH8ia0g7y%2BsLfe19XxRCMOOrg7XgZg9NSgisVGibrudXsSirZP1T0uWmORWXzobB%2BNC6wLzcKE8hDJY0dn9zfCa6RzOcymiDRwt04sOVA9FWdITaps%2BoIBQwg%2BmblNbTvvSo5%2BNaUOVTaHqGcxyziPePDUcq6M2UiZSiirom%2FmVQ97BJWpOAzm%2BIIZO%2Fm7zHepRy0%2FB%2B22hw0IjBfSXGoorb2QrMsynDpvShzX%2FK2z60T1moBAPpOku90hCLf8wzB7%2BKmgaj3GGRrOswdH%2BJC%2FqUz2NDHy9TZ%2FlHVmrpFkEWgB6BPs%2FagzPb5OGDFL4maHjVTAAJjujDI19rndzkybn1parquZWLenEISx3iat3j2%2Bn%2FT17VTOkaz4beZ5KmkUaAPXJNssrNzlw7k3ZGTSFBxCwfITiBUpIXstDyuA7L3EbUyFBLAuBUHZnB%2BZXw1ja7EV1lWBdsBXXCT4kmRejADXAvlaitC1XaEvhE84M6R4PrbKRcCJYlyY8oxlxTg6ltP6SzumImnBH4P8IZ04IkjwKV9AlIgZI%2FULYauj77tH63A0NJhdEPNMMdbf71fw6KGDIc0v%2BQSYVuLcMaWmo8f8abNKEm%2BEVc5VziaM%2BXxrjhKwUD%2Bcc4qB8K3zTVu4UoUa2u1WLMFtv3rtN63%2B6h3K6vPNv5OjklHb%2FiCV0ECj1GNU0qXqZEIEforsUb0x39fL81NcXA2IvKO6VIYMx9DvxAnmmNG0%2Bgt4uOlTLMRDWFsJ8dzqc24Eo2NqIGHmmbxHed588d5x0uzG5xBQw41UEZzNsFatcmujUEMYhd7QnkUTsFqpd5RKWUpRDr31yGrgFaalqU2n2G%2Bj8f7eufZOboV4MWzJC4eSVnBN%2B3%2FcyMJtmmes47U1T4jp%2BFCdGvtpz9hcRXiuPuahgBK1LzfFOnZ7e9zfmQ5gSjB0yOv3cnNYXF5dCcsA0OYEfeLKfWraUYY2YYn40vnmXWqEDTCmyCis7a25mL26XjL3XQzA7lUwH6SiKYUVRICRdRHWXSTosz4sc67BgnXybIXbNZPL6tWr6neFMBNtERm%2BO%2FEzY393JZDaFJCDFeW0S9jnLB9%2FilRFswwqWNXnrEB%2FZMfnKRGFMHhuSGqqkg3UpUQqGTrL%2BgU19Aq3NocToPACh%2FP3NJ8r3loorHAEWwuLBkvPxhWEco%2FtNDv%2FXRLwYGdvJidL%2F9xpkLp4yq01uu%2FC%2FCAy9vyvHGLds6WR%2Bme97aNg15oBONzt7XTnhVSRUP8q5KQf8Xd2wKz%2BON2mtletsoAA6pziGA8NO98FX%2FAMUJzzNNEeYvyQ2%2FzzN0%2BMI%2Bi1U1W7DuH6jUgVATK6aOM%2FRG0xyu4lM4hvU6DnyFQBjpa5c8WJP3zjKd4men9C7bkgMI4Q96ocuRNsDrz0nr3ArZLnliSujnSfCNUkMZcSUysyu%2FDFSdAijSyfjcfjhLxnZnlCiOE3tqQmtyXWmPolv2r3MsSQVyCROz2tGFwEin1BM2jtLHI5U3GbQzHmyJn%2F7ig7P2tTeP5LcegaxcIvE%2F9HVOoi6BLrkOJgjdhg6PzR9oW8Q%2BKen0qqJPre7qEr49TrLP38%2BFm56rNQW1h5aNWwP7tJY3QgrBucVwYiCCezUGOspvF3vc%2Fmt%2BoSqJiFkp2DHTe%2FXWezfpEoC1a5C3PvAe83o7OAm4ahSqMwojrVkMTx18TEKpZ6cZFoMA6zQ0PM7ar%2FAZaWL9dm5E%2Bk94%2BTxeWbP2ggMeot9%2BksNCY4X6EcGcAqWKPWhiYVQq5ER0X1tyGJ2Gkru1ElwMrwiSSSlYZuNpeKovVRap0Rk2YMkC4gMh8Oi4nf05%2F0LfunTeGklLMj%2FYSoAu5gcl8eaf69iLaAhT3JZyzoJAaGvieLjApHpCnmJMo1ddNx%2B8hw3AHlClnF2xsPhp7W6Ofc0TdsWTaaRPUSyhEDs2u7%2FCsj%2B%2BwvgJeDCG3yFc4QeCNwqdCwka4MUrM0psftEzfCbUgHxfAbGrC8Im322bQrrEdRRajJ%2BpbaFIgMoaru7p0Hr%2BTpISlPCnKFK796%2FWlElX4P2y93WJjuQHaAeHV9PsVAybrQ8v8ugqzLgtqbMCH0C4v%2BD7inF2KV%2BB7fEd3e3%2BeyH9rhgDPZCYwe421n0vMj7NpB%2FZkw9ir7f0IFmVy8kve07KPSpnsrv%2BscCChf%2BB24m3cN8C68TbYIX0s%2B%2BHJenOu0W5oQAB%2BNYBURbIGLVjGN2qTZmJGEbRIf5LLoQZKDexQyJ3rNhIQ6tM61Xm6EcKbe0m%2BmHzygFgnu9N5l0FdtzO3toPTDshezr6EtwEe1PL624trskl7Ygqf%2FkiXuYaKaP5hefuiEyrFj7OfxAm9VG9i1Mf45vEJcX%2F2QK336kuIZf%2BKGuvS441iJbESGO9li2onGOwKtnJIRor3SHUEb1tr4PvM19YwYE%2BGYg9QXXrGfnznLgSSLoSfo4%2FTIO3%2BKOSpt8XYye%2BTAeSFQYS6SqbStcW%2FHBC%2BjWIN2KiQyHL8DfuuvOX3wZfq3mPYzkFML%2B3j4t%2FsbSw7ZdSvt3qX5r%2B8CDaSNGravczGWoZzuunevUu7Ch%2BKcaQDiDNpYQRzhc7Dtoj5nJw0DU4VhgsScpmgqG8rx4qklzzG8E3StC8s5mO7mAnYSfs3NsptLI0hbfgeA%2BVW31DTDw1gXolvpaPbYLu3tDI0W1MeXcS%2Ft8pCqtpwnIZ1LuhiPAjF%2Fhh9TdTePF1Hi%2BNWouAkM5Rvm1cZH3W%2B9rTNf1HRDwJrnzGI6kkZ5IjEqG5cUfZ5UHpBBJpDgmfCTDHpiNs88SHP4FVya6ymQ1oj2jkcS6E1rlhVmB4wGM9nt4m8wiGsHHAL6Yn649JO1cCwfiDv6YwHFK10VJ9WBH6DeTw4lrYkYhnnWqa4%2BGzSVkgVsUO2CWPdozAvT2eWtnS6BvojRfkGne%2BRi4WAPf3GtgXYZaRoXhZwGaSnCbR5YlQDBqv8xra4gykNb%2B3e1Nx5MbH8%2F8CNPUIJpN4BfaaO2YEst1zVrwtc6vdoWo7BRRQvZ6FebLakIlPnu3AP3pLgNSi5ifBmd%2FiBpVjFO6denlqsJSdLYAmYjCnNR2uLbgkA0INn6xHOtb2EMAIWRlToNNQthW1I54R3WbVO3NzL%2BWsqyWN6zXNjs67hg4Unzlh%2FGaANy2CwmcSNuRlF17778jTFyTX7kxGQekWBriiVxZ1z3VZqr5OB7n9NEAetIf7I8rC8h85xCJ5Tv9jnqN2ZHrMlidfxbF%2F6y9MSUlRjyylSxsyDtODbz84g2yQA3gumracHA5jqVokQTXuU232mgGOQ%2BX96NosZqcUbHR%2BVer1q%2BkfMnRfDK8uUoUMhqWblUtUByp%2F%2FUk7xTqIdbyt81%2F3CpT0HypN3A1W9vMRJJoqDUoZgQnMEP9fiAy3GSdT71m7Em82DaqSYZXAT71dMpWYFpBe5Ayc5FpCgdxFyF4GlqZdZ2Rg%2FQCMF5V0wgUO40wj0g3nQOLI0kwyLM18tbeDZnFA20uYMfwxaDPfFq1kH6jss93FG6pzkOBnlYv6j3xdUXm7L7IgT2FOR0RlwCWXAHB7o4EVfzxxrrKFrd16QHR1SfAIqxLrRsnKIBjsfFEyhBS9uYvQ0D0ZEcWpKJdAJQwkRha4ezmcZlfkEq6Jik2erPO8eqiyL4FqNc9vN4LK4ZBF1hlGrk0oOyWKeMgfcY%2FpIVRFjC7M8hKuO%2FDviaSXxRWfH%2F4niPh4tu6G28eiKmD2ahrmPD%2BmMcCUaU58t1aQeLTLu85uw%2BaGPz4q8jhoZs1344Yh%2B%2BlBgcjvCJaJAHUiRjlok2%2Fdmcf2VzA4TtZYDmvnP41CvKhtPFF8VkRWhP%2BUYwTjSOviYTFbG%2BoRNRYfSEWViKridM6UxDUv7BMnucT9Bu7JcbtJ4Ox9jttmZsG%2BImBJk4a31GhZDFamg%2FFTGsSnFKbvhmo7wVvkxRH%2Bm0ZSXwOm8VsN9vcUO4LtfojYKSTuFmtTDdPNA2AaIu%2BZ2XJOz7028V9P7kYin4W%2F5Ne7s2%2Bo0JWeOzUiW5Hl5ppxxWt7r5VrA8%2BmOWtn1DMpuCMJ2w7%2BkwVJ0DtXp75EnZJGB5CThAUMflE5wgV%2FUqEnUj5l%2BmUbGITaavukYi8nFDO0Sn5IQgXGVXmlmFcs0sHlvASxvy02ih7W1Q803l6Q9r3sOkZwf0cVjFCOVSwSXPVVPtl03kpYmDkdvMtaAg1dEcvSOgzK3euwdEBpV%2BNSc0rJZm5MYOvTtyqrUQofVKcAoTUoMsxm2M8ZFPoufVQ6b%2BnsGM3RXbjGCaCR09n0XIRDyXcOXoDGSseV2C6c3ANDi%2B19w%2BYu8qC4OGtEFfpJgsEbS4LOW7qi2CDwWlmbVaVUTI91jYZidNp4vfm5q5bjHQ131wqFeLCa7eGYqn6MaxCAF3zoVeHmnRmvXqQ3TUkCm3rCpIqN6Y4m3a9gbvB7AvFfLLiPdBxv4DkRE%2Fc%2FAxwbKrdIGfxojInMwFpU8Iyl73XInoHlQZejynBNXJiYxurTnHApQdPD2TOX9yaCOBFNOVvoABnK1KB4v1NkEWtUYc0gEVoEXMM%2FJHs4D9mUKOLI9%2FRmio%2BpeYD3NoLl%2Bm68NmaM1sZtmZ%2FO0gv%2FkPmcq10NX3T1%2B3ArD3VbIcg68dxpQGm1WMVxp%2B6EQzDb4aZaZU%2FUgJpOF6zVtS8Q9V63dZHX%2F37fQN94B08JSX6CyQhk%2FyklVt9FxKJHzrntJ0G5mp6mnDiNBClv2sM%2FU9XOp57Jtq6wlIir7S&__VIEWSTATEENCRYPTED=&__EVENTVALIDATION=U%2Bp5fRmsMtgYQADpqUy%2FGh%2B8vzuApvAOStKMWsd1mWNkNNzuhn1M%2B4%2BmPVB%2BF7vJpwH1fJYKHDMn5naVX9XTcZ4%2BODUji8ENj%2Fep2%2F%2FmArM%3D&ctl00%24quotes_content_left%24submitString=5d%7Ctrue%7CAA

Response

HTTP/1.0 200 OK
Content-Type: application/octet-stream; charset=utf-8
Content-Disposition: attachment; filename=HistoricalQuotes.csv
X-Powered-By: ASP.NET
Content-Length: 377
Expires: Sun, 30 Jun 2013 03:17:04 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Sun, 30 Jun 2013 03:17:04 GMT
Set-Cookie: NSC_W.TJUFEFGFOEFS.OBTEBR.80=ffffffffc3a08e3745525d5f4f58455e445a4a423660;expires=Sun, 30-Jun-2013 03:27:04 GMT;path=/;httponly
X-Cache: MISS from NORRX0200Bay11
X-Cache-Lookup: MISS from NORRX0200Bay11:10088
Via: 1.0 NORRX0200Bay11 (squid/3.1.14)
Connection: close

"date","close","volume","open","high","low"

"2013/06/28","7.8200","14525620.0000","7.8300","7.8800","7.7800"
"2013/06/27","7.8700","13891800.0000","7.8000","7.8700","7.7600"
"2013/06/26","7.7500","23732700.0000","7.9600","7.9600","7.7200"
"2013/06/25","7.9200","16871440.0000","7.8700","7.9300","7.8300"
"2013/06/24","7.7900","29888500.0000","7.9600","7.9700","7.7050"

-

---

I'd like to get the csv file downloaded using WinHTTP, but the main purpose of this is to learn how to use WinHTTP.

[trancexx]

So that getQuotes() sets form field automatically based on symbol and time period for which you want the data for. This works fine for me:

#include "Winhttp.au3"


Local $sOut = DownloadThatThing("AA", "5d")

ConsoleWrite("!ERROR = " & @error & @CRLF)
ConsoleWrite("+>$sOut : " & @CRLF & $sOut & @CRLF & @CRLF)
;~ MsgBox(4096, "Response", StringFormat($sOut))


Func DownloadThatThing($sSymbol, $sTimePeriod = "3m")

    ; Initialize and get session handle
    Local $hOpen = _WinHttpOpen()
    ; Get connection handle
    Local $hConnect = _WinHttpConnect($hOpen, "www.nasdaq.com")

    ; Fill the form (id of the form is -getFile-)
    Local $sHTML = _WinHttpSimpleFormFill($hConnect, "symbol/" & $sSymbol & "/historical", "getFile", _
            "quotes_content_left_submitString", $sTimePeriod & "|true|" & $sSymbol) ; <- filling only this (thre rest is left default)

    Local $iErr = @error ; collect error

    ; Close handles
    _WinHttpCloseHandle($hConnect)
    _WinHttpCloseHandle($hOpen)

    ; Return whatever the result
    Return SetError($iErr, 0, $sHTML)
EndFunc

Edited June 30, 2013 by trancexx

[big_daddy]

That was less complicated than I thought. Thank you for the help.