How to code this PowerShell?

[oemript]

I have used the following url to fetch the historical data from yahoo finance. From last 16th May, 2017 the url is not working.

http://ichart.finance.yahoo.com/table.csv?s=AAL&a=8&b=7&c=2003&d=4&e=16&f=2017&g=d&ignore=.csv

Seems like they have changed the url and the new url is:

https://query1.finance.yahoo.com/v7/finance/download/AAL?period1=1494873000&period2=1494959400&interval=1d&events=history&crumb=l0aEtuOKocj

In the above changed URL has a session cookie which is crumb. Is there any idea how to get this cookie programmatically(in PowerShell)?
Does anyone have any suggestions?
Thanks in advance for any suggestions :>

Edited May 21, 2017 by oemript

[oemript]

Nobody?

[Jos]

This isn't a powershell community, but rather more an Autoit3 scripting one, so maybe not the most logical place to ask?

Jos 

[Danyfirex]

Hello. You can use Powershell WebRequest.

 

Basically You need to do a first request to yahoo finance page parse the html an get crumb then you add it to your query url and do the second web request.

 

Saludos

[MattHiggs]

1 hour ago, Danyfirex said:

Hello. You can use Powershell WebRequest.

 

Basically You need to do a first request to yahoo finance page parse the html an get crumb then you add it to your query url and do the second web request.

 

Saludos

I would try using either CURL or wget (depending on exactly what you are trying to do) to obtain the information.  These tools have an insane number of features and might let you accomplish what you want to accomplish.

[oemript]

On 5/21/2017 at 9:36 PM, Danyfirex said:

Hello. You can use Powershell WebRequest.

Basically You need to do a first request to yahoo finance page parse the html an get crumb then you add it to your query url and do the second web request.

Once yahoo finance page parse the html into a text file D:\Yahoo.txt.

Do you have any suggestions on what kind of coding can retrieve "CrumbStore" from text file?

Thanks, to everyone very much for any suggestions

 

Edited May 22, 2017 by oemript

[Danyfirex]

hello. Something like this should work.

$Data = Get-Content 'c:\yahoo.txt'
$result = [regex]::match($Data, '"CrumbStore":{"crumb":"(.*?)"').Groups[1].Value

Saludos

[oemript]

20 minutes ago, Danyfirex said:

hello. Something like this should work.

$Data = Get-Content 'c:\yahoo.txt'
$result = [regex]::match($Data, '"CrumbStore":{"crumb":"(.*?)"').Groups[1].Value

Saludos

Once $result is retrieved, such as "5SBcSO78HGZ". I would like to know on how to add "5SBcSO78HGZ" into the end of following link.

https://query1.finance.yahoo.com/v7/finance/download/^GSPC?period1=1492876180&period2=1495468180&interval=1d&events=history&crumb=

Furthermore, if I want to download the csv file, would following code be correct?

(Invoke-WebRequest -Uri "https://query1.finance.yahoo.com/v7/finance/download/^GSPC?period1=1492876180&period2=1495468180&interval=1d&events=history&crumb=5SBcSO78HGZ" -UseBasicParsing).Content | Out-File -FilePath D:\GSPC.csv

Do you have any suggestions?

Thanks, to everyone very much for any suggestions

 

Edited May 22, 2017 by oemript

[Danyfirex]

Just append $result to the url and do the request and do your own tests.

 

Saludos

Edited May 22, 2017 by Danyfirex

[oemript]

48 minutes ago, Danyfirex said:

Just append $result to the url and do the request.

 

Saludos

(Invoke-WebRequest -Uri "https://finance.yahoo.com/quote/%5EGSPC/history?p=%5EGSPC" -SessionVariable $Yahoo -UseBasicParsing).Content | Out-File -FilePath D:\Yahoo.txt

$Data = Get-Content 'd:\yahoo.txt'
$result = [regex]::match($Data, '"CrumbStore":{"crumb":"(.*?)"').Groups[1].Value
$URLresult = "https://query1.finance.yahoo.com/v7/finance/download/%5EGSPC?period1=1492876180&period2=1495468180&interval=1d&events=history&crumb=" + $result
(Invoke-WebRequest -Uri $URLresult -WebSession $Yahoo -UseBasicParsing).Content | Out-File -FilePath D:\Yahoo.csv

I get the correct URLresult, but when I download the file using Invoke-WebRequest on next step, it fails because of "Invalid cookie".

Is there anything wrong on how to handle session by using -SessionVariable and -WebSession?

Do you have any suggestions?

Thanks, to everyone very much for any suggestions

Edited May 22, 2017 by oemript

[oemript]

Referring to following coding, when I manually download file from Yahoo by clicking the download link, the data shows on different cells properly, but when I download using following coding, data is shown everything on the same single cell 1 without splitting into different column as shown below:

Coding:

$URL = "https://query1.finance.yahoo.com/v7/finance/download/%5EGSPC?period1=1493072773&period2=1495664773&interval=1d&events=history&crumb=kK.4Q7f/hsM"

$file = "C:\GSCP.csv"

(Invoke-WebRequest -Uri $URL -WebSession $Y -UseBasicParsing).Content | Out-File -FilePath $file

File:

Date,Open,High,Low,Close,Adj Close,Volume
1990-01-08,3842.000000,3942.000000,3742.000000,3850.000000,3850.000000,0

Does Powershell have other function to download file instead of using (...).Content? which "(...).Content" should be the problem on downloading as a text.

When I try following coding, the format is wrong neither.

Invoke-WebRequest -Uri $URL -WebSession $Y | Out-File -FilePath $file

Do you have any suggestions?
Thanks, to everyone very much for any suggestions

[Danyfirex]

Hello. You can do something like this.

 

(Invoke-WebRequest -Uri $URL -WebSession $mysession -UseBasicParsing).Content | ConvertFrom-Csv -Delim ',' | Export-Csv -NoTypeInformation -Path $file -UseCulture

Saludos