oemript

How to code this PowerShell?

12 posts in this topic

#1 ·  Posted (edited)

I have used the following url to fetch the historical data from yahoo finance. From last 16th May, 2017 the url is not working.

http://ichart.finance.yahoo.com/table.csv?s=AAL&a=8&b=7&c=2003&d=4&e=16&f=2017&g=d&ignore=.csv

Seems like they have changed the url and the new url is:

https://query1.finance.yahoo.com/v7/finance/download/AAL?period1=1494873000&period2=1494959400&interval=1d&events=history&crumb=l0aEtuOKocj

In the above changed URL has a session cookie which is crumb. Is there any idea how to get this cookie programmatically(in PowerShell)?
Does anyone have any suggestions?
Thanks in advance for any suggestions :>

Edited by oemript

Share this post


Link to post
Share on other sites



#2 ·  Posted

Nobody?

Share this post


Link to post
Share on other sites

#3 ·  Posted

This isn't a powershell community, but rather more an Autoit3 scripting one, so maybe not the most logical place to ask?

Jos 


Visit the SciTE4AutoIt3 Download page for the latest versions  - Beta files                                How to post scriptsource        Forum Rules
 
Live for the present,
Dream of the future,
Learn from the past.
  :)

Share this post


Link to post
Share on other sites

#4 ·  Posted

Hello. You can use Powershell WebRequest.

 

Basically You need to do a first request to yahoo finance page parse the html an get crumb then you add it to your query url and do the second web request.

 

Saludos

Share this post


Link to post
Share on other sites

#5 ·  Posted

1 hour ago, Danyfirex said:

Hello. You can use Powershell WebRequest.

 

Basically You need to do a first request to yahoo finance page parse the html an get crumb then you add it to your query url and do the second web request.

 

Saludos

I would try using either CURL or wget (depending on exactly what you are trying to do) to obtain the information.  These tools have an insane number of features and might let you accomplish what you want to accomplish.

Share this post


Link to post
Share on other sites

#6 ·  Posted (edited)

On 5/21/2017 at 9:36 PM, Danyfirex said:

Hello. You can use Powershell WebRequest.

Basically You need to do a first request to yahoo finance page parse the html an get crumb then you add it to your query url and do the second web request.

Once yahoo finance page parse the html into a text file D:\Yahoo.txt.

Do you have any suggestions on what kind of coding can retrieve "CrumbStore" from text file?

Thanks, to everyone very much for any suggestions :>

 

Edited by oemript

Share this post


Link to post
Share on other sites

#7 ·  Posted

hello. Something like this should work.

$Data = Get-Content 'c:\yahoo.txt'
$result = [regex]::match($Data, '"CrumbStore":{"crumb":"(.*?)"').Groups[1].Value

Saludos

Share this post


Link to post
Share on other sites

#8 ·  Posted (edited)

20 minutes ago, Danyfirex said:

hello. Something like this should work.


$Data = Get-Content 'c:\yahoo.txt'
$result = [regex]::match($Data, '"CrumbStore":{"crumb":"(.*?)"').Groups[1].Value

Saludos

Once $result is retrieved, such as "5SBcSO78HGZ". I would like to know on how to add "5SBcSO78HGZ" into the end of following link.

https://query1.finance.yahoo.com/v7/finance/download/^GSPC?period1=1492876180&period2=1495468180&interval=1d&events=history&crumb=

Furthermore, if I want to download the csv file, would following code be correct?

(Invoke-WebRequest -Uri "https://query1.finance.yahoo.com/v7/finance/download/^GSPC?period1=1492876180&period2=1495468180&interval=1d&events=history&crumb=5SBcSO78HGZ" -UseBasicParsing).Content | Out-File -FilePath D:\GSPC.csv

Do you have any suggestions?

Thanks, to everyone very much for any suggestions :>

 

Edited by oemript

Share this post


Link to post
Share on other sites

#9 ·  Posted (edited)

Just append $result to the url and do the request and do your own tests.

 

Saludos

Edited by Danyfirex

Share this post


Link to post
Share on other sites

#10 ·  Posted (edited)

48 minutes ago, Danyfirex said:

Just append $result to the url and do the request.

 

Saludos

(Invoke-WebRequest -Uri "https://finance.yahoo.com/quote/%5EGSPC/history?p=%5EGSPC" -SessionVariable $Yahoo -UseBasicParsing).Content | Out-File -FilePath D:\Yahoo.txt

$Data = Get-Content 'd:\yahoo.txt'
$result = [regex]::match($Data, '"CrumbStore":{"crumb":"(.*?)"').Groups[1].Value
$URLresult = "https://query1.finance.yahoo.com/v7/finance/download/%5EGSPC?period1=1492876180&period2=1495468180&interval=1d&events=history&crumb=" + $result
(Invoke-WebRequest -Uri $URLresult -WebSession $Yahoo -UseBasicParsing).Content | Out-File -FilePath D:\Yahoo.csv

I get the correct URLresult, but when I download the file using Invoke-WebRequest on next step, it fails because of "Invalid cookie".

Is there anything wrong on how to handle session by using -SessionVariable and -WebSession?

Do you have any suggestions?

Thanks, to everyone very much for any suggestions :>

Edited by oemript

Share this post


Link to post
Share on other sites

#11 ·  Posted

Referring to following coding, when I manually download file from Yahoo by clicking the download link, the data shows on different cells properly, but when I download using following coding, data is shown everything on the same single cell 1 without splitting into different column as shown below:

Coding:

$URL = "https://query1.finance.yahoo.com/v7/finance/download/%5EGSPC?period1=1493072773&period2=1495664773&interval=1d&events=history&crumb=kK.4Q7f/hsM"

$file = "C:\GSCP.csv"

(Invoke-WebRequest -Uri $URL -WebSession $Y -UseBasicParsing).Content | Out-File -FilePath $file

File:

Date,Open,High,Low,Close,Adj Close,Volume
1990-01-08,3842.000000,3942.000000,3742.000000,3850.000000,3850.000000,0

Does Powershell have other function to download file instead of using (...).Content? which "(...).Content" should be the problem on downloading as a text.

When I try following coding, the format is wrong neither.

Invoke-WebRequest -Uri $URL -WebSession $Y | Out-File -FilePath $file

Do you have any suggestions?
Thanks, to everyone very much for any suggestions :>

Share this post


Link to post
Share on other sites

#12 ·  Posted

Hello. You can do something like this.

 

(Invoke-WebRequest -Uri $URL -WebSession $mysession -UseBasicParsing).Content | ConvertFrom-Csv -Delim ',' | Export-Csv -NoTypeInformation -Path $file -UseCulture

Saludos

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now