Powershell - how to import data from aspx websites into text file?

[oemript]

Referring to following link, I would like to know on how to save data from aspx websites into text file using Powershell.


When I input any stock code, such as 5, web page display related information, and would like to know on how to save those data into Text file.


Does anyone have any suggestions?
Thanks in advance for any suggestions


http://www.hkexnews.hk/sdw/search/searchsdw.aspx

[Melba23]

oemript,

As all your threads seem to deal with Powershell, would you not be better off posting them on a more focused forum? Here we focus on Autoit and any questions solely relating to other apps/languages take very much the back seat.

M23

[ptrex]

@oemript 

Actually this is now possible in 2 ways ...

1. Using DotNet CLR 

 

See :  .NET CLR Danyfirex Examples.au3 included in the ZIP file Example 6.

Func _Example6()
    Local $oAssembly = _CLR_LoadLibrary("System.Net")
    ConsoleWrite("!$oAssembly: " & IsObj($oAssembly) & @CRLF)
    Local $oWebClient = _CLR_CreateObject($oAssembly, "System.Net.WebClient")
    ConsoleWrite("!$oWebClient: " & IsObj($oWebClient) & @CRLF)
    Local $sString = $oWebClient.DownloadString("http://checkip.dyndns.org")
    ConsoleWrite("!" & $sString & @CRLF)
    $sString = StringRegExp($sString, '((?:\d{1,3}\.){3}\d{1,3})', 3)
    MsgBox(0, "WebClient.DownloadString", "Current IP Address: " & (IsArray($sString) ? $sString[0] : "Ups something was wrong... :("))
EndFunc   ;==>_Example6

2. Using Using DotNet CLR / Powershell Automation

expandcollapse popup

#include "CLR.Au3"

    _Run_PSHost_Script('$client = new-object System.Net.WebClient' & @CRLF & '$Client.DownloadFile("https://msdn.microsoft.com/en-us/library/system.management.automation.psobject(v=vs.85).aspx",[Environment]::GetFolderPath("Desktop")+"\ASPX.txt")')

Func _Run_PSHost_Script($PSScript, $iOutput = 0)
    Local $oAssembly = _CLR_LoadLibrary("System.Management.Automation")
    ConsoleWrite("!$oAssembly: " & IsObj($oAssembly) & @CRLF)

    ; Create Object
    Local $pAssemblyType = 0
    $oAssembly.GetType_2("System.Management.Automation.PowerShell", $pAssemblyType)
    ConsoleWrite("$pAssemblyType = " & Ptr($pAssemblyType) & @CRLF)

    Local $oActivatorType = ObjCreateInterface($pAssemblyType, $sIID_IType, $sTag_IType)
    ConsoleWrite("IsObj( $oAssemblyType ) = " & IsObj($oActivatorType) & @TAB & @CRLF)

    ; Create Object
    Local $pObjectPS = 0
    $oActivatorType.InvokeMember_3("Create", 0x158, 0, 0, 0, $pObjectPS)
    ConsoleWrite("IsObject: " & IsObj($pObjectPS) & @TAB & "$pObject: " & ObjName($pObjectPS) & @CRLF)

; <<<<<<<<<<<<<<<<<<< PS COMMAND HERE >>>>>>>>>>>>>>>>>>>>

    $pObjectPS.AddScript($PSScript) 

    $objAsync = $pObjectPS.BeginInvoke()
;~      ConsoleWrite("$objAsync " & IsObj($objAsync & @TAB & "$pObject: " & ObjName($objAsync) ) & @CRLF)

    While $objAsync.IsCompleted = False
;~      ConsoleWrite($objAsync.IsCompleted & @CRLF)
        ContinueLoop
    WEnd
        ConsoleWrite("Completed : " & $objAsync.IsCompleted & @CRLF)

    $objPsCollection = $pObjectPS.EndInvoke($objAsync)

EndFunc

Enjoy 

[oemript]

I would like to know on whether above coding can be run under PowerShell or not, which seems to be .Net coding.

Do you have any suggestions?
Thanks, to everyone very much for any suggestions (^v^)

Edited September 27, 2017 by Melba23
Huge quote removed

[Melba23]

oemript,

When you reply, please use the "Reply to this topic" button at the top of the thread or the "Reply to this topic" editor at the bottom rather than the "Quote" button - responders know what they wrote and it just pads the thread unnecessarily.

M23

[ptrex]

@oemript 

1. PowerShell :

The script used and executed is plain PowerShell ?

You just copy and paste it in your PowerShell Console or PowerShell ISE.

$client = new-object System.Net.WebClient; $Client.DownloadFile("https://msdn.microsoft.com/en-us/library/system.management.automation.psobject(v=vs.85).aspx",[Environment]::GetFolderPath("Desktop")+"\ASPX.txt

Give it a try and you will see that this is working out of the box.

1.  DotNet or PowerShell :

As you might know PowerShell runs on top of .NET... , so we can access all .NET classed using PoswerShell

Because Powershell has no Cmdlet to download files from the internet. We therefore need to use a DotNET class to overcome this.

So basically we use the .NET "System.Management.Automation" Assembly in AutoIT.

Local $oAssembly = _CLR_LoadLibrary("System.Management.Automation")

To run and execute the PowerShell Script we use "System.Management.Automation.PowerShell" Class.

If you want to know more about all of this ... please check here  :

At the bottom of the first post there is a nice PDF that explains the whole concept behind it : .NET CLR Framework for AutoIT.pdf

 

[oemript]

Iin order to retrieving text by submitting 5 on following link using PowerShell and save into a text file as final result.
http://www.hkexnews.hk/sdw/search/searchsdw.aspx

$client = new-object System.Net.WebClient; $Client.DownloadFile("https://msdn.microsoft.com/en-us/library/system.management.automation.psobject(v=vs.85).aspx",[Environment]::GetFolderPath("Desktop")+"\ASPX.txt

$client = new-object System.Net.WebClient; $Client.DownloadFile("http://www.hkexnews.hk/sdw/search/searchsdw.aspx",[Environment]::GetFolderPath("Desktop")+"\ASPX.txt

I would like to know on where to insert 5 on above parameter.

Do you have any suggestions?
Thanks, to everyone very much for any suggestions (^v^)

 

Edited September 27, 2017 by oemript

[ptrex]

Hi oemript,

I am not sure why you want to use PowerShell persé, in order to get the data from a website? 

When I enter 5 I get the result listed for HSBC bank correct ?

If you want to do Webbrowser automation AutoIT has much better tools to do so... 

Maybe this can give you a good headstart : 

 

Regards (in Kowloon :-)

ptrex

 

[oemript]

I prefer to use PowerShell for this task, based on following coding, do you have any suggestions on how to insert 5 on above parameter, then download it and save as a text file.

$client = new-object System.Net.WebClient; $Client.DownloadFile("http://www.hkexnews.hk/sdw/search/searchsdw.aspx",[Environment]::GetFolderPath("Desktop")+"\ASPX.txt

Do you have any suggestions?
Thanks, to everyone very much for any suggestions (^v^)

Edited September 28, 2017 by oemript

[ptrex]

Sorry I don't have examples on this myself ... but there's plenty them around on the internet...

This is a good start already :

http://www.westerndevs.com/simple-powershell-automation-browser-based-tasks/ 

I don't have the time right now to do a fuil coding on your requirements, but maybe next week, can't make any promises though...

 

[oemript]

2 minutes ago, ptrex said:

Sorry I don't have examples on this myself ... but there's plenty them around on the internet...

This is a good start already :

http://www.westerndevs.com/simple-powershell-automation-browser-based-tasks/ 

I don't have the time right now to do a fuil coding on your requirements, but maybe next week, can't make any promises though...

 

No problem, I can wait and very appreciated your time and efforts.

Thanks, to everyone very much for any suggestions (^v^)

[ptrex]

Here you go ...

$ieObject = New-Object -ComObject 'InternetExplorer.Application'

$ieObject

$ieObject.Visible = $true

$ieObject.Navigate("http://www.hkexnews.hk/sdw/search/searchsdw.aspx")

while($ieObject.Busy) { Start-Sleep -Milliseconds 50 }

$Doc = $ieObject.Document

# $Doc.getElementsByName("txtStockCode") | % {$_.Name,$_.Type,$_.Value}

$Doc.getElementById("txtStockCode").value = "5"

$Doc.getElementsByName("btnSearch") | % {$_.Name,$_.Id} 

$Button = $Doc.getElementById("btnSearch")

$Button.click()

# $Doc.body.innerHTML

$Doc.body.innerHTML | Out-File "C:\Temp\HTML.txt" 


$ieObject.Quit()

Success !

 

Edited September 28, 2017 by ptrex

[oemript]

I have tried those coding, it seems that there is a problem on saving content into text file and ie cannot be closed automatically.

# $Doc.body.innerHTML

$Doc.body.innerHTML | Out-File "C:\Temp\HTML.txt"

Furthermore, I have tried following simple html page, the coding still get problem to save into a text file too. The text file is blank without any text.

$ieObject = New-Object -ComObject 'InternetExplorer.Application'

$ieObject

$ieObject.Visible = $true

$ieObject.Navigate("http://www.hkexnews.hk/index_c.htm")
# $Doc.body.innerHTML

$Doc.body.innerHTML | Out-File "C:\Temp\HTML.txt"
$ieObject.Quit()

Do you have any suggestions on how to solve this issue?

Thanks, to everyone very much for any suggestions (^v^)

Edited September 29, 2017 by oemript

[ptrex]

All works fine on my side ?

So best is that you take this up in a PowerShell Forum now, since these questions are really PS specific....

rgds

P.

[oemript]

1 hour ago, ptrex said:

All works fine on my side ?

So best is that you take this up in a PowerShell Forum now, since these questions are really PS specific....

rgds

P.

 

I am working on this issues on PowerShell Forum.

Thanks, to everyone very much for any suggestions (^v^)

Edited September 29, 2017 by oemript