Sign in to follow this  
Followers 0
psynegy

Read a web page's content...

29 posts in this topic

Ok, basically I have been looking through the forums here for a good hour now and found various things of interest and got stuck in with some of them... But I have been looking for a language (other than the classic C/C+/C++/VBs) which was powerful enough to allow me to 'read' a web pages contents...

For example, say the situation was that I wanted to view the number of users on my website at the current time... I have made and uploaded a PHP script that gives out a number and ONLY a number. I want that number read by the client side application and echoed out into a text box or similar.

I hope someone can help me with this, as it is vital for my current project!

Share this post


Link to post
Share on other sites



IE.au3 makes easy work of these sorts of tasks (see my sig). You can either use _IEBodyReadHTML() to get all of the HTML from the body of the page and strip out of it what you are looking for, or you can use the Document Object Model (DOM) to drill to the specific element you want and get its value.

To get more specific help, you'll need to provide a more specific example.

Dale

Ok, basically I have been looking through the forums here for a good hour now and found various things of interest and got stuck in with some of them... But I have been looking for a language (other than the classic C/C+/C++/VBs) which was powerful enough to allow me to 'read' a web pages contents...

For example, say the situation was that I wanted to view the number of users on my website at the current time... I have made and uploaded a PHP script that gives out a number and ONLY a number. I want that number read by the client side application and echoed out into a text box or similar.

I hope someone can help me with this, as it is vital for my current project!


Free Internet Tools: DebugBar, AutoIt IE Builder, HTTP UDF, MODIV2, IE Developer Toolbar, IEDocMon, Fiddler, HTML Validator, WGet, curl

MSDN docs: InternetExplorer Object, Document Object, Overviews and Tutorials, DHTML Objects, DHTML Events, WinHttpRequest, XmlHttpRequest, Cross-Frame Scripting, Office object model

Automate input type=file (Related)

Alternative to _IECreateEmbedded? better: _IECreatePseudoEmbedded  Better Better?

IE.au3 issues with Vista - Workarounds

SciTe Debug mode - it's magic: #AutoIt3Wrapper_run_debug_mode=Y Doesn't work needs to be ripped out of the troubleshooting lexicon. It means that what you tried did not produce the results you expected. It begs the questions 1) what did you try?, 2) what did you expect? and 3) what happened instead?

Reproducer: a small (the smallest?) piece of stand-alone code that demonstrates your trouble

Share this post


Link to post
Share on other sites

Ok, I got that bit sorted thanks! But how do you 'wait' for a webpage to be loaded or find out wether the page is loaded or not...?

Share this post


Link to post
Share on other sites

maybe

_IELoadWait()

Wait for a browser page load to complete before returning

in IE.au3

Most of the functions that require it call _IELoadWait( ) automatically. All of these functions have an optional parameter of "wait" So that you can turn it on or off (by default, they all wait for page load to complete before moving to the next statementYou can call _IELoadWait manually if you want to but there usually isn't a reason to.

Dale


Free Internet Tools: DebugBar, AutoIt IE Builder, HTTP UDF, MODIV2, IE Developer Toolbar, IEDocMon, Fiddler, HTML Validator, WGet, curl

MSDN docs: InternetExplorer Object, Document Object, Overviews and Tutorials, DHTML Objects, DHTML Events, WinHttpRequest, XmlHttpRequest, Cross-Frame Scripting, Office object model

Automate input type=file (Related)

Alternative to _IECreateEmbedded? better: _IECreatePseudoEmbedded  Better Better?

IE.au3 issues with Vista - Workarounds

SciTe Debug mode - it's magic: #AutoIt3Wrapper_run_debug_mode=Y Doesn't work needs to be ripped out of the troubleshooting lexicon. It means that what you tried did not produce the results you expected. It begs the questions 1) what did you try?, 2) what did you expect? and 3) what happened instead?

Reproducer: a small (the smallest?) piece of stand-alone code that demonstrates your trouble

Share this post


Link to post
Share on other sites

#6 ·  Posted (edited)

For some reason I get an error when running my sript, which happens intermittently (cant spell that word :lmao:) And I think its down to it not having loaded, because its instantly taking the body of the page and putting it into an input box...

Another question, how do you do GUI password boxes?

GuiCreate("DynaIP V1", 200, 200)
GuiSetIcon(@SystemDir & "\shell32.dll", 43)

$oIE = ObjCreate("Shell.Explorer.2")
GUICtrlCreateObj($oIE, 0, 0, 200, 180)
GUICtrlSetStyle($oIE, $WS_VISIBLE)
$oIE.navigate("http://www.nxgaming.co.uk/sigs/ip.php")

$status = GuiCtrlCreateInput("Loading...", 0, 180, 200, 20)
; GUI MESSAGE LOOP
GuiSetState()
While GuiGetMsg() <> $GUI_EVENT_CLOSE
    _IELoadWait($oIE)
    $body = _IEBodyReadHTML($oIE) 
    If($body = GUICtrlRead($status)) Then
    ;Nothing? Or is there a way to say, "If $body is NOT the same etc etc..."
    Else
        GuiCtrlSetData($status, $body)
    EndIf
WEnd
Edited by psynegy

Share this post


Link to post
Share on other sites

#7 ·  Posted (edited)

For some reason I get an error when running my sript, which happens intermittently (cant spell that word :lmao:) And I think its down to it not having loaded, because its instantly taking the body of the page and putting it into an input box...

Another question, how do you do GUI password boxes?

GuiCreate("DynaIP V1", 200, 200)
GuiSetIcon(@SystemDir & "\shell32.dll", 43)

$oIE = ObjCreate("Shell.Explorer.2")
GUICtrlCreateObj($oIE, 0, 0, 200, 180)
GUICtrlSetStyle($oIE, $WS_VISIBLE)
$oIE.navigate("http://www.nxgaming.co.uk/sigs/ip.php")

$status = GuiCtrlCreateInput("Loading...", 0, 180, 200, 20)
; GUI MESSAGE LOOP
GuiSetState()
While GuiGetMsg() <> $GUI_EVENT_CLOSE
    _IELoadWait($oIE)
    $body = _IEBodyReadHTML($oIE) 
    If($body = GUICtrlRead($status)) Then
;Nothing? Or is there a way to say, "If $body is NOT the same etc etc..."
    Else
        GuiCtrlSetData($status, $body)
    EndIf
WEnd
it looks like you had it right in your code before you edited...

$password = InputBox("Security Check", "Enter your password...", "TEST", "*")

that will store the user input into a variable that you can then check yourself with a statement like:

If $password <> "MyPassw0rd!" Then Exit

***edit***

oh yeah, welcome to the forum!

Edited by cameronsdad

1100111 00001011101111 00011101101111 00010111100100 00001111110100 00110111110010 00101101111001 0011100i didn't make up this form of encryption, but i like it.credit to the lvl 6 challenge on arcanum.co.nz

Share this post


Link to post
Share on other sites

Ah, well before I edited I realised that that was something I was testing :lmao:, but I want to have a login box (the web page bit will be hidden) and when you click login it will pop the info into the web page, but my problem is having the script look for what output the webpage is giving BEFORE its finished loading... Any ideas?

And thanks for the welcome! ;)

Share this post


Link to post
Share on other sites

Ah, well before I edited I realised that that was something I was testing :lmao:, but I want to have a login box (the web page bit will be hidden) and when you click login it will pop the info into the web page, but my problem is having the script look for what output the webpage is giving BEFORE its finished loading... Any ideas?

And thanks for the welcome! ;)

ok, it looks like you're kind of implementing some of the functionality of ie.au3 just through COM without using the actual UDF. While that may work, you're kind of reinventing the wheel, and bypassing some of the cool functionality that dale has put in (including automatic load waiting...) I've personally used ie.au3 for all kinds of web projects, and reccomend it several times a day on this forum. I'm going to play with your code a little on this side and see if i can't get it to work the way you want (although i'm going to use google.com as the url because i'm at work).

1100111 00001011101111 00011101101111 00010111100100 00001111110100 00110111110010 00101101111001 0011100i didn't make up this form of encryption, but i like it.credit to the lvl 6 challenge on arcanum.co.nz

Share this post


Link to post
Share on other sites

Just the loading problem or the whole GUI interface, including user login? I dont mind doing it but I REALLY dont mind if you do :lmao:!!!

Thanks for your help!

Share this post


Link to post
Share on other sites

(although i'm going to use google.com as the url because i'm at work).

:lmao: You call that work? ;)

[center]Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer.[/center]

Share this post


Link to post
Share on other sites

You know where he works? :lmao:

Share this post


Link to post
Share on other sites

You know where he works? :lmao:

therein lies the joke. i have a job, which i am paid for, but i've automated everything to the point that i do less than 10 minutes of 'work' a day. and i spend the rest of the day here on the forums helping people. this code worked for me... all i did was take out your loadwait()

#include<guiconstants.au3>
#include<ie.au3>
GuiCreate("DynaIP V1", 200, 200)
GuiSetIcon(@SystemDir & "\shell32.dll", 43)

$oIE = ObjCreate("Shell.Explorer.2")
GUICtrlCreateObj($oIE, 0, 0, 200, 180)
GUICtrlSetStyle($oIE, $WS_VISIBLE)
$oIE.navigate("http://google.com")

$status = GuiCtrlCreateInput("Loading...", 0, 180, 200, 20)
; GUI MESSAGE LOOP
GuiSetState()
While GuiGetMsg() <> $GUI_EVENT_CLOSE
    
    $body = _IEBodyReadHTML($oIE) 
    If($body = GUICtrlRead($status)) Then
;Nothing? Or is there a way to say, "If $body is NOT the same etc etc..."
    Else
        GuiCtrlSetData($status, $body)
    EndIf
WEnd

1100111 00001011101111 00011101101111 00010111100100 00001111110100 00110111110010 00101101111001 0011100i didn't make up this form of encryption, but i like it.credit to the lvl 6 challenge on arcanum.co.nz

Share this post


Link to post
Share on other sites

#14 ·  Posted (edited)

Nope, I am still getting that intermittant error!

Line 1137 (File "D:\Program Files\AutoIt3\beta\Include\ie.au3"):

Return $o_object.document,body.innterHTML

Return $o_object.document.body^ ERROR

Error Variable must be of type "Object".

Edited by psynegy

Share this post


Link to post
Share on other sites

Nope, I am still getting that intermittant error!

one thing that may be causing it, is that when you're creating your object, you're making it a "shell.explorer" object, but the IE.AU3 is is for working with $o_object = "InternetExplorer.Application" objects. the two may not have exactly the same methods.

1100111 00001011101111 00011101101111 00010111100100 00001111110100 00110111110010 00101101111001 0011100i didn't make up this form of encryption, but i like it.credit to the lvl 6 challenge on arcanum.co.nz

Share this post


Link to post
Share on other sites

How do you sugest I resove this problem, and if thats true, then why does it only come up every now and then? Which leads me back to this theory of it calling it before the page has loaded....

Share this post


Link to post
Share on other sites

I tried one of the other web page scripts which loads an external IE window and controls that, I again get an intermittant error!

Line 1236 (File "D:\Program Files\AutoIt3\beta\Include\IE.au3");

While ($o_object.document.readyState <> "complete") and ($o_object.document.readyState <> 4)

While ($o_object.document^ ERROR

Error: The requested action with this object has failed.

That came from a peice of example code on the IE.au3 topic, which is as follows:

; ----------------------------------------------------------------------------
;
; AutoIt Version: 3.1.0
; Author:        A.N.Other <myemail@nowhere.com>
;
; Script Function:
;   Template AutoIt script.
;
; ----------------------------------------------------------------------------

; Script Start - Add your code below here

#include <IE.au3>

$oIE = _IECreate()
_IENavigate($oIE, "http://www.google.com")

$oForm = _IEFormGetObjByName($oIE, "f")
$oQuery = _IEFormElementGetObjByName($oForm, "q")

_IEFormElementSetValue($oQuery, "place your search string here")
_IEFormSubmit($oForm)
_IELoadWait($oIE)

Exit

So... I dont know what to do... From where I stand, I can only think that IE.au3 is faulty or I have a dud version, but its a copy and paste job... Any ideas?

Share this post


Link to post
Share on other sites

if you just want to get the source why not just use _INetGetSource

i was thinking the same thing, then you wouldn't be using anything from the ie.au3 and wouldn't need to bulk up the script with an extra include

1100111 00001011101111 00011101101111 00010111100100 00001111110100 00110111110010 00101101111001 0011100i didn't make up this form of encryption, but i like it.credit to the lvl 6 challenge on arcanum.co.nz

Share this post


Link to post
Share on other sites

Well, now I have done some research into this function, I got it working and its awesome, now why wasnt this suggested RIGHT at the begging! :lmao: Anyways, thanks for all the help guys!

But that question still remains... How do you do a password box in the GUI interface without doing a prompt thing??

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0