Sign in to follow this  
Followers 0
anxiom

monitor webpages

25 posts in this topic

hello

is there a tool that will monitor all the webpages I visit and it will save specific text/links/files?

the problem is that most of similar programs require you to enter a starting web address, then specify retrieval options and then let the program do the job

I want an integrated solution to do this within my browser to work "as I browse"

thanks

Share this post


Link to post
Share on other sites

If you use Internet Explorer, simply read out new values from registry using RegEdit.

Kurt


Awaiting Diablo III..

Share this post


Link to post
Share on other sites

Func FindAllLinks($link)
    Local $Recents[26]
    For $i = 1 To 25
        $Recents[$i] = RegRead("HKCU\Software\Microsoft\Internet Explorer\TypedURLs", "url" & $i)
    Next
    Return $Recents
EndFunc

Something like that..


Awaiting Diablo III..

Share this post


Link to post
Share on other sites

thanks but I didn't say about the typed urls

I said the text, the links and the files of webpages I visit, not the urls of the webpages I visit

Share this post


Link to post
Share on other sites

thanks but I didn't say about the typed urls

I said the text, the links and the files of webpages I visit, not the urls of the webpages I visit

In the help file, look at the _IE* functions... _IELinkGetCollection... there are plenty of examples on how to use those functions in the help file and on the forum, so you'll be able to find what you need by searching a bit.

Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer.

Share this post


Link to post
Share on other sites

In the help file, look at the _IE* functions... _IELinkGetCollection... there are plenty of examples on how to use those functions in the help file and on the forum, so you'll be able to find what you need by searching a bit.

there is nothing here, or I miss something?

http://www.autoitscript.com/autoit3/docs/functions.htm

Share this post


Link to post
Share on other sites

there is nothing here, or I miss something?

http://www.autoitscript.com/autoit3/docs/functions.htm

Use the helpfile on your PC Start --> All Programs --> AutoIt V3 --> AutoIt Helpfile. Then goto the section named User Defined Functions. Look for them there.

Share this post


Link to post
Share on other sites

Functions starting with _ are UDF's (User Defined Functions). Take a look in the help file, it is there under IE management or something like that.

Share this post


Link to post
Share on other sites

thanks

IE implies that these functions work with Internet Explorer only? not with opera or firefox?

Share this post


Link to post
Share on other sites

Functions starting with _ are UDF's (User Defined Functions). Take a look in the help file, it is there under IE management or something like that.

anyone?

Share this post


Link to post
Share on other sites

is there somewhere that says all these apply to Opera and Firefox? I can't find it

Share this post


Link to post
Share on other sites

look at the UDF online help in my sig

Autoit is Windows driven as IE is also Windows and uses Windows objects. Given the ability to use IE it would be a waist of programming time to work with the latest browser flavor. While I will not say it is not possible, IE works and works well and is supported* key word here.

Share this post


Link to post
Share on other sites

Umm, did you quote the wrong thing?

It would appear so :P One too low I guess ;)

Share this post


Link to post
Share on other sites

I understand you want a ready to run solution an would say that you better look for a proxy server where you can implement some kind of triger/monitor. A part from that it should be easy enough to monitor the addressfield of IE, Firefox, Opera, offbyon, flock... But what if you want to use Links, wget?

Share this post


Link to post
Share on other sites

I understand you want a ready to run solution an would say that you better look for a proxy server where you can implement some kind of triger/monitor. A part from that it should be easy enough to monitor the addressfield of IE, Firefox, Opera, offbyon, flock... But what if you want to use Links, wget?

thanks for your reply

first, I need something that will "monitor" every webpage I visit, as I browse the net

this monitoring has to be very accurate ofcourse, which means it must not miss any webpage even the webpages that are partially loaded etc

by "monitoring webpages" I mean to grab the text, links and files of every webpage I visit

by "the text of the webpage", I mean the text that is highlighted/selected when we click ctrl+A in a webpage (included any other "hidden" text, etc)

by "the links of the webpage", I mean the links that are grabbed when we hit ctrl+alt+L in Opera or any other method that shows all the links of the webpage (included any "hidden" links, javascript links, etc)

by "the files of the webpages", I mean the files that are included in the folder that is created when we save a webpage which created an html file and a folder (and any other hidden files, embedded files, etc)

as far as I know (and if you know something else, please inform me) the available methods that can monitor web browser traffic are these:

javascript can monitor webpages as I browse the net (opera, for example, has this javascript function: document.addEventListener('DOMContentLoaded',function() { where it does things when the webpages are loaded)

internet connection sniffer can monitor webpages as I browse the net, that can sniff urls

web proxy can monitor webpages as I browse the net, as it works as a cache proxy

which would be the best and easiest method?

then I need to apply filters to specify which of the text, links and files are useful and then we need to save the filtered information

any help would be much appreciated

thank you

Share this post


Link to post
Share on other sites

A proxy server will monitor and can save every page you browse through. Then all you need is a filter. If that aint the thing you want you could start using a browser that use the IE objects and implement the stuff you want with the _IE udf. The helpfile is quite comprehensive and you will find help in the forum.

If you insist in beeing able to use any browser a proxy solution is your only choice as I see it. You can then have a seperate application monitoring the pages you hit a hotkey on to log interesting pages and lett that application extract stuff from your proxy cache or even download the pages again.

Yuo can use greasemonkey to achive the monitoring in FireFox and probably something similar in opera.

I don't know of a ready to run solution and if that is what you wanted you should have posted in chat or another forum as this forum has a focus on AutoIt scripting.

Happy hunting

Uten

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0