Sign in to follow this  
Followers 0
bboysza

Web crawler script?

3 posts in this topic

Before I start dev on a webcrawler script, I thought I should ask if any has already written one. :o

What I need to do specifically, is crawl a site, and dump all hrefs and arefs to a log file. Basically, logging the source for each image. Also, I would need to then follow any hrefs (or other jscript or asp links) to other pages and then do the same thing there.

I don't need it written for me, just looking for some ideas for the architecture.

Thanks,

Ben


Ben

Share this post


Link to post
Share on other sites



Before I start dev on a webcrawler script, I thought I should ask if any has already written one. :o

What I need to do specifically, is crawl a site, and dump all hrefs and arefs to a log file. Basically, logging the source for each image. Also, I would need to then follow any hrefs (or other jscript or asp links) to other pages and then do the same thing there.

I don't need it written for me, just looking for some ideas for the architecture.

Thanks,

Ben

Searched the word Spider... and poof: http://www.autoitscript.com/forum/index.ph...26entry125826 :geek:

[center]Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer.[/center]

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0