Collect all URLs of a web domain

[xOAT]

I am looking for a way to collect and make a list of all the web pages within a domain.

For example, if it was used on "http://www.somewebsite.com/" it would find and make a list of existing URLs such as "http://www.somewebsite.com/home.html", "http://www.somewebsite.com/fun/id=1234", or "http://www.somewebsite.com/fun/games/6873"... etc.

Thanks to anyone who responds.

Edited March 4, 2011 by xOAT

[PsaltyDS]

That's not what we do here. This is not a place to request other to write a script for, it's to help you with issues in YOUR use of AutoIt.

Download AutoIt and the SciTE4AutoIt3 editor. Read through the help file (AutoIt3.chm), run the simple tutorials there, and try some of the example scripts under functions that interest you. Then start coding your script. If you get stuck, post the part of the code you are working on with a specific AutoIt question.

[xOAT]

That's not what we do here. This is not a place to request other to write a script for, it's to help you with issues in YOUR use of AutoIt.

Download AutoIt and the SciTE4AutoIt3 editor. Read through the help file (AutoIt3.chm), run the simple tutorials there, and try some of the example scripts under functions that interest you. Then start coding your script. If you get stuck, post the part of the code you are working on with a specific AutoIt question.

I understand what you are saying, but this is not a request for a program. I have made many AutoIt programs before, but for this program I don't know how to begin.

What would be the steps to accomplish my desired program (in pseudo code)? Would I have to run thought all the possible string combinations and check if they exist? I am looking for a basic run down of the steps needed to make this.

Edited March 5, 2011 by xOAT

[smartee]

Sure you can go the way of string combinations if you have the most powerful cpu in the world and some spare time together with a speedy web connection. Or you can google "how to make a web crawler/spider"