How can I automate Webpage Extraction?

[niteshsingh]

Hello there, 

I'm an ASP.NET web application developer.

I've installed this tool and would like to learn automation possibilities related to web extraction/scrapping.

Please provide suitable resources for the above.

Also here are few more of my queries:

1. Is it possible to integrate this with asp.net or php web applications?

2. Is there an eBook or tutorials for learning the use of this tool?

3. Which language do I need to be pro efficient in while making use of this tool?

 

Looking forward to replies.

Regards,

Nitesh Singh

[JLogan3o13]

Welcome to the AutoIT forum. In answer to your questions:

 

I've installed this tool and would like to learn automation possibilities related to web extraction/scrapping.

Please provide suitable resources for the above.

 

Very vague. Please do a search on the forum, or explain in detail exactly what you are trying to accomplish.

1. Is it possible to integrate this with asp.net or php web applications?

I don't do php myself, but I know plenty of folks on the forum do. Do a search for PHP and you'll see a number of threads.

2. Is there an eBook or tutorials for learning the use of this tool?

AutoIt as a language is growing far too quickly to be able to capture it in an eBook. It has been tried in the past, and the materials are outdated almost from the moment they are published. Your best resource for learning the language is this forum. Read through the wiki, which has tutorials and programs such as AutoIT 1-2-3 to help you learn, and ask questions in the General Help and Support forum. The forum has hundreds of members dedicated to helping others improve their scripts and applications.

3. Which language do I need to be pro efficient in while making use of this tool?

AutoIt has a syntax much like BASIC, but we have developers come from many different languages. AutoIt's ease of use, and the power of the language to control most anything in a Windows environment, make it a perfect tool to integrate into your workflow, regardless of your main language. For some of us (myself included) there have been times in our careers where AutoIt was our main language.

[SmOke_N]

As far as your scraping etc request.

Look at the _IE.au3 udf in the help file, you can also search for FF.au3 and Chrome in the example script forum for firefox/chrome examples.

And for query 1:

https://www.google.com/search?q=aucgi%20site%3Ahttp%3A%2F%2Fautoitscript.com%2Fforum&rct=j

I'm also moving this post to the correct forum (General Help), you'll get more replies that was as well.

[MikahS]

I learned everything about AutoIt that I know from the forum and helpfile found here

[andrewz]

I made you a little script to automate the thread title extraction:

#include <Inet.au3>
#include <Array.au3>
#include <String.au3>
#include <AutoItConstants.au3>
#include <MsgBoxConstants.au3>

Global $url = "http://www.autoitscript.com/forum/topic/166220-how-can-i-automate-webpage-extraction/"
Global $content = _INetGetSource($url)
Global $string_A = _StringBetween($content, '<title>', '</title>')

$title_A = StringReplace($string_A[0]," - General Help and Support - AutoIt Forums","")

MsgBox(0,"Thread Title",$title_A)

You can replace the url with an inputbox so you can enter any topic url.

best regards,

Andrewz

 

EDIT: Regarding your questions:

1. Is it possible to integrate this with asp.net or php web applications?

    - Hmm yeah probably, but you have to have this script running on your

      computer and activate it with the php application.

2. Is there an eBook or tutorials for learning the use of this tool?

    - This forum + online tutorials should be enought. Use Google...

3. Which language do I need to be pro efficient in while making use of this tool?

    - AutoIT, but no need to be pro. Aswell as php to implent this script

      into your web application.

Dunno how to implent it into php tho, but a few years back when I was interested

in blackhat stuff I came across quite a few web based crypters, where u can upload

ur file and it downloads it into the server and the programm running on it then crypts

it. Then again uploads the crypted file.

Edited December 16, 2014 by andrewz

[niteshsingh]

Thanks everyone (JLogan3o13, SmOke_N, MikahS, andrewz) for providing the useful information.

So the conclusion is to do some more research.

Anyways, I enjoy research.

[MikahS]

Pleasure is ours.