Sign in to follow this  
Followers 0
niteshsingh

How can I automate Webpage Extraction?

7 posts in this topic

Hello there, 

I'm an ASP.NET web application developer.

I've installed this tool and would like to learn automation possibilities related to web extraction/scrapping.

Please provide suitable resources for the above.

Also here are few more of my queries:

1. Is it possible to integrate this with asp.net or php web applications?

2. Is there an eBook or tutorials for learning the use of this tool?

3. Which language do I need to be pro efficient in while making use of this tool?

 

Looking forward to replies.

Regards,

Nitesh Singh

Share this post


Link to post
Share on other sites



Welcome to the AutoIT forum. In answer to your questions:

 

I've installed this tool and would like to learn automation possibilities related to web extraction/scrapping.

Please provide suitable resources for the above.

 

Very vague. Please do a search on the forum, or explain in detail exactly what you are trying to accomplish.


1. Is it possible to integrate this with asp.net or php web applications?

I don't do php myself, but I know plenty of folks on the forum do. Do a search for PHP and you'll see a number of threads.


2. Is there an eBook or tutorials for learning the use of this tool?

AutoIt as a language is growing far too quickly to be able to capture it in an eBook. It has been tried in the past, and the materials are outdated almost from the moment they are published. Your best resource for learning the language is this forum. Read through the wiki, which has tutorials and programs such as AutoIT 1-2-3 to help you learn, and ask questions in the General Help and Support forum. The forum has hundreds of members dedicated to helping others improve their scripts and applications.


3. Which language do I need to be pro efficient in while making use of this tool?

AutoIt has a syntax much like BASIC, but we have developers come from many different languages. AutoIt's ease of use, and the power of the language to control most anything in a Windows environment, make it a perfect tool to integrate into your workflow, regardless of your main language. For some of us (myself included) there have been times in our careers where AutoIt was our main language.


√-1 2^3 ∑ π, and it was delicious!

Share this post


Link to post
Share on other sites

As far as your scraping etc request.

Look at the _IE.au3 udf in the help file, you can also search for FF.au3 and Chrome in the example script forum for firefox/chrome examples.

And for query 1:

https://www.google.com/search?q=aucgi%20site%3Ahttp%3A%2F%2Fautoitscript.com%2Fforum&rct=j

I'm also moving this post to the correct forum (General Help), you'll get more replies that was as well.


Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer.

Share this post


Link to post
Share on other sites

I learned everything about AutoIt that I know from the forum and helpfile found here


Snips & Scripts


My Snips: graphCPUTemp ~ getENVvars
My Scripts: Short-Order Encrypter - message and file encryption V1.6.1 ~ AuPad - Notepad written entirely in AutoIt V1.9.4

Feel free to use any of my code for your own use.                                                                                                                                                           Forum FAQ

 

Share this post


Link to post
Share on other sites

#5 ·  Posted (edited)

I made you a little script to automate the thread title extraction:

#include <Inet.au3>
#include <Array.au3>
#include <String.au3>
#include <AutoItConstants.au3>
#include <MsgBoxConstants.au3>

Global $url = "http://www.autoitscript.com/forum/topic/166220-how-can-i-automate-webpage-extraction/"
Global $content = _INetGetSource($url)
Global $string_A = _StringBetween($content, '<title>', '</title>')

$title_A = StringReplace($string_A[0]," - General Help and Support - AutoIt Forums","")

MsgBox(0,"Thread Title",$title_A)

You can replace the url with an inputbox so you can enter any topic url.

best regards,

Andrewz

 

EDIT: Regarding your questions:

1. Is it possible to integrate this with asp.net or php web applications?

    - Hmm yeah probably, but you have to have this script running on your

      computer and activate it with the php application.

2. Is there an eBook or tutorials for learning the use of this tool?

    - This forum + online tutorials should be enought. Use Google...

3. Which language do I need to be pro efficient in while making use of this tool?

    - AutoIT, but no need to be pro. Aswell as php to implent this script

      into your web application.

Dunno how to implent it into php tho, but a few years back when I was interested

in blackhat stuff I came across quite a few web based crypters, where u can upload

ur file and it downloads it into the server and the programm running on it then crypts

it. Then again uploads the crypted file.

Edited by andrewz

Share this post


Link to post
Share on other sites

Thanks everyone (JLogan3o13, SmOke_N, MikahS, andrewz) for providing the useful information.

So the conclusion is to do some more research.

Anyways, I enjoy research.

Share this post


Link to post
Share on other sites

Pleasure is ours. ;)


Snips & Scripts


My Snips: graphCPUTemp ~ getENVvars
My Scripts: Short-Order Encrypter - message and file encryption V1.6.1 ~ AuPad - Notepad written entirely in AutoIt V1.9.4

Feel free to use any of my code for your own use.                                                                                                                                                           Forum FAQ

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0