Not sure how to go about doing this

[daldous]

So, the idea is to make a script that will browse a few webpages, click on a few links and then download the page and then close the Firefox tab.

Specifically, I want to be able to start a script, have it open firefox, go to an imageboard like /hr/ in 4chan.org, open every thread on every page by middle clicking on the reply button (to open a new tab - this way, each thread = new tab).

Then, it waits until all the tabs are loaded, goes to each page, right clicks, clicks DownTHEMAll, starts dTA, and closes the tab.

Now, the tricky parts here are the following:

- clicking on /hr/

- clicking on every Reply link (to load the entire thread) across all of the pages on the board

- clicking the "next" button to go to the next page in the board

-

Anyone wanna give me pointers/ideas/neat things that would be of help whilst making a script to poke around webpages? Would it be easier in Internet Explorer as oposed to Firefox?

I'm not looking for code - I'm a newbie but I'm not disabled - just general pointers.

Thanks

[JSThePatriot]

So, the idea is to make a script that will browse a few webpages, click on a few links and then download the page and then close the Firefox tab.

Specifically, I want to be able to start a script, have it open firefox, go to an imageboard like /hr/ in 4chan.org, open every thread on every page by middle clicking on the reply button (to open a new tab - this way, each thread = new tab).

Then, it waits until all the tabs are loaded, goes to each page, right clicks, clicks DownTHEMAll, starts dTA, and closes the tab.

Now, the tricky parts here are the following:

- clicking on /hr/

- clicking on every Reply link (to load the entire thread) across all of the pages on the board

- clicking the "next" button to go to the next page in the board

-

Anyone wanna give me pointers/ideas/neat things that would be of help whilst making a script to poke around webpages? Would it be easier in Internet Explorer as oposed to Firefox?

I'm not looking for code - I'm a newbie but I'm not disabled - just general pointers.

Thanks

I would have to say that you may want to go IE due to the IE.au3 Automation Library. It should be in the UDF's section in the helpfile.

JS

[daldous]

I would have to say that you may want to go IE due to the IE.au3 Automation Library. It should be in the UDF's section in the helpfile.

JS

No can do chief. I need to use Firefox to do this and I just want to know how to click on a hyperlink. >.>

[ame1011]

No can do chief. I need to use Firefox to do this and I just want to know how to click on a hyperlink. >.>

As much as I hate IE, I use it for all of my autoit coding purposes. Firefox is for my regular web browsing. If you're dead set on using firefox, i would reccomend going about your problem in a different manner to simplify things.

Specifically, take a look at the url of the forum. Forums usually tend to have php variables at the end of the url to specify which page you wish to load.

Ex:

www.forum.com?board=1&postnum=12&page=1

just use that url to cycle through all the boards, postNums and pages.

Edited September 30, 2006 by ame1011

[daldous]

As much as I hate IE, I use it for all of my autoit coding purposes. Firefox is for my regular web browsing. If you're dead set on using firefox, i would reccomend going about your problem in a different manner to simplify things.

Specifically, take a look at the url of the forum. Forums usually tend to have php variables at the end of the url to specify which page you wish to load.

Ex:

www.forum.com?board=1&postnum=12&page=1

just use that url to cycle through all the boards, postNums and pages.

Thanks ame, but I need to use a firefox extension to perform some tasks on the page so IE is outta the question. I did find the info I needed on this page (hail the forum search function )

http://www.autoitscript.com/forum/index.ph...&hl=firefox

I think I'm set for now. It'll be a pain in the arse to do most anything besides clicking vanilla hyperlinks but I'll tough it out. The idea is to script this in AutoIT first, then to make a Firefox extension. {Though, that'll require a bit of learning and elbowgrease.}

[ame1011]

you may wanna check this out:

http://www.iol.ie/~locka/mozilla/mozilla.htm

[daldous]

Eh, might as well post what I came up with.

expandcollapse popup

; ====> HOW DO I SHOT WEB <=====
; 0. Open firefox, open a chan's board but _MAKE SURE THE LEFT HAND NAVIGATION PANE ISN'T OPEN_ 
;   A good test page would be this: http://orz.4chan.org/wg/imgboard.html
; 1. In the second tab, open google or something random
; 2. MAKE SURE THOSE TWO TABS ARE THE ONLY ONES OPEN. (Cruise Control et al, 2006)
; 1. Run the script and give it time to finish. It'll take a while so do this on an unused computer

; ==> What to modify  if it doesn't work <==
; 0. In the dTaCurrentThread func, I right click on (70, 220), and then left click on the entry for downThemAll at (125, 235)
;   Your downTHEMall entry might be somewhere in the context menu so change this. 
; 1. Change the time multiplier if the script is running too slow (the computer I made this on is very slow) so ...
; 2. The code's pretty well commented so don't be afraid to poke about


HotKeySet("{ESC}", "MyExit"); pressing escape ends the script
$timeMultiplier = 1.000;; make this lower if you find the script waiting too long between steps
Opt("WinTitleMatchMode", 2); matches the firefox window on "Mozilla Firefox" instead of the default webpage-dependant window title

WinActivate("Mozilla Firefox"); partial title matching (see above), so this code is chan independant
OpenAllBoardPages();opens all the pages on the board
Sleep(2000 * $timeMultiplier); allow the pages some residual time to load completely


for $i = 0 to 10; for each of the pages (there are 11 pages / board on most chans)
    Send("{LCTRL down}{f down}{LCTRL up}{f up}"); activates the search bar
    Sleep(200 * $timeMultiplier); delay for searchbar to open
    Send("[Reply]"); searches for a thread
    Sleep(200 * $timeMultiplier)
    
    for $j = 0 to 2; for each thread on the page (10 threads/page on most chans)
        $greenReply = PixelSearch(278, 88, 1007, 743, 0X38D878); 278, 88 - 1007, 743 is the area of firefox where [Reply] shows up
    ; 0x38D878 is the color of the search highlight that firefox uses (above, we search for [Reply] which is a hyperlink
    ; to open the whole thread
        MouseMove($greenReply[0] + 20, $greenReply[1]+10)
        Sleep(200 * $timeMultiplier)
        MouseClick("middle"); opens the thread in a new tab
        Sleep(1000 * $timeMultiplier);
        Send("{LCTRL down}{LSHIFT down}{TAB down}{LCTRL up}{LSHIFT up}{TAB up}"); goes to the tab where the current thread is
        Sleep(1000 * $timeMultiplier)
        dTaCurrentThread()
        MouseClick("left", 1013, 97); closes the thread's tab (all the pictures have been downloaded, so no need to keep it open)
        Sleep(500 * $timeMultiplier);
        Send("{LCTRL down}{TAB down}{LCTRL up}{TAB up}"); goes to the page from which the thread was spawned
        Sleep(500 * $timeMultiplier);
        MouseClick("left", 130, 735); goes to the search bar - to find the next thread
        Send("{ENTER}"); finds the next thread
    Next
    
    MouseClick("left", 1013, 97); close the current page - all 11 of the threads have been dTa'd
    if ($i == 0) Then MouseClick("left", 1013, 97); close the dupe tab
Next

Func OpenAllBoardPages()
    WinActivate("Mozilla Firefox")
    Sleep(300 * $timeMultiplier)
    Send("{LCTRL down}{f down}{LCTRL up}{f up}"); activates the search bar
    Sleep(200 * $timeMultiplier); delay for searchbar to open
    Send("[0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]"); searches for the pages index
    Sleep(200 * $timeMultiplier)
    
    for $i = 0 to 9
        MouseMove(99 + ($i*22), 709); clicks on the [1] and [2] and [3] etc until all the board's pages are open. 
        MouseClick("middle"); spawns a new tab for each board page
        Sleep(700 * $timeMultiplier)
    Next
EndFunc

Func dTaCurrentThread()
    Sleep(1500 * $timeMultiplier)
    MouseMove(70, 220)
    MouseClick("right", 70, 220); just to be safe - the mouse might be moved while the script is running
    Sleep(100 * $timeMultiplier)
    MouseMove(125, 235);
    MouseClick("left", 125, 235); clicks on the downThemAll entry in the main context menu of firefox
    Sleep(2000 * $timeMultiplier); delay for dTa to load
    WinActivate("DownThemAll! - Make your selection")
    Send("{LALT down}{s down}{LALT up}{s up}"); starts the downloads in dTa
    Sleep(4000 * $timeMultiplier)
    WinActivate("Mozilla Firefox")
    Sleep(500 * $timeMultiplier); sleep for firefox to activate
EndFunc

Func MyExit()
    Exit 
EndFunc