Sign in to follow this  
Followers 0
cal

FF.au3 and Mozrepl have become very slow for me.

10 posts in this topic

I've been using this UDF for awhile now and its always worked great. I just doing some basic reading of the html of a page.

For some reason things have gotten very slow. It still works but it takes a good 10 seconds to connect to Mozrepl and another 10 to read the source of a page. Unfortunately my entire computer changed in between working normally and now. I had a motherboard fry on another system so the one I was using was rebuilt and my main box is new. So XP to Win7, new Hardware, Firefox changed to 3.6, autoit changed versions and so forth.

Searching does not turn up much. So I think this slowness is just me. But I have no idea where to look. I have a couple scripts that use this and they both do the same thing.

The FF.au3 that I'm using is what I think is current, its 0.6.0.0b-5.

I made up the the following code to show what I mean. It just loads and stays resident.

Hotkey 1 to activate.

Hotkey Esc to exit.

It will connect to Mozrepl and show a mesgbox after connection. It then reads the source of the current page and shows it in a mesgbox.

On my system this takes a good 10 seconds to connect and another 10 to read and display the contents of the page.

I've just tried this same test code on my XP box that was rebuilt. The speed is normal.

I've also just installed autoit and tried it on my netbook. Its win7 starter. The speed is normal.

Does anyone have any ideas as to what might be going on to cause this slowness? This machine is far faster then both the others.

to run this source you will need both FF.au3 and Mozrepl installed.

$ver = 'ver 9.1'
#Tidy_Parameters = /Skip_EndRegion_Comment

#include <FF.au3>


If Not @Compiled Then Opt("TrayIconDebug", 1) ;0=no info, 1=debug line info
Opt("MouseCoordMode", 0) ;1=absolute, 0=relative
Opt("WinTitleMatchMode", 2) ;1=start, 2=subStr, 3=exact, 4=advanced
FileChangeDir(@ScriptDir)


; ensure this copy is the running one.
$g_szVersion = "Template Script"
If WinExists($g_szVersion) Then WinClose($g_szVersion)
AutoItWinSetTitle($g_szVersion)

HotKeySet("{ESC}", "Terminate")
Func Terminate()
    Exit 0
EndFunc   ;==>Terminate


HotKeySet('1', 'test')


While 1
    sleep(30000)
Wend

Func test()
    $t = _FFConnect()
    If $t = 0 Then MsgBox(0,'Line ' & @ScriptLineNumber, '---' & 'error is ' & @error & '---')
    If $t <> 0 then MsgBox(0,'Line ' & @ScriptLineNumber, '---' & 'no error connecting to mozrepl' & '---')

    $web = _FFReadHTML("html")

    MsgBox(0, 'web',$web )

EndFunc   ;==>test

Share this post


Link to post
Share on other sites



I've been using this UDF for awhile now and its always worked great. I just doing some basic reading of the html of a page.

For some reason things have gotten very slow. It still works but it takes a good 10 seconds to connect to Mozrepl and another 10 to read the source of a page. Unfortunately my entire computer changed in between working normally and now. I had a motherboard fry on another system so the one I was using was rebuilt and my main box is new. So XP to Win7, new Hardware, Firefox changed to 3.6, autoit changed versions and so forth.

Searching does not turn up much. So I think this slowness is just me. But I have no idea where to look. I have a couple scripts that use this and they both do the same thing.

The FF.au3 that I'm using is what I think is current, its 0.6.0.0b-5.

I made up the the following code to show what I mean. It just loads and stays resident.

Hotkey 1 to activate.

Hotkey Esc to exit.

It will connect to Mozrepl and show a mesgbox after connection. It then reads the source of the current page and shows it in a mesgbox.

On my system this takes a good 10 seconds to connect and another 10 to read and display the contents of the page.

I've just tried this same test code on my XP box that was rebuilt. The speed is normal.

I've also just installed autoit and tried it on my netbook. Its win7 starter. The speed is normal.

Does anyone have any ideas as to what might be going on to cause this slowness? This machine is far faster then both the others.

to run this source you will need both FF.au3 and Mozrepl installed.

$ver = 'ver 9.1'
#Tidy_Parameters = /Skip_EndRegion_Comment

#include <FF.au3>


If Not @Compiled Then Opt("TrayIconDebug", 1) ;0=no info, 1=debug line info
Opt("MouseCoordMode", 0) ;1=absolute, 0=relative
Opt("WinTitleMatchMode", 2) ;1=start, 2=subStr, 3=exact, 4=advanced
FileChangeDir(@ScriptDir)


; ensure this copy is the running one.
$g_szVersion = "Template Script"
If WinExists($g_szVersion) Then WinClose($g_szVersion)
AutoItWinSetTitle($g_szVersion)

HotKeySet("{ESC}", "Terminate")
Func Terminate()
    Exit 0
EndFunc   ;==>Terminate


HotKeySet('1', 'test')


While 1
    sleep(30000)
Wend

Func test()
    $t = _FFConnect()
    If $t = 0 Then MsgBox(0,'Line ' & @ScriptLineNumber, '---' & 'error is ' & @error & '---')
    If $t <> 0 then MsgBox(0,'Line ' & @ScriptLineNumber, '---' & 'no error connecting to mozrepl' & '---')

    $web = _FFReadHTML("html")

    MsgBox(0, 'web',$web )

EndFunc   ;==>test

Takes it at the first call of test() 10 secs, too?

The problem of your script is: Every time you calling test() a you made a new connection to FireFox / MozRepl, that slows down your script extreme. It's better you put the _FFConnect at the start or your script.

The second "problem": _FFReadHTML is very slow. If you would read only some informations from the page it's better to use _FFXPath.

Share this post


Link to post
Share on other sites

The actual code is more involved this this. It looks for wait times on pages like deposit/rapid and a few others and then sets timers. Another pulls from tvtomb and gives me a summery of ep titles and descriptions in a format I can paste directly into the memo field on my palm as well as exporting to a text file in the format I want. However If I had posted that, no one would have been able to use it to test as its kinda unique to my requirements.

Yes under normal usage I would just make one connection and do various things until done. I was not aware that _FFXPath is faster then reading the HTML. I've not used it. I'll look into that. Thanks.

The point of this code is just to test and time the connections. Something I can use to find the real problem and to compare between machines, something that anyone can run. The test code itself is not a problem as its fine on my other machines. Both the other machines make the initial connection in 1 second and the read in another 1. My main machine can take anywhere from 10 or more seconds per part. Yet this machine is far faster then the others so something is not right.

I hope that someone might be able to point out areas that I need to look at to fix the issue. Perhaps some known situation that causes this sort of delay in talking to mozrepl.

Its not addons or firewall. This one machine behaves the same with all addons other then the mozrepl disabled. Same for the firewall. Same for Antivirus.

Could it be something in the win7 64 bit. Both my other machines are 32 bit. XP is 32 and I believe win7 starter only comes in 32 bit. I don't have another 64 bit machine to test on. But considering I can't find any info on this I doubt thats the problem or a lot more people would be asking about it. Still, if someone wants to test this on another win7 64 bit machine. I'd be grateful to rule that out.

I don't see separate versions of autoit for 32 and 64 bit. Just a statement on the download page that says "AutoIt Full Installation. Includes x86 and x64 components". Is there something I should be doing differently on win7? I don't have any problems with other scripts so its not something I thought of till just now.

Interesting. I've just noticed that there is a "Run Script" and a "Run Script (x86)" on the context menu. They both run but both take the same time.

OK. I've added some timers. My main machine is consistent. 10 seconds for each part. Both the XP and the netbook take on ave 1.5 to connect and 0.5 to read.

$ver = 'ver 9.1'
#Tidy_Parameters = /Skip_EndRegion_Comment

#include <FF.au3>


If Not @Compiled Then Opt("TrayIconDebug", 1) ;0=no info, 1=debug line info
Opt("MouseCoordMode", 0) ;1=absolute, 0=relative
Opt("WinTitleMatchMode", 2) ;1=start, 2=subStr, 3=exact, 4=advanced
FileChangeDir(@ScriptDir)


; ensure this copy is the running one.
$g_szVersion = "Template Script"
If WinExists($g_szVersion) Then WinClose($g_szVersion)
AutoItWinSetTitle($g_szVersion)

HotKeySet("{ESC}", "Terminate")
Func Terminate()
    Exit 0
EndFunc   ;==>Terminate


HotKeySet('1', 'test')


While 1
    sleep(30000)
Wend

Func test()

    $begin = TimerInit()
    $t = _FFConnect()
    $dif1 = TimerDiff($begin)

    If $t = 0 Then MsgBox(0,'Line ' & @ScriptLineNumber, '---' & 'error is ' & @error & '---')
    If $t <> 0 then MsgBox(0,'Line ' & @ScriptLineNumber, '---' & 'no error connecting to mozrepl' & '---')

    $begin = TimerInit()
    $web = _FFReadHTML("html")
    $dif2 = TimerDiff($begin)

    MsgBox(0, 'web',$web )

    MsgBox(0, 'Time Taken','Connect:  ' & Round($dif1/1000,1) & @CRLF & 'Read:     ' & Round($dif2/1000,1))

EndFunc   ;==>test

Share this post


Link to post
Share on other sites

I can here only test it on WinXP and Vista 32. On booth it takes 1-2ms to connect and the same this to read this page here.

If you start your example from your editor on your "slow" machine: what is the output in the editor-console?

Share this post


Link to post
Share on other sites

Thats an interesting idea.

Here is the first bit from the Console.

>"C:\Program Files (x86)\AutoIt3\SciTE\AutoIt3Wrapper\AutoIt3Wrapper.exe" /run /prod /ErrorStdOut /in "C:\Users\Calvin\Desktop\test FF and mozrepl.au3" /autoit3dir "C:\Program Files (x86)\AutoIt3" /UserParams

+>21:21:01 Starting AutoIt3Wrapper v.2.0.1.22 Environment(Language:0409 Keyboard:00000409 OS:WIN_7/ CPU:X64 OS:X64)

>Running AU3Check (1.54.19.0) from:C:\Program Files (x86)\AutoIt3

+>21:21:01 AU3Check ended.rc:0

>Running:(3.3.4.0):C:\Program Files (x86)\AutoIt3\autoit3_x64.exe "C:\Users\Calvin\Desktop\test FF and mozrepl.au3"

_FFConnect: OS: WIN_7 WIN32_NT 7600

_FFConnect: AutoIt: 3.3.4.0

_FFConnect: FF.au3: 0.6.0.0b-5

_FFConnect: IP: 127.0.0.1

_FFConnect: Port: 4242

_FFConnect: Delay: 450ms

_FFConnect ==> General Error: Ping Error: 1

_FFConnect: Socket: 616

__FFSend: try{window.content.top.document.browserDOMWindow}catch(e){'_FFCmd_Err';};

__FFRecv:

__FFSend: try{navigator.userAgent}catch(e){'_FFCmd_Err';};

__FFRecv: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6

_FFConnect: Browser: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6

__FFSendJavascripts: Sending functions to FireFox .......... done

__FFSend: try{window.content.top.document.documentElement.innerHTML;}catch(e){'_FFCmd_Err';};

__FFRecv: <head>

It then continues with the content returned from the web page.

and the last lines. I forgot to exit hence the time.

+>21:29:38 AutoIT3.exe ended.rc:0

>Exit code: 0 Time: 519.119

But it does exit cleanly.

It does not mean much to me. But I do see a ping error to myself. Lets see what happens when I try it on my netbook. And running from a share at that.

>Running:(3.3.4.0):C:\Program Files\AutoIt3\autoit3.exe "\\TOWER\Share 3\000\test FF and mozrepl.au3"

_FFConnect: OS: WIN_7 WIN32_NT 7600

_FFConnect: AutoIt: 3.3.4.0

_FFConnect: FF.au3: 0.6.0.0b-5

_FFConnect: IP: 127.0.0.1

_FFConnect: Port: 4242

_FFConnect: Delay: 2ms

_FFConnect: Socket: 600

__FFSend: try{window.content.top.document.browserDOMWindow}catch(e){'_FFCmd_Err';};

__FFRecv:

__FFSend: try{navigator.userAgent}catch(e){'_FFCmd_Err';};

__FFRecv: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6

_FFConnect: Browser: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6

__FFSendJavascripts: Sending functions to FireFox .......... done

__FFSend: try{window.content.top.document.documentElement.innerHTML;}catch(e){'_FFCmd_Err';};

__FFRecv: <head>

That looks better. no Ping error. And a big difference in the delay. only 2ms vs 450ms.

Both were trying to read the same web page. I've tried another on the slow machine and I get that same ping error. the same delay of 450ms and the same socket. I'm not sure what this means but I'd say this ping error is involved in some way.

Share this post


Link to post
Share on other sites

The ping in the FF.au3 is used for the timing in the send and receive functions, because you can connect to a FireFox on an other machine over a network, too. If it fails the delay would be set to a high default value, like in your example.

If you set the var $_FF_CON_DELAY after _FFConnect to 2ms, have you then the same problem on your slow machine, too? e.g.:

_FFConnect()
$_FF_CON_DELAY = 2
;...

Is ping generally working on this machine?

Share this post


Link to post
Share on other sites

Now that is very interesting.

I was unaware of it but yes. This machine can't ping itself at all. It can ping the other 2 machines. The other machines that work properly can both ping this one and themselves fine. I've never seen a machine that can't ping itself before.

I added your line as directed. And yes. That made a big difference. The connect takes the same time but the read is now consistently a normal 0.3 seconds. I tried adding it before the connect as well but that is just the same as if its not even there. I assume its a var that can be set after the connection to control things.

The ping error is still in the console for this machine. It looks like the "can't ping myself" may be the heart of my issue. I'll see what I can do to find the cause of that and fix it.

Thanks.

Share this post


Link to post
Share on other sites

I have confirmed the "can't ping myself" is the real issue here. Turns out its Zonealarm at fault. I had not though it was as turning it off did not fix the issue. But it seems zonealarm does not fully shut down when you tell it to. You have to uninstall it. as soon as I did that I was able to ping myself and FF.au3 and Mozrepl started working at normal speeds.

Reinstalling zonealarm brings the issue back. Yet no settings are different on this machine vs my other machines that work properly. I will either find a fix for zonealarm or dump it.

Thanks for your help Stilgar, your questions, suggestions and comments helped point the way to the real issue.

Thankyou.

Share this post


Link to post
Share on other sites

And lastly I just want to say this has been solved for anyone who comes across this and has the same issue. I removed Zonealarm from this machine and went with "PC Tools Firewall Plus". Its working great and FF.au3 is working great as well.

Thanks again Stilgar.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0