Arclite86

check if url exist function?

25 posts in this topic

#1 ·  Posted (edited)

i am using this script that I found on this forum 

in list1 i have about 20 google.nl ulrs to test if it doesnt skip a good url .only 15 urls pass the test so its only 75% accurate

 

;Timeout values in milliseconds
$ResolveTimeout = 500
$ConnectTimeout = 500
$SendTimeout = 500
$ReceiveTimeout = 500
;
Global $oMyError = ObjEvent("AutoIt.Error", "MyErrFunc")
$oHttpRequest = ObjCreate("MSXML2.ServerXMLHTTP")
$sTestUrl = $asplit3
;
With $oHttpRequest
   .SetTimeouts ($ResolveTimeout, $ConnectTimeout, $SendTimeout, $ReceiveTimeout)
   .Open ("GET", $sTestUrl)
   .Send
   Select
      Case .Status = 200;No problem!
        ConsoleWrite($sTestUrl & " is a valid, available URL" & @CR)
_GUICtrlListBox_AddString($List2, GUICtrlRead($list1))
      Case .Status = 404;Not found
        ConsoleWrite($sTestUrl & " could not be found (404 Error)" & @CR)
      Case Else;Some other problem
        ConsoleWrite("An unexpected HTTP Status value was returned: " & .Status & @CR)
        sleep(1000)
        clean()
   EndSelect
EndWith

clean() = the samescript to test it a second time

 

does anybody have a better alternative to test an ur

Edited by Arclite86

Share this post


Link to post
Share on other sites



Why not just use a simple ping script?


Get Scite to add a popup when you use a 3rd party UDF -> http://www.autoitscript.com/autoit3/scite/docs/SciTE4AutoIt3/user-calltip-manager.html

Share this post


Link to post
Share on other sites

i have no idea what you mean by that, can you give me an example for what you mean

Share this post


Link to post
Share on other sites

#5 ·  Posted (edited)

Yes, exactly as computergroove pointed out. This is how I use in one of my apps to check for general internet connectivity. It will work the same to check for the existence of a url, if it doesn't exist the ping will fail at the dns level.
 

Func __checkConn()
    Local $ping = 0
    $ping += Ping("google.com", 700)
    $ping += Ping("microsoft.com", 700)
    $ping += Ping("mysite.org", 700)
    If Not $ping Then Return 0
    Return 1
EndFunc

 

Edited by MagnumXL
format

Share this post


Link to post
Share on other sites

i have test it

;$websiteread = read from list

Local $iPing = Ping($websiteread, 1000)

    If $iPing Then ; If a value greater than 0 was returned then display the following message.
       ConsoleWrite("exist! The roundtrip-time took: " & $iPing & "ms." & @CR)
          
    Else
       ConsoleWrite("Not Exist, An error occurred with @error value of: " & @error & @CR)
;~     MsgBox(0,"tes urlt","" & $websiteread)
    EndIf

so every time it did not pass the ping I would get a mgsbox to test it my self , and there where a allot urls that actual did work

so its far from accurate

Share this post


Link to post
Share on other sites

Try...

ConsoleWrite( __checkConn("https://www.autoitscript.com/forum/bullshitpage") & @LF)
ConsoleWrite( __checkConn("https://www.autoitscript.com/forum/") & @LF)
ConsoleWrite( __checkConn("http://www.narcolepticmonkeytumor.com") & @LF)

Func __checkConn($url)
    Return (StringLen(InetRead($url, 1)) > 0)
EndFunc

 


AutoIt Absolute Beginners    Require a serial    Pause Script    Video Tutorials by Morthawt

Monkey's are, like, natures humans.

Share this post


Link to post
Share on other sites

i dont know if i used your code correct:

it keeps saying false 3 times in the consolewrite

#include <ButtonConstants.au3>
#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <StaticConstants.au3>
#include <WindowsConstants.au3>
#include <IE.au3>
#include <String.au3>

#include <MsgBoxConstants.au3>
#include <ButtonConstants.au3>
#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <GUIListBox.au3>
#include <WindowsConstants.au3>
#include <FF V0.6.0.1b-14.au3>

#include <staticConstants.au3>
#include <IE.au3>
#include <String.au3>
#include <Excel.au3>
#include <Array.au3>
#include <FileConstants.au3>


#include <ButtonConstants.au3>
#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <StaticConstants.au3>
#include <WindowsConstants.au3>

#Region ### START Koda GUI section ### Form=
$Form1_1_1_1 = GUICreate("program", 1161, 578, 239, 113)
$List1 = GUICtrlCreateList("", 896, 24, 225, 487)
$List2 = GUICtrlCreateList("", 664, 24, 225, 487)
$test2 = GUICtrlCreateButton("test2", 168, 520, 75, 25)
#EndRegion ### END Koda GUI section ###


Func __checkConn(   $read123)
    $read123 = GUICtrlRead($list1)


    Return (StringLen(InetRead( $read123, 1)) > 0)

    $iIndex += 1

            If $iIndex = _GUICtrlListBox_GetCount($List1) Then $iIndex = 0
            _GUICtrlListBox_SetCurSel($List1, $iIndex)
EndFunc

While 1
    $nMsg = GUIGetMsg()
    Switch $nMsg
        Case $GUI_EVENT_CLOSE
case $test2

ConsoleWrite( __checkConn("https://www.autoitscript.com/forum/bullshitpage") & @LF)
ConsoleWrite( __checkConn("https://www.autoitscript.com/forum/") & @LF)
ConsoleWrite( __checkConn("http://www.narcolepticmonkeytumor.com") & @LF)

EndSwitch
WEnd

btw i have testet the 2 previous scripts:

the first script (old autoit script):      with 42(existing urls) it only had 13 so thats:        30,95%. accurate

the ping script:                                 with 42(existing urls) it had 28  so thats:              66,67%.. accurate

 

it is getting better but still far from good and acceptable

 

Share this post


Link to post
Share on other sites
#include <ButtonConstants.au3>
#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <StaticConstants.au3>
#include <WindowsConstants.au3>
#include <IE.au3>
#include <String.au3>

#include <MsgBoxConstants.au3>
#include <ButtonConstants.au3>
#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <GUIListBox.au3>
#include <WindowsConstants.au3>
;#include <FF V0.6.0.1b-14.au3>

#include <staticConstants.au3>
#include <IE.au3>
#include <String.au3>
#include <Excel.au3>
#include <Array.au3>
#include <FileConstants.au3>


#include <ButtonConstants.au3>
#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <StaticConstants.au3>
#include <WindowsConstants.au3>

#Region ### START Koda GUI section ### Form=
$Form1_1_1_1 = GUICreate("program", 1161, 578, 239, 113)
$iInput = GUICtrlCreateInput("url to check", 10, 10, 300)
$test2 = GUICtrlCreateButton("test2", 168, 520, 75, 25)
#EndRegion ### END Koda GUI section ###

GUISetState()

Func __checkConn($read123)
    ;$read123 = GUICtrlRead($list1)


    Return (StringLen(InetRead($read123, 1)) > 0)

    ;$iIndex += 1

    ;If $iIndex = _GUICtrlListBox_GetCount($List1) Then $iIndex = 0
    ;_GUICtrlListBox_SetCurSel($List1, $iIndex)
EndFunc   ;==>__checkConn

While 1
    $nMsg = GUIGetMsg()
    Switch $nMsg
        Case $GUI_EVENT_CLOSE
            Exit
        Case $test2

            ConsoleWrite(__checkConn(GUICtrlRead($iInput)) & @LF)
            ;ConsoleWrite(__checkConn("https://www.autoitscript.com/forum/") & @LF)
            ;ConsoleWrite(__checkConn("http://www.narcolepticmonkeytumor.com") & @LF)

    EndSwitch
WEnd

 


AutoIt Absolute Beginners    Require a serial    Pause Script    Video Tutorials by Morthawt

Monkey's are, like, natures humans.

Share this post


Link to post
Share on other sites

i have tested it only passed 21 urls from the 42  so its only 50% accurate

there has to be a better accurate way

is there a function when opening the url in IE and checking with a function if the page has succefully openend and not a 404 error

is there a function  for that ?

Share this post


Link to post
Share on other sites

#14 ·  Posted (edited)

random urls i used:

http://www.alfaisaliahhotels.com/
imperial.ac.uk
cas.org
address.com
ird.fr
http://www.iprimus.com.au/
http://www.actua.com/
mobilenations.com
http://niagara.edu/
http://www.qimrberghofer.edu.au/
http://www.healthlawyers.org
http://www.cdc.gov/
http://www.apextechservices.com/
liferay.com
ualr.edu/
sanctuarysoil.com
detroitchamber.com
live.com
ucla.edu
kqed.org
hsph.harvard.edu
yahoo.com
imperial.ac.uk
bizrate.com
blacklakeorganic.net
jdevents.com
nationaljournal.com
consumeraffairs.com
gmail.com
elex.com
delacyexecutive.co.uk
ivr.org.uk
ftc.gov
hemmings.com
mlive.com
mlive.com
imperial.ac.uk
.ac.uk
latimes.com
bizrate.com
mlive.com
rice.edu

the urls that dit not past the ping method

http://www.alfaisaliahhotels.com/
imperial.ac.uk
cas.org
address.com
ird.fr
http://www.iprimus.com.au/
http://www.actua.com/
mobilenations.com
http://niagara.edu/
http://www.qimrberghofer.edu.au/
http://www.healthlawyers.org
http://www.cdc.gov/
http://www.apextechservices.com/
liferay.com
ualr.edu/

it seems that it are always the same urls that dont pas the test

Edited by Arclite86

Share this post


Link to post
Share on other sites

#15 ·  Posted (edited)

#include <Array.au3>
#include <File.au3>

Global $aUrls, $PingTimeout = 1000

_FileReadToArray("urls.txt", $aUrls)

For $i = 1 To $aUrls[0]
    If _UrlExists($aUrls[$i]) Then
        ConsoleWrite("Exists: " & $aUrls[$i] & @LF)
        ContinueLoop
    EndIf
    ConsoleWrite("Not Exists Or No Response in " & $PingTimeout / 1000 & " Seconds: " & $aUrls[$i] & @LF)
Next

Func _UrlExists($url)
    If (StringLen(InetRead($url, 1)) > 0) Then Return 1
    If Ping($url, $PingTimeout) Then Return 2
    Return 0
EndFunc   ;==>__checkConn
Exists: http://www.alfaisaliahhotels.com/
Exists: imperial.ac.uk
Not Exists Or No Response in 1 Seconds: cas.org
Not Exists Or No Response in 1 Seconds: address.com
Not Exists Or No Response in 1 Seconds: ird.fr
Exists: http://www.iprimus.com.au/
Exists: http://www.actua.com/
Not Exists Or No Response in 1 Seconds: mobilenations.com
Exists: http://niagara.edu/
Exists: http://www.qimrberghofer.edu.au/
Exists: http://www.healthlawyers.org
Exists: http://www.cdc.gov/
Exists: http://www.apextechservices.com/
Not Exists Or No Response in 1 Seconds: liferay.com
Not Exists Or No Response in 1 Seconds: ualr.edu/
Exists: sanctuarysoil.com
Exists: detroitchamber.com
Not Exists Or No Response in 1 Seconds: live.com
Exists: ucla.edu
Exists: kqed.org
Exists: hsph.harvard.edu
Exists: yahoo.com
Exists: imperial.ac.uk
Exists: bizrate.com
Exists: blacklakeorganic.net
Exists: jdevents.com
Exists: nationaljournal.com
Exists: consumeraffairs.com
Exists: gmail.com
Exists: elex.com
Exists: delacyexecutive.co.uk
Exists: ivr.org.uk
Exists: ftc.gov
Exists: hemmings.com
Exists: mlive.com
Exists: mlive.com
Exists: imperial.ac.uk
Not Exists Or No Response in 1 Seconds: .ac.uk
Exists: latimes.com
Exists: bizrate.com
Exists: mlive.com
Exists: rice.edu

EDIT:

I cannot be bothered going through all those that failed, but I can tell you that ird.fr and cas.org do not exist, their www. counterparts do though.

So for something like this to work as you expect you need to build on the function and add some of your own logic.

Edited by JohnOne

AutoIt Absolute Beginners    Require a serial    Pause Script    Video Tutorials by Morthawt

Monkey's are, like, natures humans.

Share this post


Link to post
Share on other sites

arclite86,

Try this.  It reformats the url and works for all except ".ac.uk" which appears to be a typo.  I couldn't work out the regexp for the replacement so I used the goofy looking case stmt for the reformat.

#include <Array.au3>
#include <File.au3>

Global $aUrls, $PingTimeout = 1000

_FileReadToArray(@ScriptDir & "\urls.txt", $aUrls)

For $i = 1 To UBound($aUrls) - 1
    Switch StringLeft($aUrls[$i], 3)
        Case 'www'
            $aUrls[$i] = 'http://' & $aUrls[$i]
        Case 'htt'
            ContinueLoop
        Case Else
            $aUrls[$i] = 'http://www.' & $aUrls[$i]
    EndSwitch
Next

For $i = 1 To $aUrls[0]
    If _UrlExists($aUrls[$i]) Then
        ConsoleWrite("Exists: " & $aUrls[$i] & @LF)
        ContinueLoop
    EndIf
    ConsoleWrite("Not Exists Or No Response in " & $PingTimeout / 1000 & " Seconds: " & $aUrls[$i] & @LF)
Next

Func _UrlExists($url)
    If (StringLen(InetRead($url, 1)) > 0) Then Return 1
    If Ping($url, $PingTimeout) Then Return 2
    Return 0
EndFunc   ;==>_UrlExists

kylomas

1 person likes this

Forum Rules         Procedure for posting code

"I like pigs.  Dogs look up to us.  Cats look down on us.  Pigs treat us as equals."

- Sir Winston Churchill

Share this post


Link to post
Share on other sites
#include <Array.au3>
#include <File.au3>

Global $aUrls, $PingTimeout = 1000

_FileReadToArray("urls.txt", $aUrls)

For $i = 1 To $aUrls[0]
    If _UrlExists($aUrls[$i]) Then
        ConsoleWrite("Exists: " & $aUrls[$i] & @LF)
        ContinueLoop
    EndIf
    ConsoleWrite("Not Exists Or No Response in " & $PingTimeout / 1000 & " Seconds: " & $aUrls[$i] & @LF)
Next

Func _UrlExists($url)
    If (StringLen(InetRead($url, 1)) > 0) Then Return 1
    If Ping($url, $PingTimeout) Then Return 2
    Return 0
EndFunc   ;==>__checkConn
Exists: http://www.alfaisaliahhotels.com/
Exists: imperial.ac.uk
Not Exists Or No Response in 1 Seconds: cas.org
Not Exists Or No Response in 1 Seconds: address.com
Not Exists Or No Response in 1 Seconds: ird.fr
Exists: http://www.iprimus.com.au/
Exists: http://www.actua.com/
Not Exists Or No Response in 1 Seconds: mobilenations.com
Exists: http://niagara.edu/
Exists: http://www.qimrberghofer.edu.au/
Exists: http://www.healthlawyers.org
Exists: http://www.cdc.gov/
Exists: http://www.apextechservices.com/
Not Exists Or No Response in 1 Seconds: liferay.com
Not Exists Or No Response in 1 Seconds: ualr.edu/
Exists: sanctuarysoil.com
Exists: detroitchamber.com
Not Exists Or No Response in 1 Seconds: live.com
Exists: ucla.edu
Exists: kqed.org
Exists: hsph.harvard.edu
Exists: yahoo.com
Exists: imperial.ac.uk
Exists: bizrate.com
Exists: blacklakeorganic.net
Exists: jdevents.com
Exists: nationaljournal.com
Exists: consumeraffairs.com
Exists: gmail.com
Exists: elex.com
Exists: delacyexecutive.co.uk
Exists: ivr.org.uk
Exists: ftc.gov
Exists: hemmings.com
Exists: mlive.com
Exists: mlive.com
Exists: imperial.ac.uk
Not Exists Or No Response in 1 Seconds: .ac.uk
Exists: latimes.com
Exists: bizrate.com
Exists: mlive.com
Exists: rice.edu

EDIT:

I cannot be bothered going through all those that failed, but I can tell you that ird.fr and cas.org do not exist, their www. counterparts do though.

So for something like this to work as you expect you need to build on the function and add some of your own logic.

works pretty nice 34 out of 42  80,95% accurate

Share this post


Link to post
Share on other sites

arclite86,

Try this.  It reformats the url and works for all except ".ac.uk" which appears to be a typo.  I couldn't work out the regexp for the replacement so I used the goofy looking case stmt for the reformat.

#include <Array.au3>
#include <File.au3>

Global $aUrls, $PingTimeout = 1000

_FileReadToArray(@ScriptDir & "\urls.txt", $aUrls)

For $i = 1 To UBound($aUrls) - 1
    Switch StringLeft($aUrls[$i], 3)
        Case 'www'
            $aUrls[$i] = 'http://' & $aUrls[$i]
        Case 'htt'
            ContinueLoop
        Case Else
            $aUrls[$i] = 'http://www.' & $aUrls[$i]
    EndSwitch
Next

For $i = 1 To $aUrls[0]
    If _UrlExists($aUrls[$i]) Then
        ConsoleWrite("Exists: " & $aUrls[$i] & @LF)
        ContinueLoop
    EndIf
    ConsoleWrite("Not Exists Or No Response in " & $PingTimeout / 1000 & " Seconds: " & $aUrls[$i] & @LF)
Next

Func _UrlExists($url)
    If (StringLen(InetRead($url, 1)) > 0) Then Return 1
    If Ping($url, $PingTimeout) Then Return 2
    Return 0
EndFunc   ;==>_UrlExists

kylomas

42 out of 42 urls 100% accurate!!!! thank you so much kylomas!!:D:lmao:

Share this post


Link to post
Share on other sites

#19 ·  Posted (edited)

n

Edited by Arclite86

Share this post


Link to post
Share on other sites

arclite86,

Run this...

#include <Array.au3>
#include <File.au3>
#include <date.au3>

Global $aUrls



_FileReadToArray(@ScriptDir & "\urls.txt", $aUrls)

For $i = 1 To UBound($aUrls) - 1
    Switch StringLeft($aUrls[$i], 3)
        Case 'www'
            $aUrls[$i] = 'http://' & $aUrls[$i]
        Case 'htt'

            ContinueLoop

        Case Else
            $aUrls[$i] = 'http://www.' & $aUrls[$i]
    EndSwitch

Next

For $i = 1 To $aUrls[0]
    If _UrlExists($aUrls[$i]) Then
        ConsoleWrite(_Now() & " Exists: " & $aUrls[$i] & @LF)
        ContinueLoop

    EndIf

    ConsoleWrite(_Now() & ' ' & $aUrls[$i] & ' is not reachable' & @LF)
Next

Func _UrlExists($url)
    If (StringLen(InetRead($url, 1)) > 0) Then Return 1
    Return 0
EndFunc   ;==>_UrlExists

Note the timings...

Also, the ping is not necessary...

kylomas


Forum Rules         Procedure for posting code

"I like pigs.  Dogs look up to us.  Cats look down on us.  Pigs treat us as equals."

- Sir Winston Churchill

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now