Jump to content

check if url exist function?


Recommended Posts

i am using this script that I found on this forum 

in list1 i have about 20 google.nl ulrs to test if it doesnt skip a good url .only 15 urls pass the test so its only 75% accurate

 

;Timeout values in milliseconds
$ResolveTimeout = 500
$ConnectTimeout = 500
$SendTimeout = 500
$ReceiveTimeout = 500
;
Global $oMyError = ObjEvent("AutoIt.Error", "MyErrFunc")
$oHttpRequest = ObjCreate("MSXML2.ServerXMLHTTP")
$sTestUrl = $asplit3
;
With $oHttpRequest
   .SetTimeouts ($ResolveTimeout, $ConnectTimeout, $SendTimeout, $ReceiveTimeout)
   .Open ("GET", $sTestUrl)
   .Send
   Select
      Case .Status = 200;No problem!
        ConsoleWrite($sTestUrl & " is a valid, available URL" & @CR)
_GUICtrlListBox_AddString($List2, GUICtrlRead($list1))
      Case .Status = 404;Not found
        ConsoleWrite($sTestUrl & " could not be found (404 Error)" & @CR)
      Case Else;Some other problem
        ConsoleWrite("An unexpected HTTP Status value was returned: " & .Status & @CR)
        sleep(1000)
        clean()
   EndSelect
EndWith

clean() = the samescript to test it a second time

 

does anybody have a better alternative to test an ur

Edited by Arclite86
Link to comment
Share on other sites

Yes, exactly as computergroove pointed out. This is how I use in one of my apps to check for general internet connectivity. It will work the same to check for the existence of a url, if it doesn't exist the ping will fail at the dns level.
 

Func __checkConn()
    Local $ping = 0
    $ping += Ping("google.com", 700)
    $ping += Ping("microsoft.com", 700)
    $ping += Ping("mysite.org", 700)
    If Not $ping Then Return 0
    Return 1
EndFunc

 

Edited by MagnumXL
format
Link to comment
Share on other sites

i have test it

;$websiteread = read from list

Local $iPing = Ping($websiteread, 1000)

    If $iPing Then ; If a value greater than 0 was returned then display the following message.
       ConsoleWrite("exist! The roundtrip-time took: " & $iPing & "ms." & @CR)
          
    Else
       ConsoleWrite("Not Exist, An error occurred with @error value of: " & @error & @CR)
;~     MsgBox(0,"tes urlt","" & $websiteread)
    EndIf

so every time it did not pass the ping I would get a mgsbox to test it my self , and there where a allot urls that actual did work

so its far from accurate

Link to comment
Share on other sites

Try...

ConsoleWrite( __checkConn("https://www.autoitscript.com/forum/bullshitpage") & @LF)
ConsoleWrite( __checkConn("https://www.autoitscript.com/forum/") & @LF)
ConsoleWrite( __checkConn("http://www.narcolepticmonkeytumor.com") & @LF)

Func __checkConn($url)
    Return (StringLen(InetRead($url, 1)) > 0)
EndFunc

 

AutoIt Absolute Beginners    Require a serial    Pause Script    Video Tutorials by Morthawt   ipify 

Monkey's are, like, natures humans.

Link to comment
Share on other sites

i dont know if i used your code correct:

it keeps saying false 3 times in the consolewrite

#include <ButtonConstants.au3>
#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <StaticConstants.au3>
#include <WindowsConstants.au3>
#include <IE.au3>
#include <String.au3>

#include <MsgBoxConstants.au3>
#include <ButtonConstants.au3>
#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <GUIListBox.au3>
#include <WindowsConstants.au3>
#include <FF V0.6.0.1b-14.au3>

#include <staticConstants.au3>
#include <IE.au3>
#include <String.au3>
#include <Excel.au3>
#include <Array.au3>
#include <FileConstants.au3>


#include <ButtonConstants.au3>
#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <StaticConstants.au3>
#include <WindowsConstants.au3>

#Region ### START Koda GUI section ### Form=
$Form1_1_1_1 = GUICreate("program", 1161, 578, 239, 113)
$List1 = GUICtrlCreateList("", 896, 24, 225, 487)
$List2 = GUICtrlCreateList("", 664, 24, 225, 487)
$test2 = GUICtrlCreateButton("test2", 168, 520, 75, 25)
#EndRegion ### END Koda GUI section ###


Func __checkConn(   $read123)
    $read123 = GUICtrlRead($list1)


    Return (StringLen(InetRead( $read123, 1)) > 0)

    $iIndex += 1

            If $iIndex = _GUICtrlListBox_GetCount($List1) Then $iIndex = 0
            _GUICtrlListBox_SetCurSel($List1, $iIndex)
EndFunc

While 1
    $nMsg = GUIGetMsg()
    Switch $nMsg
        Case $GUI_EVENT_CLOSE
case $test2

ConsoleWrite( __checkConn("https://www.autoitscript.com/forum/bullshitpage") & @LF)
ConsoleWrite( __checkConn("https://www.autoitscript.com/forum/") & @LF)
ConsoleWrite( __checkConn("http://www.narcolepticmonkeytumor.com") & @LF)

EndSwitch
WEnd

btw i have testet the 2 previous scripts:

the first script (old autoit script):      with 42(existing urls) it only had 13 so thats:        30,95%. accurate

the ping script:                                 with 42(existing urls) it had 28  so thats:              66,67%.. accurate

 

it is getting better but still far from good and acceptable

 

Link to comment
Share on other sites

#include <ButtonConstants.au3>
#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <StaticConstants.au3>
#include <WindowsConstants.au3>
#include <IE.au3>
#include <String.au3>

#include <MsgBoxConstants.au3>
#include <ButtonConstants.au3>
#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <GUIListBox.au3>
#include <WindowsConstants.au3>
;#include <FF V0.6.0.1b-14.au3>

#include <staticConstants.au3>
#include <IE.au3>
#include <String.au3>
#include <Excel.au3>
#include <Array.au3>
#include <FileConstants.au3>


#include <ButtonConstants.au3>
#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <StaticConstants.au3>
#include <WindowsConstants.au3>

#Region ### START Koda GUI section ### Form=
$Form1_1_1_1 = GUICreate("program", 1161, 578, 239, 113)
$iInput = GUICtrlCreateInput("url to check", 10, 10, 300)
$test2 = GUICtrlCreateButton("test2", 168, 520, 75, 25)
#EndRegion ### END Koda GUI section ###

GUISetState()

Func __checkConn($read123)
    ;$read123 = GUICtrlRead($list1)


    Return (StringLen(InetRead($read123, 1)) > 0)

    ;$iIndex += 1

    ;If $iIndex = _GUICtrlListBox_GetCount($List1) Then $iIndex = 0
    ;_GUICtrlListBox_SetCurSel($List1, $iIndex)
EndFunc   ;==>__checkConn

While 1
    $nMsg = GUIGetMsg()
    Switch $nMsg
        Case $GUI_EVENT_CLOSE
            Exit
        Case $test2

            ConsoleWrite(__checkConn(GUICtrlRead($iInput)) & @LF)
            ;ConsoleWrite(__checkConn("https://www.autoitscript.com/forum/") & @LF)
            ;ConsoleWrite(__checkConn("http://www.narcolepticmonkeytumor.com") & @LF)

    EndSwitch
WEnd

 

AutoIt Absolute Beginners    Require a serial    Pause Script    Video Tutorials by Morthawt   ipify 

Monkey's are, like, natures humans.

Link to comment
Share on other sites

i have tested it only passed 21 urls from the 42  so its only 50% accurate

there has to be a better accurate way

is there a function when opening the url in IE and checking with a function if the page has succefully openend and not a 404 error

is there a function  for that ?

Link to comment
Share on other sites

Care to give the urls?

EDIT:

I'm not suggesting this method should work absolutely, but I'd like to test as it's worked on every url I've tested it on.

Edited by JohnOne

AutoIt Absolute Beginners    Require a serial    Pause Script    Video Tutorials by Morthawt   ipify 

Monkey's are, like, natures humans.

Link to comment
Share on other sites

random urls i used:

http://www.alfaisaliahhotels.com/
imperial.ac.uk
cas.org
address.com
ird.fr
http://www.iprimus.com.au/
http://www.actua.com/
mobilenations.com
http://niagara.edu/
http://www.qimrberghofer.edu.au/
http://www.healthlawyers.org
http://www.cdc.gov/
http://www.apextechservices.com/
liferay.com
ualr.edu/
sanctuarysoil.com
detroitchamber.com
live.com
ucla.edu
kqed.org
hsph.harvard.edu
yahoo.com
imperial.ac.uk
bizrate.com
blacklakeorganic.net
jdevents.com
nationaljournal.com
consumeraffairs.com
gmail.com
elex.com
delacyexecutive.co.uk
ivr.org.uk
ftc.gov
hemmings.com
mlive.com
mlive.com
imperial.ac.uk
.ac.uk
latimes.com
bizrate.com
mlive.com
rice.edu

the urls that dit not past the ping method

http://www.alfaisaliahhotels.com/
imperial.ac.uk
cas.org
address.com
ird.fr
http://www.iprimus.com.au/
http://www.actua.com/
mobilenations.com
http://niagara.edu/
http://www.qimrberghofer.edu.au/
http://www.healthlawyers.org
http://www.cdc.gov/
http://www.apextechservices.com/
liferay.com
ualr.edu/

it seems that it are always the same urls that dont pas the test

Edited by Arclite86
Link to comment
Share on other sites

#include <Array.au3>
#include <File.au3>

Global $aUrls, $PingTimeout = 1000

_FileReadToArray("urls.txt", $aUrls)

For $i = 1 To $aUrls[0]
    If _UrlExists($aUrls[$i]) Then
        ConsoleWrite("Exists: " & $aUrls[$i] & @LF)
        ContinueLoop
    EndIf
    ConsoleWrite("Not Exists Or No Response in " & $PingTimeout / 1000 & " Seconds: " & $aUrls[$i] & @LF)
Next

Func _UrlExists($url)
    If (StringLen(InetRead($url, 1)) > 0) Then Return 1
    If Ping($url, $PingTimeout) Then Return 2
    Return 0
EndFunc   ;==>__checkConn
Exists: http://www.alfaisaliahhotels.com/
Exists: imperial.ac.uk
Not Exists Or No Response in 1 Seconds: cas.org
Not Exists Or No Response in 1 Seconds: address.com
Not Exists Or No Response in 1 Seconds: ird.fr
Exists: http://www.iprimus.com.au/
Exists: http://www.actua.com/
Not Exists Or No Response in 1 Seconds: mobilenations.com
Exists: http://niagara.edu/
Exists: http://www.qimrberghofer.edu.au/
Exists: http://www.healthlawyers.org
Exists: http://www.cdc.gov/
Exists: http://www.apextechservices.com/
Not Exists Or No Response in 1 Seconds: liferay.com
Not Exists Or No Response in 1 Seconds: ualr.edu/
Exists: sanctuarysoil.com
Exists: detroitchamber.com
Not Exists Or No Response in 1 Seconds: live.com
Exists: ucla.edu
Exists: kqed.org
Exists: hsph.harvard.edu
Exists: yahoo.com
Exists: imperial.ac.uk
Exists: bizrate.com
Exists: blacklakeorganic.net
Exists: jdevents.com
Exists: nationaljournal.com
Exists: consumeraffairs.com
Exists: gmail.com
Exists: elex.com
Exists: delacyexecutive.co.uk
Exists: ivr.org.uk
Exists: ftc.gov
Exists: hemmings.com
Exists: mlive.com
Exists: mlive.com
Exists: imperial.ac.uk
Not Exists Or No Response in 1 Seconds: .ac.uk
Exists: latimes.com
Exists: bizrate.com
Exists: mlive.com
Exists: rice.edu

EDIT:

I cannot be bothered going through all those that failed, but I can tell you that ird.fr and cas.org do not exist, their www. counterparts do though.

So for something like this to work as you expect you need to build on the function and add some of your own logic.

Edited by JohnOne

AutoIt Absolute Beginners    Require a serial    Pause Script    Video Tutorials by Morthawt   ipify 

Monkey's are, like, natures humans.

Link to comment
Share on other sites

arclite86,

Try this.  It reformats the url and works for all except ".ac.uk" which appears to be a typo.  I couldn't work out the regexp for the replacement so I used the goofy looking case stmt for the reformat.

#include <Array.au3>
#include <File.au3>

Global $aUrls, $PingTimeout = 1000

_FileReadToArray(@ScriptDir & "\urls.txt", $aUrls)

For $i = 1 To UBound($aUrls) - 1
    Switch StringLeft($aUrls[$i], 3)
        Case 'www'
            $aUrls[$i] = 'http://' & $aUrls[$i]
        Case 'htt'
            ContinueLoop
        Case Else
            $aUrls[$i] = 'http://www.' & $aUrls[$i]
    EndSwitch
Next

For $i = 1 To $aUrls[0]
    If _UrlExists($aUrls[$i]) Then
        ConsoleWrite("Exists: " & $aUrls[$i] & @LF)
        ContinueLoop
    EndIf
    ConsoleWrite("Not Exists Or No Response in " & $PingTimeout / 1000 & " Seconds: " & $aUrls[$i] & @LF)
Next

Func _UrlExists($url)
    If (StringLen(InetRead($url, 1)) > 0) Then Return 1
    If Ping($url, $PingTimeout) Then Return 2
    Return 0
EndFunc   ;==>_UrlExists

kylomas

Forum Rules         Procedure for posting code

"I like pigs.  Dogs look up to us.  Cats look down on us.  Pigs treat us as equals."

- Sir Winston Churchill

Link to comment
Share on other sites

#include <Array.au3>
#include <File.au3>

Global $aUrls, $PingTimeout = 1000

_FileReadToArray("urls.txt", $aUrls)

For $i = 1 To $aUrls[0]
    If _UrlExists($aUrls[$i]) Then
        ConsoleWrite("Exists: " & $aUrls[$i] & @LF)
        ContinueLoop
    EndIf
    ConsoleWrite("Not Exists Or No Response in " & $PingTimeout / 1000 & " Seconds: " & $aUrls[$i] & @LF)
Next

Func _UrlExists($url)
    If (StringLen(InetRead($url, 1)) > 0) Then Return 1
    If Ping($url, $PingTimeout) Then Return 2
    Return 0
EndFunc   ;==>__checkConn
Exists: http://www.alfaisaliahhotels.com/
Exists: imperial.ac.uk
Not Exists Or No Response in 1 Seconds: cas.org
Not Exists Or No Response in 1 Seconds: address.com
Not Exists Or No Response in 1 Seconds: ird.fr
Exists: http://www.iprimus.com.au/
Exists: http://www.actua.com/
Not Exists Or No Response in 1 Seconds: mobilenations.com
Exists: http://niagara.edu/
Exists: http://www.qimrberghofer.edu.au/
Exists: http://www.healthlawyers.org
Exists: http://www.cdc.gov/
Exists: http://www.apextechservices.com/
Not Exists Or No Response in 1 Seconds: liferay.com
Not Exists Or No Response in 1 Seconds: ualr.edu/
Exists: sanctuarysoil.com
Exists: detroitchamber.com
Not Exists Or No Response in 1 Seconds: live.com
Exists: ucla.edu
Exists: kqed.org
Exists: hsph.harvard.edu
Exists: yahoo.com
Exists: imperial.ac.uk
Exists: bizrate.com
Exists: blacklakeorganic.net
Exists: jdevents.com
Exists: nationaljournal.com
Exists: consumeraffairs.com
Exists: gmail.com
Exists: elex.com
Exists: delacyexecutive.co.uk
Exists: ivr.org.uk
Exists: ftc.gov
Exists: hemmings.com
Exists: mlive.com
Exists: mlive.com
Exists: imperial.ac.uk
Not Exists Or No Response in 1 Seconds: .ac.uk
Exists: latimes.com
Exists: bizrate.com
Exists: mlive.com
Exists: rice.edu

EDIT:

I cannot be bothered going through all those that failed, but I can tell you that ird.fr and cas.org do not exist, their www. counterparts do though.

So for something like this to work as you expect you need to build on the function and add some of your own logic.

works pretty nice 34 out of 42  80,95% accurate

Link to comment
Share on other sites

arclite86,

Try this.  It reformats the url and works for all except ".ac.uk" which appears to be a typo.  I couldn't work out the regexp for the replacement so I used the goofy looking case stmt for the reformat.

#include <Array.au3>
#include <File.au3>

Global $aUrls, $PingTimeout = 1000

_FileReadToArray(@ScriptDir & "\urls.txt", $aUrls)

For $i = 1 To UBound($aUrls) - 1
    Switch StringLeft($aUrls[$i], 3)
        Case 'www'
            $aUrls[$i] = 'http://' & $aUrls[$i]
        Case 'htt'
            ContinueLoop
        Case Else
            $aUrls[$i] = 'http://www.' & $aUrls[$i]
    EndSwitch
Next

For $i = 1 To $aUrls[0]
    If _UrlExists($aUrls[$i]) Then
        ConsoleWrite("Exists: " & $aUrls[$i] & @LF)
        ContinueLoop
    EndIf
    ConsoleWrite("Not Exists Or No Response in " & $PingTimeout / 1000 & " Seconds: " & $aUrls[$i] & @LF)
Next

Func _UrlExists($url)
    If (StringLen(InetRead($url, 1)) > 0) Then Return 1
    If Ping($url, $PingTimeout) Then Return 2
    Return 0
EndFunc   ;==>_UrlExists

kylomas

42 out of 42 urls 100% accurate!!!! thank you so much kylomas!!:D:lmao:

Link to comment
Share on other sites

arclite86,

Run this...

#include <Array.au3>
#include <File.au3>
#include <date.au3>

Global $aUrls



_FileReadToArray(@ScriptDir & "\urls.txt", $aUrls)

For $i = 1 To UBound($aUrls) - 1
    Switch StringLeft($aUrls[$i], 3)
        Case 'www'
            $aUrls[$i] = 'http://' & $aUrls[$i]
        Case 'htt'

            ContinueLoop

        Case Else
            $aUrls[$i] = 'http://www.' & $aUrls[$i]
    EndSwitch

Next

For $i = 1 To $aUrls[0]
    If _UrlExists($aUrls[$i]) Then
        ConsoleWrite(_Now() & " Exists: " & $aUrls[$i] & @LF)
        ContinueLoop

    EndIf

    ConsoleWrite(_Now() & ' ' & $aUrls[$i] & ' is not reachable' & @LF)
Next

Func _UrlExists($url)
    If (StringLen(InetRead($url, 1)) > 0) Then Return 1
    Return 0
EndFunc   ;==>_UrlExists

Note the timings...

Also, the ping is not necessary...

kylomas

Forum Rules         Procedure for posting code

"I like pigs.  Dogs look up to us.  Cats look down on us.  Pigs treat us as equals."

- Sir Winston Churchill

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...