Jump to content
Sign in to follow this  
youtuber

Problem with deleting the same duplicate

Recommended Posts

I want to delete the same ones and separate the unique ones, but I think there is a problem

#include <Array.au3>

$aWebAdress = "https://autoitscripttr.blogspot.com/atom.xml"

$aGetNewConnect = Connections($aWebAdress)

$aUrlregex1 = StringRegExp($aGetNewConnect, "(?i)href=(?:'|(?:&quot;))([^'&]+)",3)

If isArray($aUrlregex1)  then
    For $c = 0 to UBound($aUrlregex1)-1
        ConsoleWrite(_ArrayRemoveDuplicate($aUrlregex1[$c]) & @CRLF)
    Next
EndIf


Func _ArrayRemoveDuplicate($fArray)
_ArraySort($fArray,0,0)
Dim $i2 = 0
Dim $duplicateindex[1]
For $index = 1 to UBound($fArray) - 1
  If $fArray[$index] = $fArray[$index-1] Then
   $duplicateindex[$i2] = $index
   $i2 = $i2 + 1
   ReDim $duplicateindex[$i2]
  EndIf
Next
For $dupindex = 0 To UBound($duplicateindex) - 1
_ArrayDelete($fArray,$duplicateindex[$dupindex])
Next
Return $fArray
EndFunc

Func Connections($address)
    Local $oHTTP = ObjCreate("winhttp.winhttprequest.5.1")
    $oHTTP.Open("GET", $address, False)
    $oHTTP.SetRequestHeader("User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:55.0) Gecko/20100101 Firefox/55.0")
    $oHTTP.SetRequestHeader("Accept-Language", "tr-TR,tr;q=0.8,en-US;q=0.5,en;q=0.3")
    $oHTTP.Send()
    If @error Then
        ConsoleWrite("Line : " & @ScriptLineNumber & " Connect Error " & @CRLF)
        $oHTTP = 0
        Return SetError(1)
    EndIf

    If $oHTTP.Status = 200 Then
        Local $sReceived = $oHTTP.ResponseText
        $oHTTP = Null
        Return $sReceived
    EndIf
   $oHTTP = Null
    Return -1
EndFunc

 

Share this post


Link to post
Share on other sites

If you have the array already, and want just the unique items, why is _ArrayUnique not working for you?


"Profanity is the last vestige of the feeble mind. For the man who cannot express himself forcibly through intellect must do so through shock and awe" - Spencer W. Kimball

How to get your question answered on this forum!

Share this post


Link to post
Share on other sites

is this a better option to delete the same url addresses?

$aGetNewConnect = Connections($aWebAdress)

$aUrlregex1 = StringRegExp($aGetNewConnect, "(?i)href=(?:'|(?:&quot;))([^'&]+)",3)

If isArray($aUrlregex1)  then
   $aUniques2=_ArrayUnique($aUrlregex1)

 Dim $aSimilar[UBound($aUrlregex1)]
    $c = 0
    For $i = 1 To UBound($aUniques2) - 1
        $aPos = _ArrayFindAll($aUrlregex1, $aUniques2[$i])
        If UBound($aPos) > 0 Then
            For $j = 0 To UBound($aPos) - 1
                $aSimilar[$c] = $aUrlregex1[$aPos[$j]]
                $c += 1
            Next
        EndIf
    Next
    ReDim $aSimilar[$c]
    $aUniques3=_ArrayUnique($aSimilar, 0, 0, 1)
    _ArraySort($aUniques3)
    For $i = 1 to UBound($aUniques3) - 1
ConsoleWrite($aUniques3[$i] & @CRLF)
    Next
EndIf

 

Share this post


Link to post
Share on other sites

Using the code in your first post and JLogan's suggestion, (if that's the result you want) the snippet below removes dupe strings (42 original, 37 with dupes removed)

#include <Array.au3>

$aWebAdress = "https://autoitscripttr.blogspot.com/atom.xml"

$aGetNewConnect = Connections($aWebAdress)

$aUrlregex1 = StringRegExp($aGetNewConnect, "(?i)href=(?:'|(?:&quot;))([^'&]+)",3)

_ArrayDisplay($aUrlregex1)

If isArray($aUrlregex1) then
    $as_Unique = _ArrayUnique($aUrlregex1)
EndIf

_ArrayDisplay($as_Unique)

Func Connections($address)
    Local $oHTTP = ObjCreate("winhttp.winhttprequest.5.1")
    $oHTTP.Open("GET", $address, False)
    $oHTTP.SetRequestHeader("User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:55.0) Gecko/20100101 Firefox/55.0")
    $oHTTP.SetRequestHeader("Accept-Language", "tr-TR,tr;q=0.8,en-US;q=0.5,en;q=0.3")
    $oHTTP.Send()
    If @error Then
        ConsoleWrite("Line : " & @ScriptLineNumber & " Connect Error " & @CRLF)
        $oHTTP = 0
        Return SetError(1)
    EndIf

    If $oHTTP.Status = 200 Then
        Local $sReceived = $oHTTP.ResponseText
        $oHTTP = Null
        Return $sReceived
    EndIf
   $oHTTP = Null
    Return -1
EndFunc

 

Share this post


Link to post
Share on other sites
1 hour ago, youtuber said:

Is that enough for that?

I don't know what that means but, if you want to only get the unique values from the first array ($aUrlregex1) and write those strings to the console then your script will do that

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...