Jump to content

The way to remove the links that are duplicates?

Recommended Posts


yo, i need some help.

Im training with _StringsBetween, on big site! Alot fun, My script turned 8 times created a text file with a weight of 1GB. I saw that in the file was a lot of repetition. Is it possible to somehow set the string to save the file, only one URL without repetition?

#include <ButtonConstants.au3>
#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <INet.au3>
#include <StringConstants.au3>
#include <File.au3>

;I'm coming for blood, no code of conduct, no law.
;I'm coming for blood, no code of conduct, no law.
#Region ### START Koda GUI section ### Form=
$Form1 = GUICreate("Form1", 1211, 812, 43, 110)
$Button1 = GUICtrlCreateButton("Button1", 0, 8, 249, 81)
$Button2 = GUICtrlCreateButton("Button2", 350, 8, 249, 81)
$Edit1 = GUICtrlCreateEdit("0", 8, 128, 609, 257, BitOR($ES_CENTER,$ES_AUTOHSCROLL,$ES_READONLY,$ES_WANTRETURN))
$Edit2 = GUICtrlCreateEdit("2", 632, 0, 577, 809)
#EndRegion ### END Koda
local $iFileSize = FileGetSize('')

Func VisitFrontPage()
    local $Liczba = _FileCountLines(@ScriptDir&'\data\links.txt')
    local $liczba2 = GUICtrlSetData($Edit1,Random(1,$Liczba,1))
    local $Liczba3 = GUICtrlRead($Edit1)
    local $Liczba4 = FileReadLine(@ScriptDir&'\data\links.txt',$Liczba3)
$data = _INetGetSource ( FileReadLine(@ScriptDir&'\data\links.txt',Random(1,$Liczba,1)))
$linki = StringRegExp($data, '<a href="http://www.wykop.pl/link/(.*?)/" title=""',3)
For $q = 0 To UBound($linki) -1
$linki = StringRegExp($data, 'href="http://www.wykop.pl/ludzie/(.*?)/">',3) ; pobieranie ludzi co dodali znaleziska
For $w = 0 To UBound($linki) -1
$linki = StringRegExp($data, '<a href="http://www.wykop.pl/ludzie/(.*?)/" title="',3) ; pobieranie ludzi co sa na stronie z mikro
For $e = 0 To UBound($linki) -1
$linki = StringRegExp($data, '<a class="tag create" href="http://www.wykop.pl/tag/(.*?)/"><em>',3) ; pobieranie ludzi co sa na stronie z mikro
For $r = 0 To UBound($linki) -1
$linki = StringRegExp($data, '<class="showTagSummary" href="http://www.wykop.pl/tag/(.*?)">',3) ; pobieranie ludzi co sa na stronie z mikro
For $t = 0 To UBound($linki) -1
 $linki = StringRegExp($data, 'href="http://www.wykop.pl/ludzie/(.*?)/" title=""',3) ; pobieranie ludzi z znaleziska, komentarze
For $a = 0 To UBound($linki) -1
 $linki = StringRegExp($data, 'href="http://www.wykop.pl/tag/index/(.*?)/"',3) ; pobierane tagi ze znaleziska,
For $s = 0 To UBound($linki) -1
 $linki = StringRegExp($data, '<a class="clearfix" href="http://www.wykop.pl/link/(.*?)/?utm_source',3) ; pobierane znaleziska z prawego menu
For $d = 0 To UBound($linki) -1
GuiCtrlSetData($Edit2, GuiCtrlRead($Edit2)+1)

While 1
    $nMsg = GUIGetMsg()
    Switch $nMsg
        Case $Button1
        Case $Button2
        Case $GUI_EVENT_CLOSE



Share this post

Link to post
Share on other sites

Why do you make the function VisitFrontPage recursive and what's the purpose of

local $iFileSize = FileGetSize('')

This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Similar Content

    • Valnurat
      By Valnurat
      My color of pixel exists more than one place.
      How can I control the position so I find the exact position before I continue in my code?
    • nacerbaaziz
      By nacerbaaziz
      Hi dear
      With this script you can print Unicode text in the CMD screen
      the script is  easy to use
      just you  write the text that contains Unicode in the first input
      and the script automatically reflect the code in the second input
       you can copy the text to the clipboard
      or you can try printing the text in the CMD window
      I apologize to everyone for colors and shape if not appropriate
      I'm a blind man and I do not see
      Thank you for your understanding
      Greetings to all of youCmdUtM.au3
    • Abdulla060
      By Abdulla060
      Hello all, i have about 20 arrays in my script and i need to delete the first element of each, i have the name of the arrays in another array (its used in another part of the script)  and i tried to passing the names to from that array to the _ArrayDelete function but nothing happens and i get an error code :1 which state that i'm trying to delete an array which is not an array
      code example
      #include <Array.au3> global $allarrays[3] = ['array1','array2','array3'] global $array1[3] = ['1', '2', '3'] global $array2[3] = ['1', '2', '3'] global $array3[3] = ['1', '2', '3'] for $i = 0 to UBound($allarrays) - 1 _ArrayDelete($allarrays[$i], 0) Next _ArrayDisplay($array1) obviously i can just use  _arraydelete over and over however i'm looking for a more elegant way of doing this
    • kartune
      By kartune
      Hello, I am getting this error message when running my script:
      Case $aButiD[0] To $aButiD[$iTotButtons - 1]
      Case ^ ERROR
      Error:  Array variable has incorrect number of subscripts or subscript dimension range exceeded.
      The script's purpose is to read my ini file and create buttons for the sections and section contents.
      The first part of the script creates buttons for the sections of the ini, once you click on the section button it should open another set of buttons of all the contents under the section
      In this case, the section in the .ini file is called "TPA"
      on the Func flavorco()
      It opens the contents of "TPA" into buttons.
      Right now, the section it is reading is determined by $sSection.
      It is only when I try to change $sSection = "TPA"
      $sSection = $flavorco
      that the error starts to appear
      Here is the script
      #include <MsgBoxConstants.au3> #include <WinAPIFiles.au3> #include <Array.au3> #include <GUIConstantsEx.au3> #include <WindowsConstants.au3> #include <ButtonConstants.au3> $optbar = IniReadSectionNames("test.ini") If Not IsArray($optbar) Then Exit Local $Button[UBound($optbar) - 1] $mGUI = GUICreate('Restock Flavor Ingredients', 10 + (130 * UBound($optbar)), 100) For $x = 1 To UBound($optbar) - 1 $var = IniReadSection("test.ini", $optbar[$x]) If @error Then ContinueLoop $X_Coord = 10 + (($x - 1) * 100) $Button[$x - 1] = GUICtrlCreateButton($optbar[$x], $X_Coord, 30, 70, 30) Next GUISetState() While 1 $mMsg = GUIGetMsg() If $mMsg = $GUI_EVENT_CLOSE Then GUIDelete($mGUI) ExitLoop EndIf For $i = 0 To UBound($Button) - 1 If $mMsg = $Button[$i] Then MsgBox(0, '$Msg = ' & $mMsg, GuiCtrlRead($Button[$i]) & ' Pressed', 2) Global $flavorco = $Button ; sets the flavor company's variable Call('flavorco') EndIf Next WEnd Func flavorco() ;opened flavor company's flavors MsgBox(0, '$Msg = ' & $mMsg, GuiCtrlRead($Button[$i]) & $flavorco, 0) Local $sFilePath = @ScriptDir & "\Test.ini" Local $sSection = $Button Local $aArray = IniReadSection($sFilePath, $sSection) _ArrayDelete($aArray, 0) ;_ArrayDisplay($aArray) Local $iIndex Local $iTotButtons = UBound($aArray) Local $iNumPerRow = 5 Local $iButWidth = 100 Local $iButDepth = 30 Local $aButiD[$iTotButtons] Local $hGUI = GUICreate('Ingredients Stock "' & $sFilePath & '"', 10 + ($iButWidth + 10) * $iNumPerRow, _ 10 + Ceiling($iTotButtons / $iNumPerRow) * ($iButDepth + 10), -1, $WS_EX_TOPMOST) For $i = 0 To $iTotButtons - 1 ; $x = $iXBorder + (($iRectWidth + $iSpacing) * Mod($i, $iNumCols)) $x = 10 + (($iButWidth + 10) * Mod($i, $iNumPerRow)) ; $y = $iYBorder + (($iRectDepth + $iSpacing) * Int($i / $iNumCols)) $y = 10 + (($iButDepth + 10) * Int($i / $iNumPerRow)) $aButiD[$i] = GUICtrlCreateButton($aArray[$i][0], $x, $y, $iButWidth, $iButDepth) ;, $BS_NOTIFY) Next GUISetState(@SW_SHOW) While 1 $msg = GUIGetMsg() Switch $msg Case $GUI_EVENT_CLOSE GUIDelete($hGUI) ExitLoop Case $aButiD[0] To $aButiD[$iTotButtons - 1] ;clicking on a flavor name $iIndex = ($msg - $aButiD[0]) $RSINGRflavor = $aArray[$iIndex][0] ;flavor name $RSINGRml = $aArray[$iIndex][1] ; ml $RSINGRgal = $aArray[$iIndex][1]*0.000264172 ;converts ml to gal Call('openflavor') EndSwitch WEnd EndFunc Func openflavor() #Region ### START Koda GUI section ### Form=F:\Karl\AutoIt\FJ-Stock JP\GUI Forms\RSINGRSUBFORM.kxf ;opens flavor to view stock and restock submit $RSINGRSUBFORM = GUICreate($sSection & " " & $RSINGRflavor, 242, 213, 530, 269) $RSINGRFlavorTitle = GUICtrlCreateLabel($sSection & " " & $RSINGRflavor, 32, 8, 200, 28) GUICtrlSetFont(-1, 16, 800, 0, "MS Sans Serif") $RSINGRstockcaption = GUICtrlCreateLabel("Current Stock in ml:", 16, 48, 96, 17) $RSINGRstockml = GUICtrlCreateLabel($RSINGRml, 120, 48, 114, 25) GUICtrlSetFont(-1, 8, 800, 0, "MS Sans Serif") $RSINGRInput = GUICtrlCreateInput("ml", 40, 112, 153, 21) $RSINGRSUBMIT = GUICtrlCreateButton("SUBMIT", 16, 152, 81, 33) $RSINGRCLOSE = GUICtrlCreateButton("CLOSE", 145, 151, 81, 33) $RSINGRgalcaption = GUICtrlCreateLabel("Current Stock in gal:", 16, 80, 97, 17) $RSINGRstockgal = GUICtrlCreateLabel($RSINGRGAL, 120, 80, 114, 25) GUICtrlSetFont(-1, 8, 800, 0, "MS Sans Serif") GUISetState(@SW_SHOW) #EndRegion ### END Koda GUI section ### ; $msg = "" While 1 $nMsg = GUIGetMsg() Switch $nMsg Case $GUI_EVENT_CLOSE GUIDelete($RSINGRSUBFORM) ExitLoop Case $RSINGRCLOSE GUIDelete($RSINGRSUBFORM) ExitLoop EndSwitch WEnd EndFunc  
      Any help is much appreciated!  Thanks.
      My guess is that $Button isn't an actual string??  Whenever I use msgbox it shows up the string exactly, no numbers attached.  So i'm not sure whats going on here.
    • Rammanan
      By Rammanan
      Hi all, Below is my script,
      Pls advise me if any wrong on that....File exit in location but the return -1.
      FileOpen($file, 0)
         $line = FileReadLine($file)
      Local $aArray = _FileListToArrayRec($serverpath, "*|*.txt", $FLTAR_FILES, $FLTAR_RECUR, $FLTAR_SORT, $FLTAR_NOPATH)
      If @error Then
              MsgBox($MB_SYSTEMMODAL, "Ooops!", "No files found")
          ;_ArrayDisplay($aArray, "Sorted tree")
                 Local $hSearch = FileFindFirstFile($line)

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.