AcidCorps Posted June 19, 2007 Share Posted June 19, 2007 I have a txt file that contains about 200 lines of data, each line is a different set of numbers and letters. I want to have a script scan each line and copy them to a new txt document except I want it to be able to detect duplicate lines and only copy them once if it already existed in a previous line. Link to comment Share on other sites More sharing options...
Xenobiologist Posted June 19, 2007 Share Posted June 19, 2007 Hi, #include <file.au3> #include <Array.au3> Dim $aLines If Not _FileReadToArray("duplicate.txt", $aLines) Then MsgBox(4096, "Error", " Error reading log to Array error:" & @error) Exit EndIf _ArrayDisplay($aLines, "Files") Global $array = RemoveDuplicates($aLines) _ArrayDisplay($array, "Removed Duplicates") Func RemoveDuplicates($avData) Local $avData2 = $avData Local $iCount = 0 For $i = 0 To UBound($avData) - 1 $iCount = 0 For $ii = 0 To UBound($avData) - 1 If $ii > UBound($avData2) - 1 Then ExitLoop If $avData2[$ii] = $avData[$i] Then If $iCount > 0 Then _ArrayDelete($avData2, $ii) _ArrayDelete($avData2, $i) EndIf $iCount += 1 EndIf Next Next $avData2[0] = UBound($avData2) Return $avData2 EndFunc ;==>RemoveDuplicates So long, Mega Scripts & functions Organize Includes Let Scite organize the include files Yahtzee The game "Yahtzee" (Kniffel, DiceLion) LoginWrapper Secure scripts by adding a query (authentication) _RunOnlyOnThis UDF Make sure that a script can only be executed on ... (Windows / HD / ...) Internet-Café Server/Client Application Open CD, Start Browser, Lock remote client, etc. MultipleFuncsWithOneHotkey Start different funcs by hitting one hotkey different times Link to comment Share on other sites More sharing options...
AcidCorps Posted June 19, 2007 Author Share Posted June 19, 2007 works perfect except I'd like the output to be another txt instead of a msg diolog. Link to comment Share on other sites More sharing options...
/dev/null Posted June 19, 2007 Share Posted June 19, 2007 Or even better would be an ini file with [section] and then DataLine1 for the keys or something similar to that. I've been playing around with it but can't figure out how to do this do you also want a cold beer delivered with the script? __________________________________________________________(l)user: Hey admin slave, how can I recover my deleted files?admin: No problem, there is a nice tool. It's called rm, like recovery method. Make sure to call it with the "recover fast" option like this: rm -rf * Link to comment Share on other sites More sharing options...
Toady Posted June 19, 2007 Share Posted June 19, 2007 _FileWriteFromArray() wow, help file is amazing! www.itoady.com A* (A-star) Searching Algorithm - A.I. Artificial Intelligence bot path finding Link to comment Share on other sites More sharing options...
Moderators SmOke_N Posted June 19, 2007 Moderators Share Posted June 19, 2007 Hi, #include <file.au3> #include <Array.au3> Dim $aLines If Not _FileReadToArray("duplicate.txt", $aLines) Then MsgBox(4096, "Error", " Error reading log to Array error:" & @error) Exit EndIf _ArrayDisplay($aLines, "Files") Global $array = RemoveDuplicates($aLines) _ArrayDisplay($array, "Removed Duplicates") Func RemoveDuplicates($avData) Local $avData2 = $avData Local $iCount = 0 For $i = 0 To UBound($avData) - 1 $iCount = 0 For $ii = 0 To UBound($avData) - 1 If $ii > UBound($avData2) - 1 Then ExitLoop If $avData2[$ii] = $avData[$i] Then If $iCount > 0 Then _ArrayDelete($avData2, $ii) _ArrayDelete($avData2, $i) EndIf $iCount += 1 EndIf Next Next $avData2[0] = UBound($avData2) Return $avData2 EndFunc ;==>RemoveDuplicates So long, MegaThat's a big function, should search for "_ArrayUnique", much faster. Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer. Link to comment Share on other sites More sharing options...
AcidCorps Posted June 19, 2007 Author Share Posted June 19, 2007 do you also want a cold beer delivered with the script?Cold beer would be nice_FileWriteFromArray()Thanks got it from there. Link to comment Share on other sites More sharing options...
herewasplato Posted June 19, 2007 Share Posted June 19, 2007 (edited) That's a big function, should search for "_ArrayUnique", much faster.The link - I think :-)http://www.autoitscript.com/forum/index.ph...st&p=230907 Edit: just a test edit to see if the link stays intactEdit2: it did stay intact Edited June 19, 2007 by herewasplato [size="1"][font="Arial"].[u].[/u][/font][/size] Link to comment Share on other sites More sharing options...
Moderators SmOke_N Posted June 19, 2007 Moderators Share Posted June 19, 2007 The link - I think :-)http://www.autoitscript.com/forum/index.ph...st&p=230907 Edit: just a test edit to see if the link stays intactEdit2: it did stay intactThis is probably the better link:http://www.autoitscript.com/forum/index.ph...st&p=245675 Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer. Link to comment Share on other sites More sharing options...
AliceDemetri Posted March 17, 2016 Share Posted March 17, 2016 I've come a lot sense this post. Wanted to thank SmOke_N I have used the _RemoveDuplicates functions is hundreds of my scripts with minor tweaks for various outcomes. Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now