phatzilla

Optimizing array difference search speed (or other solutions welcome!)

19 posts in this topic

#1 ·  Posted (edited)

Hi gang,

I'm in a bit of a pickle here, the gist of it is that i have one "main" file with ~200k lines, and literally hundreds of other files each ranging from 1k - 100k lines.

I need to go through each of the "other" files and (separately) compare them to the "main" file, and save the differences between the two (no duplicates)

 

The issue is that each comparison (especially if the "other" file has 50+ k lines) takes over a minute each... 

Is there anyway to cut this time down? As far as i know im using the most optimized array difference script


Here's a rough mockup of the script im currently using

note: the main file has all unique lines, and the "other" files wont ever have any lines that *DO NOT* appear in the main file, if that helps

#include <array.au3>
#include <file.au3>



global $info_file
global $compare_file
global $Differece

$info_file = FileReadToArray("info-file(200k lines).txt")




$compare_file = FileReadToArray("compare-file(100k lines).txt")




$Differece = _Diff($info_file, $compare_file, 0) ; get the difference between 2 arrays, NO duplicates
_ArrayDisplay($Differece)







;=================================================
; Function Name:   _Diff($Set1, $Set2 [, $GetAll=0 [, $Delim=Default]])
; Description::    Find values in $Set1 that do not occur in $Set2
; Parameter(s):    $Set1    set 1 (1D-array or delimited string)
;                  $Set2    set 2 (1D-array or delimited string)
;      optional:   $GetAll  0 - only one occurence of every difference are shown (Default)
;                           1 - all differences are shown, allowing duplicates
;      optional:   $Delim   Delimiter for strings (Default use the separator character set by Opt("GUIDataSeparatorChar") )
; Return Value(s): Succes   1D-array of values from $Set1 that do not occur in $Set2
;                  Failure  -1  @error  set, that was given as array, isn't 1D-array
; Note:            Comparison is case-sensitive! - i.e. Number 9 is different to string '9'!
; Author(s):       BugFix (bugfix@autoit.de) Modified by ParoXsitiC for Faster _Diff (Formally _GetIntersection)
;=================================================
Func _Diff(ByRef $Set1, ByRef $Set2, $GetAll = 0, $Delim = Default)
    Local $o1 = ObjCreate("System.Collections.ArrayList")
    Local $o2 = ObjCreate("System.Collections.ArrayList")
    Local $oDiff1 = ObjCreate("System.Collections.ArrayList")
    Local $tmp, $i
    If $GetAll <> 1 Then $GetAll = 0
    If $Delim = Default Then $Delim = Opt("GUIDataSeparatorChar")
    If Not IsArray($Set1) Then
        If Not StringInStr($Set1, $Delim) Then
            $o1.Add($Set1)
        Else
            $tmp = StringSplit($Set1, $Delim, 1)
            For $i = 1 To UBound($tmp) - 1
                $o1.Add($tmp[$i])
            Next
        EndIf
    Else
        If UBound($Set1, 0) > 1 Then Return SetError(1, 0, -1)
        For $i = 0 To UBound($Set1) - 1
            $o1.Add($Set1[$i])
        Next
    EndIf

    If Not IsArray($Set2) Then
        If Not StringInStr($Set2, $Delim) Then
            $o2.Add($Set2)
        Else
            $tmp = StringSplit($Set2, $Delim, 1)
            For $i = 1 To UBound($tmp) - 1
                $o2.Add($tmp[$i])
            Next
        EndIf
    Else
        If UBound($Set2, 0) > 1 Then Return SetError(1, 0, -1)
        For $i = 0 To UBound($Set2) - 1
            $o2.Add($Set2[$i])
        Next
    EndIf

    For $tmp In $o1
        If Not $o2.Contains($tmp) And ($GetAll Or Not $oDiff1.Contains($tmp)) Then $oDiff1.Add($tmp)
    Next

    If $oDiff1.Count <= 0 Then Return 0

    Local $aOut[$oDiff1.Count]
    $i = 0
    For $tmp In $oDiff1
        $aOut[$i] = $tmp
        $i += 1
    Next
    Return $aOut
EndFunc   ;==>_Diff

 

Edited by phatzilla

Share this post


Link to post
Share on other sites



Merge the 2 arrays into one, then run that merged array through _ArrayUnique to get the array with no repeats.


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites

You could directly try Scripting.Dictionary , see here
 

1 person likes this

Share this post


Link to post
Share on other sites

_ArrayUnique uses a scripting dictionary

 


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites

#5 ·  Posted (edited)

BrewManNH,

Will _arrayunique remove ALL duplicate entries?   If i recall, it simply keeps one of the entries.

If you run:

1
3
3
2

through _arrayunique, you'll get

1
3
2

I need it to be
1
2
 

 

 

What i need is a new array that's the difference between "main file" and "other file".

So if main file array is

2
1
4
5
3

and other file array is

2
3
1

new array should be

4
5

 

In other words, i dont want to preserve any instances of the duplicate lines.  the _Diff function achieves this, but i need to get it a bit faster, if possible...

Or, failing that, a new solution.

 

Edited by phatzilla

Share this post


Link to post
Share on other sites

What's in the files that you're comparing between the 2? What do some of the entries look like? I personally would sort the array to be searched and then use _ArrayBinarySearch to quickly search the array for any entries that are duplicates. I'd need a sample of a couple of files you're comparing, even generic ones that the format is similar to what you're working with, I can probably create something that will do this very fast. Using objects on arrays probably isn't very efficient.


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites

#7 ·  Posted (edited)

They are text files.

Each line is about 100-120 chars in length alphanumeric

Edited by phatzilla

Share this post


Link to post
Share on other sites

#8 ·  Posted (edited)

_ArrayUnique uses a scripting dictionary

I know, I said directly because the OP asked for "save the differences between the two"  and _ArrayUnique doesn't do that

Edited by mikell

Share this post


Link to post
Share on other sites

How do lines differ, typically? Are they 99% the same or much more random? Are the differences located near the head, near the tail or can they differ anywhere in between?

This can dramatically change the speed of comparisons. Have you tried comparing the hash of the strings (MD5 will do here): comparing MD5s is pretty fast.

Depending on actual contents, SQLite might prove useful for your use case.


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

#10 ·  Posted (edited)

Actually, i just adjusted my script based on mikells suggestion....



 

global $a = FileReadToArray("info-file(200k lines).txt")
global $b = FileReadToArray("compare-file(100k lines).txt")

Local $sda = ObjCreate("Scripting.Dictionary")
Local $sdb = ObjCreate("Scripting.Dictionary")
Local $sdc = ObjCreate("Scripting.Dictionary")

For $i In $a
    $sda.Item($i)
Next
For $i In $b
    $sdb.Item($i)
Next

For $i In $a
    If $sdb.Exists($i) Then $sdc.Item($i)
Next
$asd3 = $sdc.Keys()

For $i In $asd3
    If $sda.Exists($i) Then $sda.Remove($i)
    If $sdb.Exists($i) Then $sdb.Remove($i)
Next
$asd1 = $sda.Keys()
$asd2 = $sdb.Keys()

_ArrayDisplay($asd1, "$asd1")
_ArrayDisplay($asd2, "$asd2")
_ArrayDisplay($asd3, "$asd3")


Now this is much faster... $ASD1 is the only array that i need as it contains the lines that are *ONLY* found in "info" file (because the "other" compare file will never contain any lines that aren't found in info file).  Is there any way to speed this script up?  I dont really need the other 2 new arrays asd2,asd3.  asd1 contains all the information i need.

Edited by phatzilla

Share this post


Link to post
Share on other sites

#11 ·  Posted (edited)

I have a  small doubt. You people see the above code ?. Is this safe to use the same variable name twice or thrice in the same script ?. I mean "$i". 

Edit - Never mind i got the answer. 

Edited by kcvinu

My Contributions

UDF Link Viewer   --- A tool to visit the links of some most important UDFs 

 Includer_2  ----- A tool to type the #include statement automatically 

 Digits To Date  ----- date from 3 integer values

PrintList ----- prints arrays into console for testing.

 Alert  ------ An alternative for MsgBox 

 MousePosition ------- A simple tooltip display of mouse position

GRM Helper -------- A littile tool to help writing code with GUIRegisterMsg function

Access_UDF  -------- An UDF for working with access database files. (.*accdb only)

 

Share this post


Link to post
Share on other sites

kcvinu,

To answer your question for any later readers: Yes, it is quite safe to use the same variable name in several For...Next loops as long as each loop is entirely separate and has terminated before the next one starts. If there is any overlap or nesting then the loop count variables must be differentiated.

M23

1 person likes this

Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

In this case you can spare significant steps:

Local $a = FileReadToArray("info-file(200k lines).txt")
Local $sda = ObjCreate("Scripting.Dictionary")

For $i In $a
    $sda.Item($i)
Next

; for each "compare-file" do
; see _FileListToArray and friends

    ; also $a content is no more needed,
    ; we reuse it for all target files
    $a = FileReadToArray("compare-file(100k lines).txt")

    For $i In $a
        If $sda.Exists($i) Then $sda.Remove($i)
    Next
; next file

Local $asd1 = $sda.Keys()
$sda = 0
_ArrayDisplay($asd1, "$asd1")

 


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

@Melba23 Thank you for the clarification. 


My Contributions

UDF Link Viewer   --- A tool to visit the links of some most important UDFs 

 Includer_2  ----- A tool to type the #include statement automatically 

 Digits To Date  ----- date from 3 integer values

PrintList ----- prints arrays into console for testing.

 Alert  ------ An alternative for MsgBox 

 MousePosition ------- A simple tooltip display of mouse position

GRM Helper -------- A littile tool to help writing code with GUIRegisterMsg function

Access_UDF  -------- An UDF for working with access database files. (.*accdb only)

 

Share this post


Link to post
Share on other sites

kcvinu,
Note also that the $i is declared as Local in the For loop (Thus Saith The Help File) but keeps its last value as long as it is not re-used

For $i = 1 to 10
   If $i = 5 Then Exitloop
Next
;...
Msgbox(0,"", $i)

:)

Share this post


Link to post
Share on other sites

Problem is solved!(for now).  Thank you for everyones help....  Got the time down to ~10 seconds max for the biggest files

Share this post


Link to post
Share on other sites

phatzilla,

Just for grins can you run your largest files through the following.  I'm interested in comparing the results to the dictionary method...

#include <FileConstants.au3>
#include <sqlite.au3>
#include <array.au3>

Local $sSQLiteMod = 'C:\Program Files (x86)\AutoIt3\install\Extras\SQLite\sqlite3.exe'

; do NOT use AutoIt macros for the next two file names

Local $sMasterFile = "fully.qualified.filename" ; your file with ALL entries
Local $sDetailFile = "fully.qualified.filename" ; your file with SOME entries

$st = TimerInit()

Local $sIn = 'create table if not exists mstr (c1);' & _
        'create table if not exists det (c1);' & _
        @CRLF & '.import ' & $sMasterFile & ' MSTR' & _
        @CRLF & '.import ' & $sDetailFile & ' DET' & _
        @CRLF & 'SELECT DISTINCT c1 FROM mstr WHERE c1 Not IN (SELECT DISTINCT c1 FROM det);'

Local $Out = _TempFile()
Local $in = _TempFile()

FileWrite($in, $sIn)

Local $sCmd = @ComSpec & ' /c "' & $sSQLiteMod & '" < ' & $in & ' > ' & $Out
Local $pid = RunWait($sCmd, @WorkingDir, @SW_HIDE)
ProcessWaitClose($pid)
ConsoleWrite('Time to find entries unique to mstr = ' & Round(TimerDiff($st) / 1000, 3) & ' seconds' & @CRLF)
ConsoleWrite(StdoutRead($pid) & @CRLF)
ConsoleWrite('! out ' & @CRLF & FileRead($Out) & @CRLF)

FileDelete($Out)
FileDelete($in)

kylomas


Forum Rules         Procedure for posting code

"I like pigs.  Dogs look up to us.  Cats look down on us.  Pigs treat us as equals."

- Sir Winston Churchill

Share this post


Link to post
Share on other sites

Hmm, i dont seem to have sqlite3.exe

Share this post


Link to post
Share on other sites

#19 ·  Posted (edited)

For this kind of handling, which requires to be fast enough, I use an external program from the CoreUtils for Windows (http://gnuwin32.sourceforge.net/packages/coreutils.htm) : uniq.exe. This tool can quickly extract/remove duplicate lines in a text file. In your case, you just have to put the 2 file contents in 1 file and remove the duplicates (it requires the file to be sorted first)

So, using the same idea than BrewManNH with _ArrayUniq, you can do something like this :

#Include <Array.au3>

Local $sMainFile = @ScriptDir & "\mainFile.txt"
Local $sPartialFile = @ScriptDir & "\secondFile.txt"

Local $sTempFile = @ScriptDir & "\tempFile.txt", $sResultFile = @ScriptDir & "\resultFile.txt"

; Concatenate the 2 files in 1
Local $sfile = FileWrite($sTempFile, FileRead($sMainFile) & FileRead($sPartialFile) )

; Sort the file and then remove all duplicate lines
RunWait(@ComSpec & ' /c sort "' & $sTempFile & '" | uniq -u > "' & $sResultFile & '"', "C:\Program Files (x86)\GnuWin32\bin", @SW_HIDE)
FileDelete($sTempFile)

Local $aResult = FileReadToArray($sResultFile)
_ArrayDisplay($aResult)

I know the problem is solved, but maybe someone else could be interested by this method...

Edited by jguinch
1 person likes this

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now

  • Similar Content

    • wakillon
      By wakillon
      Mp3SearchEngine v2.0.0.6

      May be some of you know Songr .
      This script do the same job, it can find more mp3 files but is not as fast as Songr.
       
       



      Sites used are music search engine Websites designed for LEGAL entertainment purposes only.
      Thanks to Brett Francis, Prog@ndy and Eukalyptus for >Bass Udf, trancex for >WinHttp Udf and the AutoIt Community for his help.

       
      Changes of v1.0.8.5
       
      Three websites replaced cause they are dead or use now js.
      All search engines updated ( not without difficulties for audiodump)
      I use RAGrid.dll for the first listview (more fast and stable, but with some inconvenients to manage the no-edit of cells)
      Input queries are saved ( the twenty latest)
      I use now an mp3 pre-Load management before playing and a double progressbar for visualize pre-load and play, where you can click for directly go play in the loaded part.
      Most includes needed are embedded and all external files are embedded in script with >BinaryToAu3Kompressor .
      Multi downloads available with embedded downloader.exe
       
      Changes of v1.0.8.8
      Search on audiodump and myfreemp3 fixed.
      New buttons.
      Added Gui Menu.
      Titles are no more editable.
      New "About" with >TaskDialog (Thanks Prog@andy)
      Query button permit now to check / uncheck all checkboxes
      And some few fixes and cleaning.
      Really more stable now.
      Changes of v1.0.9.2
      Dilandau is replaced by mp3chief and mp3ili by mp3clan 
      Search on mp3juices, baseofmp3 and soundcloud fixed.
      Soso now provide m4a (aac) instead of mp3 ( m4a can be played by MSE)
      Added possibility to encode automaticaly to mp3, aac or ogg ( at the end of download) using bassenc.dll and command line tools : lame, faac and oggenc.
       
      Changes of v1.0.9.3   mp3skull fixed mp3chief fixed myfreemp3 fixed mp3clan changed to tusmp3  mp3juices changed to emp3world baseofmp3 changed to imp3 and some minor improvements.  
      Version 2.0.0.6
      Most previous websites used are dead or have changed the way to get links, 
      so instead of try to repair the previous version, i have created a complete new version.
      The main tendency is the simplification :
      Only one website : audiodump (Up to 500 results by request)
      Script use now the little pearl created by Ward : curl.au3
      It permit to create tasks (get source and get multi mp3) in asynchronous mode.
      So now, no need to use several executables and no more gui who do not respond in case of connection problems. 
      Script use Bass.dll X86 loaded in memory for play songs.
      Result is light and fast, but don't abuse of audiodump servers who are not beasts of race.
      Warning : For avoid errors with curl.au3, you'll need to comment the line 63 : ;~ #Include <BinaryCall.au3>
      @AutoItX64 not supported and only tested on Win7X64 and Win8.1X64.
      As your browser, use Ctrl+w for remove the current Tab.(if there is no search or download running from it)
      And also Ctrl+q for set/remove Gridlines.
      Events are displayed to the bottom of the Gui.
       
      Version 2.0.1.1
      Added a Paste Button.
      Querry list is now correctly saved.
      Querry Combo is now sorted in alphabetical order
      After a 'No match', the next search will use the previous empty listview.
      Bug when removing tabs is corrected.
      Added string correction for the request that, in the previous version, was not always able to return a correct result.
       
      A big thanks to Ward for his great UDF, and Nina my favorite tester, (who between us is also my third daughter), for his precious advices .
      previous downloads : 1703
       
      As there is no more script downloads count, source and executable are available in the downloads section

      Enjoy ! 
      July 2017 Project Discontinued due to website changes
    • Jibberish
      By Jibberish
      I am trying to add data to an array and I keep getting the error "Subscript used on non-accessible variable ".
      #include <array.au3> ; Includes, etc ~ ~ ~ Local $aOptionsArray[3] $aOptionsArray = CheckboxesAndRadioButtons() _ArrayDisplay($aOptionsArray) Func CheckboxesAndRadioButtons() ; Create Checkboxes and Radio Buttons and read the results ~ ~ ~ ; Radio Buttons to Array ; $aOptions[0] = $bTestSelectForever ; $aOptions[1] = $bTestSelect3Times ; $aOptions[2] = $bTestSelectOnce If $bSelect1 = 1 Then $aOptions[0] = "True" Else $aOptions[0] = "False" <<< This is where the error occurs. EndIf If $bSelect2 = 1 Then $aOptions[1] = "True" Else $aOptions[1] = "False" EndIf If $bSelect3 = 1 Then $aOptions[2] = "True" Else $aOptions[2] = "False" EndIf Return $aOptions EndFunc Is putting data into an array while in a If - Then - Else illegal?
    • anoig
      By anoig
      Hi all, 

      First, I want to give a huge shout-out to the community. I'm completely self-taught, and have never had to actually ask a question before because the forum is that good at answering questions and explaining things. However, I'm kind of stumped here, and I've been stuck on this problem for almost a full day.

      I'm working on a script to populate drafts of deeds at work. I have the main GUI and  a function (ctrl($n) and read()) for adding fields to find and data to replace it with to an array for later use with _word_docfindreplace. All of that works. However, due to the way I have the forms set up, I need to create additional fields and info based on the data that's there. Specifically, if there's only one buyer, I need to add the field [Buyer1&2] and the data in $aArray_Base for [Buyer 1]. I also need to add a field [Buyer 2] and have a blank data set in the next column over in the array, and I need to do the same for the Seller. To this end, the function parties() sets boolean variables $2buyers and $2sellers accordingly. Then, I have buyers() and sellers() to populate the data. 

      The problem that I'm running into is that each function works... when ONLY the buyer 1 name field is filled, and when ONLY the seller 1 field is filled.

      So if I fill Buyer 1 Name and save it, the data is populated correctly. But when I fill Buyer 1 and Seller 1 name, only the buyer 1 data populates correctly. Worse, when I fill several fields, neither populate correctly. I have no idea why this happens. I've added messageboxes to debug throughout the entire process and can't pinpoint what's causing the issue. The entire script is below. The function(s) in question are buyers() and sellers(). Only Sellers() has messageboxes throughout.

      Can someone help walk me through what might be causing this and help me find a solution? Thanks a ton in advance, and sorry for the wall of text.
      -Anoig
       
    • InunoTaishou
      By InunoTaishou
      Cleaning up some old folders and found this little snippit. Think I was using this back when I was trying to create 2d maps for a game I was making in GDI+ (that never got finished). Pretty straight forward, searches an array for an array.
      #include <Array.Au3> Search() Func Search() Local $aArrayToFind[][] = [["V", "V", "V", "V", "V"], ["V", "V", "V", "V", "V"], ["V", "V", "V", "V", "B"], ["V", "V", "V", "V", "B"], ["V", "V", "V", "V", "B"]] Local $aArrayToSearch[][] = [["R", "R", "R", "R", "R", "R", "R", "R", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "X", "V", "V", "V", "V", "V", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T"], _ ["R", "R", "R", "R", "R", "R", "R", "R", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "X", "V", "V", "V", "V", "X", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T"], _ ["R", "R", "R", "R", "R", "R", "R", "R", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "X", "V", "V", "V", "V", "V", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T"], _ ["R", "R", "R", "R", "R", "R", "R", "R", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "X", "V", "V", "V", "V", "V", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T"], _ ["R", "R", "R", "R", "R", "R", "R", "R", "X", "X", "X", "X", "X", "X", "X", "X", "X", "X", "X", "V", "V", "V", "V", "V", "T", "T", "T", "T", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "T", "T", "T", "T", "T", "T"], _ ["R", "R", "R", "R", "R", "R", "R", "R", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "T", "T", "T", "T", "T", "T"], _ ["R", "R", "R", "R", "R", "R", "R", "R", "V", "X", "T", "T", "T", "T", "T", "T", "T", "T", "X", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "T", "T", "T", "T", "T", "T"], _ ["R", "R", "R", "R", "R", "R", "R", "R", "V", "X", "T", "T", "T", "T", "T", "T", "T", "T", "X", "X", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "B", "B", "B", "B", "B", "B", "V", "V", "V", "T", "T", "T", "T", "T", "T"], _ ["R", "R", "R", "R", "R", "R", "R", "R", "V", "X", "T", "T", "T", "T", "T", "T", "T", "T", "X", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "B", "B", "B", "B", "B", "B", "V", "V", "V", "T", "T", "T", "T", "T", "T"], _ ["R", "R", "R", "R", "R", "R", "R", "R", "V", "X", "T", "T", "T", "T", "T", "T", "T", "T", "X", "X", "V", "V", "V", "X", "B", "B", "B", "B", "B", "V", "V", "V", "V", "B", "B", "B", "B", "B", "B", "V", "V", "V", "T", "T", "T", "T", "T", "T"], _ ["R", "R", "R", "R", "R", "R", "R", "R", "V", "X", "T", "T", "T", "T", "T", "T", "T", "T", "X", "V", "V", "V", "V", "V", "B", "B", "B", "B", "B", "V", "V", "V", "X", "V", "V", "B", "D", "B", "V", "V", "V", "V", "T", "T", "T", "T", "T", "T"], _ ["R", "R", "R", "R", "R", "R", "R", "R", "V", "X", "T", "T", "T", "T", "T", "T", "T", "T", "X", "X", "V", "V", "V", "X", "B", "D", "B", "B", "B", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "T", "T", "T", "T", "T", "T"], _ ["R", "R", "R", "R", "R", "R", "R", "R", "V", "X", "T", "T", "T", "T", "T", "T", "T", "T", "X", "V", "V", "V", "V", "V", "V", "V", "V", "X", "X", "L", "L", "L", "L", "L", "L", "L", "L", "L", "L", "L", "L", "L", "T", "T", "T", "T", "T", "T"], _ ["R", "R", "R", "R", "R", "R", "R", "R", "V", "X", "T", "T", "T", "T", "T", "T", "T", "T", "X", "X", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "T", "T", "T", "T", "T", "T"], _ ["R", "R", "R", "R", "R", "R", "R", "R", "V", "X", "T", "T", "T", "T", "T", "T", "T", "T", "X", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "T", "T", "T", "T", "T", "T"], _ ["R", "R", "R", "R", "R", "R", "V", "V", "V", "X", "T", "T", "T", "T", "T", "T", "T", "T", "X", "X", "V", "V", "V", "V", "V", "V", "V", "V", "V", "X", "X", "X", "X", "X", "X", "X", "X", "X", "X", "X", "X", "X", "T", "T", "T", "T", "T", "T"], _ ["V", "V", "V", "V", "V", "V", "V", "V", "V", "X", "X", "X", "X", "X", "X", "X", "X", "X", "X", "V", "X", "V", "V", "X", "B", "B", "B", "B", "B", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "T", "T", "T", "T", "T", "T"], _ ["V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "B", "B", "B", "B", "B", "V", "V", "V", "V", "V", "M", "M", "M", "M", "V", "V", "V", "V", "T", "T", "T", "T", "T", "T"], _ ["V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "X", "B", "D", "B", "B", "B", "V", "V", "V", "V", "V", "M", "M", "M", "M", "V", "V", "V", "V", "T", "T", "T", "T", "T", "T"], _ ["V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "M", "M", "D", "M", "V", "V", "V", "V", "T", "T", "T", "T", "T", "T"], _ ["R", "R", "R", "R", "R", "R", "R", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "T", "T", "T", "T", "T", "T"], _ ["R", "R", "R", "R", "R", "R", "R", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "T", "T", "T", "T", "T", "T"], _ ["R", "R", "R", "R", "R", "R", "R", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "T", "T", "T", "T", "T", "T"], _ ["R", "R", "R", "R", "R", "R", "R", "T", "T", "T", "T", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "C", "C", "C", "C", "C", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "T", "T", "T", "T", "T", "T"], _ ["R", "R", "R", "R", "R", "R", "R", "T", "T", "T", "T", "V", "T", "T", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "C", "C", "C", "C", "C", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "T", "T", "T", "T", "T", "T"], _ ["R", "R", "R", "R", "R", "R", "R", "V", "V", "V", "V", "V", "T", "T", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "C", "C", "C", "C", "C", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "T", "T", "T", "T", "T", "T"], _ ["R", "R", "R", "R", "R", "R", "R", "V", "V", "V", "V", "W", "W", "W", "W", "W", "W", "V", "V", "V", "V", "V", "V", "V", "C", "C", "D", "C", "C", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "T", "T", "T", "T", "T", "T"], _ ["R", "R", "R", "R", "R", "R", "R", "V", "V", "V", "V", "W", "W", "W", "W", "W", "W", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "T", "T", "T", "T", "T", "T"], _ ["R", "R", "R", "R", "R", "R", "R", "V", "V", "V", "V", "W", "W", "W", "W", "W", "W", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "T", "T", "T", "T", "T", "T"], _ ["R", "R", "R", "R", "R", "R", "R", "V", "V", "V", "V", "W", "W", "W", "W", "W", "W", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "T", "T", "T", "T", "T", "T"], _ ["R", "R", "R", "R", "R", "R", "R", "L", "L", "L", "L", "W", "W", "W", "W", "W", "W", "L", "L", "L", "V", "V", "V", "L", "L", "L", "L", "L", "L", "L", "L", "L", "L", "L", "L", "L", "L", "L", "L", "L", "L", "L", "T", "T", "T", "T", "T", "T"], _ ["R", "R", "R", "R", "R", "R", "R", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "X", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "T", "T", "T", "T", "T", "T"], _ ["R", "R", "R", "R", "R", "R", "R", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "T", "T", "T", "T", "T", "T"], _ ["R", "R", "R", "R", "R", "R", "R", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "T", "T", "T", "T", "T", "T"], _ ["R", "R", "R", "R", "R", "R", "T", "T", "T", "T", "T", "T", "T", "T", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "T", "T", "T", "T", "T", "T"], _ ["R", "R", "R", "R", "R", "R", "T", "T", "T", "T", "T", "T", "T", "T", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "V", "T", "T", "T", "T", "T", "T"], _ ["R", "R", "R", "R", "R", "R", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "V", "V", "V", "V", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T"], _ ["R", "R", "R", "R", "R", "R", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "V", "V", "V", "V", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T"], _ ["R", "R", "R", "R", "R", "R", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "V", "V", "V", "V", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T"], _ ["R", "R", "R", "R", "R", "R", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "V", "V", "V", "V", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T", "T"]] Local $aArrayInArray = ArrayInArray($aArrayToSearch, $aArrayToFind) If (@Error) Then MsgBox("", "Error", "Error doing search: " & @Error) Else If ($aArrayInArray[0] = 1) Then ToolTip("Start Coords = " & $aArrayInArray[1] & ", " & $aArrayInArray[2] & @LF & _ "End Coords = " & $aArrayInArray[1] + UBound($aArrayToFind, $UBOUND_ROWS) - 1 & ", " & $aArrayInArray[2] + UBound($aArrayToFind, $UBOUND_COLUMNS) - 1 & @LF & _ "(Approx) Center coords = " & $aArrayInArray[1] + Int(UBound($aArrayToFind, $UBOUND_ROWS) / 2) & ", " & $aArrayInArray[2] + Int(UBound($aArrayToFind, $UBOUND_COLUMNS) / 2), 0, 0) $aArrayToSearch[$aArrayInArray[1]][$aArrayInArray[2]] = "Start" $aArrayToSearch[$aArrayInArray[1] + UBound($aArrayToFind, $UBOUND_ROWS) - 1][$aArrayInArray[2] + UBound($aArrayToFind, $UBOUND_COLUMNS) - 1] = "End" $aArrayToSearch[$aArrayInArray[1] + Int(UBound($aArrayToFind, $UBOUND_ROWS) / 2)][$aArrayInArray[2] + Int(UBound($aArrayToFind, $UBOUND_COLUMNS) / 2)] = "(Approx) Center" _ArrayDisplay($aArrayToSearch, "ArrayInArray") ElseIf ($aArrayInArray[0] > 1) Then MsgBox("", "Multiple Matches", "Multiple Matches Found For Search") Else MsgBox("", "No Matches", "No Matches Found For Search") EndIf EndIf EndFunc ;==>Start Func ArrayInArray($aArrayToSearch, $aArrayToFind) Local $aReturn[3] = [0, -1, -1] If (Not IsArray($aArrayToSearch)) Then Return SetError(1, 0, $aReturn) If (Not IsArray($aArrayToFind)) Then Return SetError(2, 0, $aReturn) Local $iRowsToSearch = UBound($aArrayToSearch, $UBOUND_ROWS) Local $iColumnsToSearch = UBound($aArrayToSearch, $UBOUND_COLUMNS) Local $iRowsToFind = UBound($aArrayToFind, $UBOUND_ROWS) Local $iColumnsToFind = UBound($aArrayToFind, $UBOUND_COLUMNS) If ($iRowsToFind > $iRowsToSearch) Then Return SetError(3, 0, $aReturn) If ($iColumnsToFind > $iColumnsToSearch) Then Return SetError(4, 0, $aReturn) For $iRow = 0 To $iRowsToSearch - $iRowsToFind For $iColumn = 0 To $iColumnsToSearch - $iColumnsToFind Local $bValid = False For $i = 0 To $iRowsToFind - 1 For $p = 0 To $iColumnsToFind - 1 If ($aArrayToFind[$i][$p] = "" Or $aArrayToFind[$i][$p] = $aArrayToSearch[$iRow + $i][$iColumn + $p]) Then $bValid = True Else $bValid = False ExitLoop 2 EndIf Next Next If ($bValid) Then ; Number of valid results found $aReturn[0] += 1 ; row of the last valid result found $aReturn[1] = $iRow ; column of the last valid result found $aReturn[2] = $iColumn EndIf Next Next Return $aReturn EndFunc ;==>ArrayInArray ArrayInArray Returns an array: [0] = number of results found, [1] = row of the last valid result found, [2] = column of the last valid result found
    • distancesprinter
      By distancesprinter
      _ArrayDisplay($aArray, "Window Title", "1:", 0, Default, "Column") ; Expected results are rows 1 to the end of the array, all columns. The result is rows 0-1, all columns. The API reference is here:
      https://www.autoitscript.com/autoit3/docs/libfunctions/_ArrayDisplay.htm
       
      Am I doing something wrong?