Jump to content

Array extract - removing rows that has a blank field


Recommended Posts

Hi guys,

See attached for an array example.

to simplify what i want to achieve,  I want to split this array into 9 different csv files.

the first file would contain the list of "key" and the corresponding "ID1", the second would have "key" and the "ID2", the third would have "key" and "ID3"

However, I want to remove all the rows that don't have an ID recorded in the respective ID2, ID3 4...5...6 etc, so the file only contains row items with a key and the ID.

Would be the best way to loop through the rows and delete the row if the array field is blank - would I then need to repeat that row ID to check that the row that its replaced is also empty (ie the one after the one I just deleted)? I see this getting messy.

or _arraySort, and delete everything below the last filled row? <-- this might be best?

Or should I use the excel UDF, apply a filter (not selecting the blanks), then create/export to the array->csv?

 

Super keen to hear your thoughts.

thanks!

 

 

 

 

IDArray.csv

Link to post
Share on other sites

so, i currently do that, to build the array in its entirety, its the looping to clean that I'm not sure about. Noting that the full data set has 200k rows. Will this be a problem for an array?

Edited by MrCheese
Link to post
Share on other sites
6 minutes ago, MrCheese said:

I thought about things logically earlier, but wanted some input before I went forth.

_FileReadToArray(), array variables, For...Next loop with Step -1 ( from the "bottom" to the "top" of the array ), and some If conditions :)
I've already made a solution, but let's see yours :)

Click here to see my signature:

Spoiler

ALWAYS GOOD TO READ:

 

Link to post
Share on other sites

For the fun  :)

$s = FileRead("IDArray.csv")

For $i = 0 to 8
  $k = StringRegExpReplace($s, '(?imx)^(\d+|key) , (?:\w*,){' & $i & '} , .*$\R?', "")
  $k = StringRegExpReplace($k, '(?imx)^(\d+|key) , (?:\w*,){' & $i & '} (\w+) .*$', "$1,$2")
 ; Msgbox(0,"", $k)
  FileWrite("IDArray_ID" & $i+1 & ".csv", $k)
Next

Will this work on a 200k rows text ? I don't know ...     o:)

Link to post
Share on other sites
4 hours ago, FrancescoDiMuro said:

_FileReadToArray(), array variables, For...Next loop with Step -1 ( from the "bottom" to the "top" of the array ), and some If conditions :)
I've already made a solution, but let's see yours :)

Thanks! yes, starting from the bottom! Removes the issue I was wondering about. I'll have a solution tomorrow morning.

Link to post
Share on other sites

also, if the ID# portion itself does not matter, just the contents

$s = FileRead("IDArray.csv")

msgbox(0, '' , stringregexpreplace(StringTrimLeft($s , 41) , ",+" , ","))

 

Edited by iamtheky

,-. .--. ________ .-. .-. ,---. ,-. .-. .-. .-.
|(| / /\ \ |\ /| |__ __||| | | || .-' | |/ / \ \_/ )/
(_) / /__\ \ |(\ / | )| | | `-' | | `-. | | / __ \ (_)
| | | __ | (_)\/ | (_) | | .-. | | .-' | | \ |__| ) (
| | | | |)| | \ / | | | | | |)| | `--. | |) \ | |
`-' |_| (_) | |\/| | `-' /( (_)/( __.' |((_)-' /(_|
'-' '-' (__) (__) (_) (__)

Link to post
Share on other sites
1 hour ago, mikell said:

Will this work on a 200k rows text ? I don't know ...   

i am nearly sure as none off these limits:

Quote
VAR_SUBSCRIPT_ELEMENTS 16,777,216 Maximum number of elements for an array.
VAR_SUBSCRIPT_MAX 64 Maximum number of subscripts for an array.

is reached (by my interepreting the opening post).

Link to post
Share on other sites
2 hours ago, AutoBert said:

i am nearly sure as none off these limits is reached

Hmm. This

$t = TimerInit()
$s = FileRead("IDArray.csv") ; currently 38 lines
Local $r
For $i = 1 to 100000
  $r &= $s & @crlf
Next
$s = $r

For $i = 0 to 8
  $k = StringRegExpReplace($s, '(?imx)^(\d+|key) , (?:\w*,){' & $i & '}, .*$\R?', "")
  $k = StringRegExpReplace($k, '(?imx)^(\d+|key) , (?:\w*,){' & $i & '}(\w+) .*$', "$1,$2")
;  Msgbox(0,"", $k)
;;;;;;;  FileWrite("IDArray_ID" & $i+1 & ".csv", $k)
Next
Msgbox(0,"", TimerDiff($t)/1000 )

works, but if I try more then I get "error allocating memory"
Anyway more than 3800k lines means a huge amount of data, so it would surely be a much better way to manage this using a SQLite database  :)

Edited by mikell
Link to post
Share on other sites
5 minutes ago, FrancescoDiMuro said:

It could be definitely a better way to remove blank rows!

It depends on the data, the needed use in the script, and so on  :)
The regex way is probably much faster than using the _Array* funcs. On the example above I didn't try the latter but it should give a little more than 40s I guess  ;)

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By TheAlienDoctor
      Hi, I was looking into creating a script that would detect if a file exists, then move it (and in some cases rename it, depending on the file) as well as write to a log file.
      The issue is, there is a lot of files that need to be moved, sometimes some files will exist and others won't depending on the use-case. However if a file does exist, it will always be going into the same directory with the same name.
      Currently I have an array nested inside of the array, with each array inside that array having both the old and new directory, and then a For loop to actually run through and do the file transferring. The issue I am having is how to call the Array inside of the array, because how do I specify  which the old directory is and which the new is?
      Global $FileTransfer[2000] = [Global $Dir1[2] = ["original dir 1", "new dir 1"], Global $Dir2[2] = ["original dir 2", "new dir 2"]] For $FileTransfer = [0] To [1] Step +1 If FileExists({original dir}) Then FileMove({original dir}, new dir, 1) FileOpen("log.latest.txt", 1) FileWrite("log.latest.txt", "{original dir} found, moved it to new dir." & @CRLF) FileClose("log.latest.txt") Else FileOpen("log_latest.txt", 1) FileWrite("log_latest.txt", "{original dir} not found, ignoreing it." & @CRLF) FileClose("log_latest.txt") EndIf Next I have put what I want the old and new directory to be for each array in {}, so hopefully its easier to tell which part is working and whats not.

      I am still reasonably new to AutoIT, any help is appreciated. Thankyou
    • By arunkw
      I have a spreadsheet - daily routine which has two columns: activity and time as shown here
      | Activity             | Time     | |----------------------|----------| | Sleep               |  6:00 am | | Toilet              |  6:15 am | | Get ready for gym  |  6:30 am | | Exercise            |  7:50 am | | ... more things      |  9:00 pm | | ... still more       | 10:45 pm | | Sleep               |  6:00 am |   I wanted to find out, say in C1 which activity is current for me using now() I.e., if it’s 6:45am on my watch, it should show me Exercise  in C1 Thanks to Adam D. PE, this formula works like magic to get the result =VLOOKUP(MOD(NOW(),1),{B2:B,A2:A},2,1)   Now, I want to reproduce same result in autoit, how to do that? To have easy solution say, I copy-paste spreadsheet data in array directly in code, right? Use for loop and run the above vlookup function and show the answer using tooltip. How to achieve this? please help.  
    • By goku200
      I have an Autoit script that lists files from a folder into an array list. Is there a way to separate the filenames by an underscore and include the id, version, name and date into separate columns in Excel.
      Example of filename:
      12345_v1.0_TEST Name [12345]_01.01.2022.html
      12345 would be in one column
      v1.0 would be in another column
      TEST Name [12345] would be in another column
      01.01.2022 would be in another column
      .html would be in another column
      Note: filenames always change each day.
      Here is my code that lists the files into column C and then writes the column Headers into Column D, E, F, G. Just need some help with separating them into columns by the _ delimiter
       
    • By mLipok
      Usually when I collect data from DataBase I need to give EndUser a possibility to select rows which should be taken in the processing loop.
      I was searching on the forum and I'm not able to find any UDF or even example of how to select data from array.
      I have my own solutions but I think they are not worth posting on the forum as it is very old code and I am looking for a better solution.

      Could anybody point me to some examples/solutions ?

      Thank you in advance.
      @mLipok
    • By EmilyLove
      I have a string containing the full path of an executable and an array of executables without their paths. I am trying to compare the string to the list in the array and if a match is found, remove it from the array. The entry get removed from the array successfully, and after checking its return result, uses it to update the ubound if it succeeded, but it doesn't want to update to the new value. Any ideas what I am doing wrong? It acts like it is read-only.
      #include <Array.au3> #include <File.au3> Local $sApp_Exe = "F:\App\Nextcloud\nextcloud.exe" Local $aWaitForEXEX = [3, "Nextcloud.exe", "nextcloudcmd.exe", "QtWebEngineProcess.exe"] For $h = 1 To $aWaitForEXEX[0] If StringInStr($sApp_Exe, $aWaitForEXEX[$h]) <> 0 Then $iRet = _ArrayDelete($aWaitForEXEX, $h) If $iRet <> -1 Then $aWaitForEXEX[0] = $iRet ;this line doesn't work. $aWaitForEXEX[0] doesn't update and shortly gives Error: Array variable has incorrect number of subscripts or subscript dimension range exceeded.: _ArrayDisplay($aWaitForEXEX) EndIf Next  
×
×
  • Create New...