Sign in to follow this  
Followers 0
Read

Finding combinations and remove line

92 posts in this topic

#1 ·  Posted (edited)

I have very BIG .txt file and PHP script not enough - very slow.

Welll....

I have the combinations in the document (.txt)

01 02 07 14 32 37

01 02 07 14 32 36

01 02 07 14 32 35

01 03 05 06 07 44

01 03 05 06 07 43

01 03 05 06 07 42

01 04 05 06 07 16

PHP:

$find= array(array(01, 02, 14, 37), array(01, 03, 06, 07, 43));

Script searches for the document and if found numbers (array) then delete/remove this combination - line.

Example:

Script searching and found: 01 02 07 14 32 37 and 01 03 05 06 07 43

01 02 07 14 32 37 (DELETE)

01 02 07 14 32 36

01 02 07 14 32 35

01 03 05 06 07 44

01 03 05 06 07 43 (DELETE)

01 03 05 06 07 42

01 04 05 06 07 16

How to make it?

ENG bad :)

Thank you

Edited by Read

Share this post


Link to post
Share on other sites



Read,

How big is "big" and how do you specify the string(s) to delete?

kylomas


Forum Rules         Procedure for posting code

"I like pigs.  Dogs look up to us.  Cats look down on us.  Pigs treat us as equals."

- Sir Winston Churchill

Share this post


Link to post
Share on other sites

File is 200MB - big Yes?

String: Don't know. Maybe you know better, how to find and delete line :)

Share this post


Link to post
Share on other sites

Read,

Working something up...but there are a bunch of regex heavy weights who will probably have something for you before I am done typing this...

kylomas


Forum Rules         Procedure for posting code

"I like pigs.  Dogs look up to us.  Cats look down on us.  Pigs treat us as equals."

- Sir Winston Churchill

Share this post


Link to post
Share on other sites

Read,

Is there a min and max value for the numbers?

Are there always 6 numbers per string?

Are the numbers always 2 chars long?

kylomas


Forum Rules         Procedure for posting code

"I like pigs.  Dogs look up to us.  Cats look down on us.  Pigs treat us as equals."

- Sir Winston Churchill

Share this post


Link to post
Share on other sites

#6 ·  Posted (edited)

Hard my English!!!

Combinations is 6 numbers

01 02 07 14 32 37

01 02 07 14 32 36

Array is 5

$find= array(array(01, 02, 03, 04, 05));

Edit:

Are the numbers always 2 chars long? You mean "array"? Then 900

Edited by Read

Share this post


Link to post
Share on other sites

Read,

I am not understanding you. What do you mean by array is 5? Show the format for the .txt file and we'll start from there.

kylomas


Forum Rules         Procedure for posting code

"I like pigs.  Dogs look up to us.  Cats look down on us.  Pigs treat us as equals."

- Sir Winston Churchill

Share this post


Link to post
Share on other sites

@Read

This is how I understand it:

Every line of the file has 6 numbers. The array has 5 numbers. Every line that contains all of the 5 numbers should be deleted.

I'm not sure if "Then 900" means that the numbers can be 3 characters ("900") long.

Is this correct?


My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2017-04-18 - Version 1.4.8.0) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX (NEW 2017-02-27 - Version 1.3.1.0) - Download - General Help & Support - Example Scripts - Wiki
ExcelChart (2015-04-01 - Version 0.4.0.0) - Download - General Help & Support - Example Scripts
Excel - Example Scripts - Wiki
Word - Wiki
PowerPoint (2015-06-06 - Version 0.0.5.0) - Download - General Help & Support

Tutorials:
ADO - Wiki

 

Share this post


Link to post
Share on other sites

Water!

Yes! Every line of the file has 6 numbers. The array has 5 numbers. Every line that contains all of the 5 numbers should be deleted.

I can be defined, 20 or 40 or 150 but a total of 900.

My PHP script I used Array. AutoIT- don't know.

Share this post


Link to post
Share on other sites

What do you mean by "I can be defined, 20 or 40 or 150 but a total of 900."?

Is this the number of records in the file or the number of entries in the array?


My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2017-04-18 - Version 1.4.8.0) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX (NEW 2017-02-27 - Version 1.3.1.0) - Download - General Help & Support - Example Scripts - Wiki
ExcelChart (2015-04-01 - Version 0.4.0.0) - Download - General Help & Support - Example Scripts
Excel - Example Scripts - Wiki
Word - Wiki
PowerPoint (2015-06-06 - Version 0.0.5.0) - Download - General Help & Support

Tutorials:
ADO - Wiki

 

Share this post


Link to post
Share on other sites

Right! or the number of entries in the array.

Share this post


Link to post
Share on other sites

Read,

Can you express this is psuedo-logic?

kylomas


Forum Rules         Procedure for posting code

"I like pigs.  Dogs look up to us.  Cats look down on us.  Pigs treat us as equals."

- Sir Winston Churchill

Share this post


Link to post
Share on other sites

kylomas!

What is psuedo-logic? Logically explain?

My english is limited :)

Share this post


Link to post
Share on other sites

Read,

Yes, I understand (and am very patient).

Like: .txt file format = nn nn nn nn nn nn (per line - line is terminated by @crlf)

where: nn = any number between 1 and 99

each line always contains 6 numbers and numbers do NOT repeat within a single line

logic: get number string to delete

open file

find number string in file and delete

close file

Something similar to the above.

kylomas


Forum Rules         Procedure for posting code

"I like pigs.  Dogs look up to us.  Cats look down on us.  Pigs treat us as equals."

- Sir Winston Churchill

Share this post


Link to post
Share on other sites

Read,

This just ocurred to me. If you have a working example in PHP, post that so that we can see the logic and possibly the file format.

kylomas


Forum Rules         Procedure for posting code

"I like pigs.  Dogs look up to us.  Cats look down on us.  Pigs treat us as equals."

- Sir Winston Churchill

Share this post


Link to post
Share on other sites

Kylomas!

Something like this!!!!

Share this post


Link to post
Share on other sites

Read,

Are you trying to post something?

What does this mean?

Something like this!!!!

kylomas


Forum Rules         Procedure for posting code

"I like pigs.  Dogs look up to us.  Cats look down on us.  Pigs treat us as equals."

- Sir Winston Churchill

Share this post


Link to post
Share on other sites

#18 ·  Posted (edited)

Try this:

#include <array.au3>
$sComb = StringReplace(StringStripCR(FileRead(@ScriptDir & "Combi.txt")), @LF, "|")  & "|"

$aComb = StringSplit($sComb, "|", 2)

$sNd1 = "01 02 07 14 32 37"
$sNd2 = "01 03 05 06 07 43"

_ArrayDisplay($aComb)
SearchAndDestroy($aComb, $sNd1)
SearchAndDestroy($aComb, $sNd2)

_ArrayDisplay($aComb)
$hFile = FileOpen(@ScriptDir & "Combi-New.txt", 2)
FileWrite($hFile, _ArrayToString($aComb, @LF))
FileClose($hFile)

Func SearchAndDestroy(ByRef $array, $search)
    If Not IsArray($array) Then Return SetError(1, 0, 0)
    Local $ai = _ArrayFindAll($array, $search)
    If @error Then Return SetError(2, @error, 0)
    Local $i
    For $i = 0 To UBound($ai) - 1
        _ArrayDelete($array, $ai[$i] - $i)
    Next
    Return 1
EndFunc

Combi.txt is the file with the combinations.

imho 200 mb is not really big.

Br,

UEZ

Edit: changed the code -> removed StringRegEx and added _ArrayFindAll() to catch all search strings

Edited by UEZ

Please don't send me any personal message and ask for support! I will not reply!

Selection of finest graphical examples at Codepen.io

The own fart smells best!
Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!
¯\_(ツ)_/¯  ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ

Share this post


Link to post
Share on other sites

UEZ

200 mb is not really big

It's big for PHP :)

Code.

Something is wrong.

My .txt file and comb..

01 02 07 14 32 37

01 02 07 14 32 36

01 02 07 14 32 35

01 03 05 06 07 44

01 03 05 06 07 43

01 03 05 06 07 42

01 04 05 06 07 16

and array:

$sNd1 = "01 02 14 32 37"

Right now must delete 01 02 07 14 32 37 not 01 04 05 06 07 16

Share this post


Link to post
Share on other sites

Read,

You want to delete any entry in your source file that contains ALL numbers in $sNd1? (could be any number of digits?)

kylomas


Forum Rules         Procedure for posting code

"I like pigs.  Dogs look up to us.  Cats look down on us.  Pigs treat us as equals."

- Sir Winston Churchill

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0