Storing data

[Phaser]

Hi everyone

I have 8500 sets of data in this format 01,99,83,27,12,30,114,299, and so on, I need to search this data so I thought of storing it in an array, $data[8500]=[01,99,83,27,12,30,114,299,....................] looks like an array wont hold that much data, any suggestions?

I will be searching it later with current data in chunks like this

$getdatamatch = "99,83"

Any pointers as to how I can proceed?

[99ojo]

Hi,

it's not really clear for me, what you have.

Have you 8500 numbers: 01,99,83,27,12,30........ -> amount 8500 -> This isn't a problem, arrays can store 2^24 (16 777 216).

or

have you 8500 datasets like:

1. dataset: 01,99,83,27,12,30........

2. dataset: 02,77,23,09,88 ....

.

.

.

8500 dataset: .....

Next question is, how you get your data / sets? From file, from database......

;-))

Stefan

Edited May 20, 2010 by 99ojo

[Phaser]

Hi Stefan

The data is currently in a .txt file so it is one long line of data seperated by a comma , I tried to put it all as an array

$data[8500]=[01,99,83,27,12,30,02,77,23,09,88........] ---->>>> 8500 total indices

but it gave an error in the array pointing to somewhere around 1500 indices into the array, which made me think the data was too long

Missing right bracket ')' in expression.:

Am I doing it correctly?

[Tvern]

Try

$hFile = FileOpen("file.txt",)
$sData = FileRead($hFile)
FileClose($hFile)
$aData = StringSplit($sData,",")

[Phaser]

Hi

I dont get anything out from that??

MsgBox(0,"box name", $aData)

I wasn't trying to read the txt file, as I had used all those numbers in the array that gave the error, wont the array be the best way so I can search it later?

[Melba23]

Phaser,

I would think that you are running into the AutoIt line limit of 4095 characters when you declare your array. At 3 chars per element, that would give you an error about 1500 elements in.

Try splitting the declaration like this - although you can make each "line" much longer:

$data[8500]=[01,99,83,27,12, _
    30,02,77,23,09,55,35,27, _
    88........]

Just make sure that each "line" is less that 4095 characters long.

M23

[99ojo]

Hi,

add #include <array.au3> at the beginning of your script.

Change MsgBox(0,"box name", $aData) to _ArrayDispplay ($aData), because StringSplit returns an array.

Jumping on Tvern's code:

#include <array.au3>
$hFile = FileOpen("file.txt",)
$sData = FileRead($hFile)
FileClose($hFile)
$aData = StringSplit($sData,",")
_ArrayDispplay ($aData)

;-))

Stefan

[AndyG]

Hi, searching in arrays is slower than in strings.

You could "transform" the string in the file into an array with stringsplit()

$data = FileRead("test.txt")   ;file with CSV
$array=stringsplit($data,",",3) ;splits string into an array

$search = "02,77,23"  ;searchstring

;search in text
$pos = StringInStr($data, $search, 1, 1)
If $pos Then  ;string is found
    MsgBox(0, "StringInstr()", "Pattern " & $search & " found at position " & $pos)
Else
    MsgBox(0, "StringInstr()", "Pattern " & $search & " not found")
EndIf

;search
$searcharray=stringsplit($search,",",3)
$foundflag=0
for $i=0 to ubound($array)-1-ubound($searcharray)  ;all numbers in the array
    if $searcharray[0]=$array[$i] then ;first number match
for $x=1 to ubound($searcharray)-1   ;compare the numbers in the array with the numbers in the searchstring
if $searcharray[$x]<>$array[$i+$x] then ContinueLoop 2 ;if one of the following numbers doesnt match, then next number in array
        Next
        msgbox(0,"Data","Pattern " & $search & " found at array[" & $i&"] and following")
        $foundflag=1 ;flag that pattern has matched
        ExitLoop
    EndIf
next
if $foundflag=0 then msgbox(0,"Data","Pattern " & $search & " not found in array")

 

[Phaser]

Thank you very much AndyG, I think I can work with what you have posted

@ other members, thanks for helping out, AndyG's code may do everything I was after.

[Malkey]

Here is an array free example.

Local $sStr = ""
Local $sSearchFor = "34,35"

; $Data = FileRead("test.txt")
; or
For $i = 1 To 8500
    $sStr &= StringRight("0" & Random(1, 99, 1), 2) & ","
Next
$Data = StringTrimRight($sStr, 1) ; Remove last comma
ConsoleWrite($Data & @CRLF)


StringReplace($Data, $sSearchFor, $sSearchFor)
Local $iNum = @extended
Local $iFirstLocation = (StringInStr($Data, $sSearchFor) - 1) / 3

ConsoleWrite('"' & $sSearchFor & '" number of occurrences = ' & $iNum & @CRLF)
ConsoleWrite("First found position = " & $iFirstLocation & @CRLF)

MsgBox(0, "Results", '"' & $sSearchFor & '" number of occurrences = ' & $iNum & @CRLF & _
        "First found position = " & $iFirstLocation)

;or

If StringInStr($Data, $sSearchFor) Then
    MsgBox(0, "Results", '"' & $sSearchFor & '"' & ' found')
Else
    MsgBox(0, "Results", '"' & $sSearchFor & '"' & ' not found')
EndIf

[Phaser]

Hi Guys

Just had time to revisit this thread as I am not getting exaclty what I wanted, although I didn't actually mention it first time, AndyGs' method is what I am using currently and it gets the first found result, malkeys' finds how many times it was in the string, what I really need is to find the 2 number sequence and grab the next number, for example

string/array 12,86,44,21,79,33

search for 44,21

find it and grab 79 to use in my next calculation

The above is what I am using BUT if the next scenario is

string/array 12,86,44,21,79,33,01,51,44,21,28,93

search for 44,21

find it and grab 79 AND 28 to use in my next calculation

I am guessing an array is the only way to do this as I can grab the next key

$data[$i+2]; plus 2 as it works from the first number of the sequence

But for the life of me cannot figure out how to change AndyGs' code, any ideas?

[Melba23]

Phaser,

This will get the numbers following the first 2 matches with the pattern you define:

expandcollapse popup

Global $aFound[2]

$sData = "12,86,44,21,79,33,01,51,44,21,28,93"

$aArray = StringSplit($sData, ",", 3) ;splits string into an array

$sSearch = "44,21" ;searchstring

; We will work on this copy of the data
$sSection = $sData

For $i = 0 To 1

    ;search in text
    $iPos = StringInStr($sSection, $sSearch, 1, 1)

    If $iPos Then ;string is found
        ; Determine start position of next number
        $iNext = $iPos + StringLen($sSearch) + 1
        $sNext = ""
        ; Add digits until next , or eol
        For $j = $iNext To $iNext + 5
            $sTemp = StringMid($sSection, $j, 1)
            If $sTemp = "," Or $sTemp = "" Then
                ; If we reach , or eol
                ExitLoop
            Else
                ; Add digit to number
                $sNext &= $sTemp
            EndIf
        Next
        $aFound[$i] = $sNext
        ; Strip the used numbers form the front of the string for teh next search
        $sSection = StringMid($sSection, $iNext)
    EndIf
Next
; Check if we got both values
If $aFound[1] <> "" Then
    MsgBox(0, "Next numbers", $aFound[0] & " - " & $aFound[1])
Else
    MsgBox(0, "StringInstr()", "Pattern " & $sSearch & " not found twice")
EndIf

Global $aFound[2]

;search
$sSearcharray = StringSplit($sSearch, ",", 3)
$iIndex = 0
For $i = 0 To UBound($aArray) - 1 - UBound($sSearcharray) ;all numbers in the array
    If $sSearcharray[0] = $aArray[$i] Then ;first number match
        For $x = 1 To UBound($sSearcharray) - 1 ;compare the numbers in the array with the numbers in the searchstring
            If $sSearcharray[$x] <> $aArray[$i + $x] Then ContinueLoop 2 ;if one of the following numbers doesnt match, then next number in array
        Next
        ; Add next element to array
        $iNext = $i + UBound($sSearcharray)
        $aFound[$iIndex] = $aArray[$iNext]
        $iIndex += 1
    EndIf
Next
; Check if we got both values
If $aFound[1] <> "" Then
    MsgBox(0, "Next numbers", $aFound[0] & " - " & $aFound[1])
Else
    MsgBox(0, "StringInstr()", "Pattern " & $sSearch & " not found twice")
EndIf

Does it do what you want?

M23

[Phaser]

M23 as usual, perfect, however small nigle I think

C:\Program Files\AutoIt3\SciTE\phaser\template\datacapture.au3 (91) : ==> Array variable has incorrect number of subscripts or subscript dimension range exceeded.:

$aFound[$iIndex] = $aArray[$iNext]

^ ERROR

The sequence of numbers can appear 1 or many times, on average it's about 5 to 7 times, I want the 5 to 7 following numbers and will probably do a looping bit of work on them, maybe push them to an array the walk through them, I just need to get the numbers initially, thanks again

MMM are there 2 seperate methods posted above? I am using the forst one, and have adjusted it to array size 5 and loop count 0 to 4 now it catches up to 5, can it be dynamic?

Edited June 8, 2010 by Phaser

[Melba23]

Phaser,

Declare the $aFound array to be bigger than the largest number of matches you may find - that will prevent the "subscript out of bounds" error. You will have to then walk through the array until you find an empty element to find the total number of matches - or use $iIndex in the second case.

And you will also need to change the If statement - perhaps checking that you have at least the minimum number of matches you require.

Let me know if you have any problems getting it to work.

M23