Sign in to follow this  
Followers 0
golfinhu

help in speed of program!

10 posts in this topic

#1 ·  Posted (edited)

hello people, okay?

people, with the help of water, I created these two functions in that it opens tags when it encounters a string, and closes it in the first line break

well, it works, just ...

the files that I use, everyone has more than 80000 lines, so it ends up taking too long, about 3 hours or more ...

I was wondering if you have a faster way than this, because here it reads line by line for finding everything, which makes the speed of the program!

thanks!

following codes:

Func open_tag($InFile)

    Local $aInput
    _FileReadToArray($InFile, $aInput)
    Local $iCount = 1
    While 1

        If $aInput[$iCount] = "Especificações:" Or $aInput[$iCount] = "Especificação:" Then
            $aInput[$iCount] = "<TagEspecificacao>"
            _ArrayInsert($aInput, $iCount + 1, "Especificações:")
            $iCount += 1
        EndIf


        If $aInput[$iCount] = "Dimensões aproximadas(AxLxP):" Or $aInput[$iCount] = "Medidas Aproximadas:" Or $aInput[$iCount] = "Dimensões aproximadas:" Or $aInput[$iCount] = "Dimensões:" Then
            $aInput[$iCount] = "<TagDimensoes>"
            _ArrayInsert($aInput, $iCount + 1, "Dimensões Aproximadas:")
            $iCount += 1
        EndIf


        If $aInput[$iCount] = "Garantia:" Then
            $aInput[$iCount] = "<TagGarantia>"
            _ArrayInsert($aInput, $iCount + 1, "Garantia:")
            $iCount += 1
        EndIf


        If $aInput[$iCount] = "Marca:" Then
            $aInput[$iCount] = "<TagMarca>"
            _ArrayInsert($aInput, $iCount + 1, "Marca:")
            $iCount += 1
        EndIf


        If $aInput[$iCount] = "Conteúdo do produto:" Or $aInput[$iCount] = "Itens Inclusos:" Or $aInput[$iCount] = "Conteudo do Produto:" Or $aInput[$iCount] = "Conteúdo:" Then
            $aInput[$iCount] = "<TagConteudo>"
            _ArrayInsert($aInput, $iCount + 1, "Conteúdo:")
            $iCount += 1
        EndIf


        $iCount += 1
        If $iCount > UBound($aInput) - 1 Then ExitLoop
    WEnd
    $aInput[0] = UBound($aInput) - 1
    _FileWriteFromArray($InFile, $aInput, 1)
EndFunc   ;==>open_tag

Func Close_tag($InFile)

    Local $aInput
    _FileReadToArray($InFile, $aInput) ; the file to open to array
    Local $Tag = ""
    Local $iCount = 1 ; the line of array
    While 1
        If StringLeft($aInput[$iCount], 2) <> "</" And StringLeft($aInput[$iCount], 1) = "<" Then ;if line contains a opned tag!
            $Tag = StringTrimLeft(StringTrimRight($aInput[$iCount], 1), 1) ;set the tag the word that is between "< >"
            $char_pos = StringInStr($aInput[$iCount], ">", 1, 1)

            If StringTrimLeft($aInput[$iCount], $char_pos) = "" And $Tag <> "link_produto" And $Tag <> "produto" And $Tag <> "imagem" And $Tag <> "descricao" And $Tag <> 'loja nome="Jovial Magazine"' Then

                For $i = 1 To UBound($aInput) ;loop to find the first blank line
                    If $aInput[$iCount + $i] = "" Then ;if the first blank line
                        $aInput[$iCount + $i] = "</" & $Tag & ">"
                        _ArrayInsert($aInput, $iCount + $i + 1, "")
                        $iCount += 1
                        ExitLoop
                    EndIf
                Next
            EndIf
        EndIf
        $iCount += 1
        If $iCount > UBound($aInput) - 1 Then ExitLoop
    WEnd
    $aInput[0] = UBound($aInput) - 1
    _FileWriteFromArray($InFile, $aInput, 1)
EndFunc   ;==>Close_Tag
Edited by golfinhu

Share this post


Link to post
Share on other sites



golfinhu,

open_tag function:

This function is slow becasue of all the _ArrayInsert lines. Each of them forces a ReDim of the array, which is slow on any array, but must be glacial on an 80000 element one! :huggles:

Try adding the required word to the same line each time, preceded by a <newline> - that way the array stays the same size and you avoid the ReDims. When you rewrite the file, AutoIt reinserts the <newline> between each element of the array, so the end result should be the same. I have not tested it, but give it a run and see if it works more quickly - I see no reason why it should not:

Func open_tag($InFile)

      Local $aInput
    _FileReadToArray($InFile, $aInput)
    Local $iCount = 1
    While 1

        If $aInput[$iCount] = "Especificações:" Or $aInput[$iCount] = "Especificação:" Then
            $aInput[$iCount] = "<TagEspecificacao>" & @CRLF & "Especificações:"
            ;_ArrayInsert($aInput, $iCount + 1, "Especificações:")
            $iCount += 1
        EndIf


        If $aInput[$iCount] = "Dimensões aproximadas(AxLxP):" Or $aInput[$iCount] = "Medidas Aproximadas:" Or $aInput[$iCount] = "Dimensões aproximadas:" Or $aInput[$iCount] = "Dimensões:" Then
            $aInput[$iCount] = "<TagDimensoes>" & @CRLF & "Dimensões Aproximadas:"
            ;_ArrayInsert($aInput, $iCount + 1, )
            $iCount += 1
        EndIf


        If $aInput[$iCount] = "Garantia:" Then
            $aInput[$iCount] = "<TagGarantia>" & @CRLF & "Garantia:"
            ;_ArrayInsert($aInput, $iCount + 1, "Garantia:")
            $iCount += 1
        EndIf


        If $aInput[$iCount] = "Marca:" Then
            $aInput[$iCount] = "<TagMarca>" & @CRLF & "Marca:"
            ;_ArrayInsert($aInput, $iCount + 1, "Marca:")
            $iCount += 1
        EndIf


        If $aInput[$iCount] = "Conteúdo do produto:" Or $aInput[$iCount] = "Itens Inclusos:" Or $aInput[$iCount] = "Conteudo do Produto:" Or $aInput[$iCount] = "Conteúdo:" Then
            $aInput[$iCount] = "<TagConteudo>" & @CRLF & "Conteúdo:"
            ;_ArrayInsert($aInput, $iCount + 1, "Conteúdo:")
            $iCount += 1
        EndIf

        $iCount += 1
        If $iCount > UBound($aInput) - 1 Then ExitLoop
    WEnd
    
    _FileWriteFromArray($InFile, $aInput, 1)
    
EndFunc   ;==>open_tag

I will look at the second function after dinner. :D

M23


Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

#3 ·  Posted (edited)

I will try and answer has been improved!

thanks for the reply!

Edit:

Yes, improved a lot!

thanks for your help!

Edited by golfinhu

Share this post


Link to post
Share on other sites

golfinhu.

Actually it was much quicker than I thought as it is almost the same problem and solution:

Func Close_tag($InFile)

    Local $aInput
    _FileReadToArray($InFile, $aInput) ; the file to open to array
    Local $Tag = ""
    Local $iCount = 1 ; the line of array
    While 1

        ; Why a double If???? If the first 2 chars are "</" then the first char MUST be "<"!!!!!!!!!!!!!!
        If StringLeft($aInput[$iCount], 2) <> "</" Then ; And StringLeft($aInput[$iCount], 1) = "<" Then ;if line contains a opned tag!

            $Tag = StringTrimLeft(StringTrimRight($aInput[$iCount], 1), 1) ;set the tag the word that is between "< >"
            $char_pos = StringInStr($aInput[$iCount], ">", 1, 1)
            If StringTrimLeft($aInput[$iCount], $char_pos) = "" Then
                Switch $Tag
                    Case "link_produto", "produto", "imagem", "descricao", 'loja nome="Jovial Magazine"'
                            ; Do nothing
                    Case Else
                        For $i = $iCount To $aInput[0] -  ;loop from here to the end to find the first blank line
                            If $aInput[$i] = "" Then      ;if the first blank line
                                $aInput[$i] = "</" & $Tag & ">" & @CRLF ; As before add a blank line by adding a <newline> and avoiding the ReDim
                                ;_ArrayInsert($aInput, $iCount + $i + 1, "")
                                ;$iCount += 1
                                ExitLoop
                            EndIf
                        Next
                EndSwitch
            EndIf
            
        EndIf
        $iCount += 1
        If $iCount > $aInput[0] Then ExitLoop
    WEnd
    $aInput[0] = UBound($aInput) - 1
    _FileWriteFromArray($InFile, $aInput, 1)
EndFunc   ;==>Close_Tag

Let me know if it helps. :D

M23


Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

golfinhu.

Actually it was much quicker than I thought as it is almost the same problem and solution:

Func Close_tag($InFile)

    Local $aInput
    _FileReadToArray($InFile, $aInput) ; the file to open to array
    Local $Tag = ""
    Local $iCount = 1 ; the line of array
    While 1

        ; Why a double If???? If the first 2 chars are "</" then the first char MUST be "<"!!!!!!!!!!!!!!
        If StringLeft($aInput[$iCount], 2) <> "</" Then ; And StringLeft($aInput[$iCount], 1) = "<" Then ;if line contains a opned tag!

            $Tag = StringTrimLeft(StringTrimRight($aInput[$iCount], 1), 1) ;set the tag the word that is between "< >"
            $char_pos = StringInStr($aInput[$iCount], ">", 1, 1)
            If StringTrimLeft($aInput[$iCount], $char_pos) = "" Then
                Switch $Tag
                    Case "link_produto", "produto", "imagem", "descricao", 'loja nome="Jovial Magazine"'
                            ; Do nothing
                    Case Else
                        For $i = $iCount To $aInput[0] -  ;loop from here to the end to find the first blank line
                            If $aInput[$i] = "" Then      ;if the first blank line
                                $aInput[$i] = "</" & $Tag & ">" & @CRLF ; As before add a blank line by adding a <newline> and avoiding the ReDim
                                ;_ArrayInsert($aInput, $iCount + $i + 1, "")
                                ;$iCount += 1
                                ExitLoop
                            EndIf
                        Next
                EndSwitch
            EndIf
            
        EndIf
        $iCount += 1
        If $iCount > $aInput[0] Then ExitLoop
    WEnd
    $aInput[0] = UBound($aInput) - 1
    _FileWriteFromArray($InFile, $aInput, 1)
EndFunc   ;==>Close_Tag

Let me know if it helps. :D

M23

working!

and in the case of two 'ifs' is because the file does not contain only tags, it also contains the text!

then he has to find a line that contains an open tag!

thankyou so much!

Share this post


Link to post
Share on other sites

golfinhu,

Glad I could help! :D

What sort of speed increase did you get? Do you have any figures?

M23


Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

golfinhu,

Glad I could help! :D

What sort of speed increase did you get? Do you have any figures?

M23

about 1 minute per conversion =)

before it took about 2 ~ 3 hours

a very considerable improvement

hehe

Thank you so!

Share this post


Link to post
Share on other sites

golfinhu,

I told you ReDim was slow!!!!! :D

M23


Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

yes, and he was right xD

Share this post


Link to post
Share on other sites

#10 ·  Posted (edited)

golfinhu.

Actually it was much quicker than I thought as it is almost the same problem and solution:

Func Close_tag($InFile)

 Local $aInput
 _FileReadToArray($InFile, $aInput) ; the file to open to array
 Local $Tag = ""
 Local $iCount = 1 ; the line of array
 While 1

        ; Why a double If???? If the first 2 chars are "</" then the first char MUST be "<"!!!!!!!!!!!!!!
 If StringLeft($aInput[$iCount], 2) <> "</" Then ; And StringLeft($aInput[$iCount], 1) = "<" Then ;if line contains a opned tag!

 $Tag = StringTrimLeft(StringTrimRight($aInput[$iCount], 1), 1) ;set the tag the word that is between "< >"
 $char_pos = StringInStr($aInput[$iCount], ">", 1, 1)
 If StringTrimLeft($aInput[$iCount], $char_pos) = "" Then
                Switch $Tag
                    Case "link_produto", "produto", "imagem", "descricao", 'loja nome="Jovial Magazine"'
                            ; Do nothing
                    Case Else
                        For $i = $iCount To $aInput[0] - ;loop from here to the end to find the first blank line
                            If $aInput[$i] = "" Then ;if the first blank line
                                $aInput[$i] = "</" & $Tag & ">" & @CRLF ; As before add a blank line by adding a <newline> and avoiding the ReDim
                                ;_ArrayInsert($aInput, $iCount + $i + 1, "")
                                ;$iCount += 1
                                ExitLoop
                            EndIf
                        Next
                EndSwitch
 EndIf
            
 EndIf
 $iCount += 1
 If $iCount > $aInput[0] Then ExitLoop
 WEnd
 $aInput[0] = UBound($aInput) - 1
 _FileWriteFromArray($InFile, $aInput, 1)
EndFunc ;==>Close_Tag

Let me know if it helps. :D

M23

Here are even more minor speed improvements:

- accessing the same array element many times inside inner loop avoided

- StringMid is quicker than StringTrim

Func Close_tag($InFile)

    Local $aInput, $line
    _FileReadToArray($InFile, $aInput) ; the file to open to array
    Local $Tag = ""
    Local $iCount = 1 ; the line of array
    Local $iMax = $aInput[0] ; number of lines
    
    While 1
        $line = $aInput[$iCount]
        
    If StringLeft($line, 2) <> "</" Then ;if line contains a opned tag!
;~  If StringTrimLeft($line, StringInStr($line, ">", 1, 1)) = "" Then
    If StringMid($line, StringInStr($line, ">") + 1) = "" Then
;~              $Tag = StringTrimLeft(StringTrimRight($line, 1), 1) ;set the tag the word that is between "< >"
                $Tag = StringMid($line, 2, StringLen($line)-2) ;set the tag the word that is between "< >"
    Switch $Tag
    Case "link_produto", "produto", "imagem", "descricao", 'loja nome="Jovial Magazine"'
    ; Do nothing
    Case Else
    For $i = $iCount To $iMax ;loop from here to the end to find the first blank line
    If $aInput[$i] = "" Then    ;if the first blank line
    $aInput[$i] = "</" & $Tag & ">" & @CRLF ; As before add a blank line by adding a <newline> and avoiding the ReDim
    ExitLoop
    EndIf
    Next
    EndSwitch
    EndIf
    EndIf
    $iCount += 1
    If $iCount > $iMax Then ExitLoop
    WEnd
    
    $aInput[0] = UBound($aInput) - 1
    _FileWriteFromArray($InFile, $aInput, 1)
EndFunc ;==>Close_Tag
Edited by Zedna

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0