VIP

How to replace faster?

4 posts in this topic

#1 ·  Posted (edited)

EG:

List search string (>5000 value):

\00\1b\|\00\8b\|\00\8b\|\00\95\|\00\ab\|\00\dd\|\00\f0\|\01\27\|\01\37\|\01\4f\|\01\92\|\01\95\|\01\b5\|\01\bc\|\02\0a\|\02\12\|\02\1d\|\02\a3\|

String  (It is one big file >30MB!):

X:\f\c\g\a0\e3\a0e307c6659c7a66836e2981ea9ee6d8.png
X:\f\c\g\a8\15\a815f6f11e281e378d310393894b9da9.png
X:\f\c\g\cc\89\cc89671b7dcf1adad300145d27dd3f83.png
X:\f\c\g\30\45\30459a904e7bce61a744e92772edcba0.png
X:\f\c\g\ef\db\efdb9d0d837858ce3a209775b8e07d66.png
X:\f\c\g\b6\0b\b60b11095b84935cc069a1f997a7e5ad.png
X:\f\c\g\24\01\2401d3834b4119790b76ff44d7397aab.png
X:\f\c\g\89\0b\890b5b9b02e5ec56ef6b8d6a542cb7fd.png
X:\f\c\g\90\6f\906f44428cc33593f008be47afb2288b.png
X:\f\c\g\6b\b5\6bb5c9d6ed6e48db41863675b6b0937b.png
X:\f\c\g\c3\9c\c39c341ad690cf76a090989f5b0e3e5b.png
X:\f\c\g\dc\ea\dceac588a90e67d65f0b6e2fcb43b7ae.png
X:\f\c\g\63\af\63affc4b4f13707042b45a2ccdf8882a.png
X:\f\c\g\11\b6\11b6ac483fb453ecbfeed25903740cc5.png
X:\f\c\g\2b\10\2b100ea9addad0cfc9193927ca8d211a.png
X:\f\c\g\37\26\3726bd8892826dbc1c2fa1d859504ac7.png

Replace string:

\New\String\

Autoitscript:

Local $replacestring="\New\String\"
    Local $string=FileRead("File.ext")
    Local $searchstring=StringSplit(FileRead("ListSearchString.txt"),"|")
    
    For $i=1 to $searchstring[0]
        ConsoleWrite($searchstring&@CRLF)
        $string=StringReplace($string,$searchstring[$i],$replacestring)
    Next
    FileWrite("NewFileReplaced.ext",$string)

It works very long on my PC CPU 16-Core-i7

Edited by You

Regards,
 

Share this post


Link to post
Share on other sites



#2 ·  Posted

Well you have plenty of data to manage.

utoIt is interpreted. This means there is a C++ interpreter reading each line and executing it in runtime. So don't expect great performance from interpreted languages (expect, instead, ease of coding).

You could try using C/C++ or some other compiled language, but I'd really rather feeding a database engine with this data and using SQL to manage it...

1 person likes this

My stuff

Spoiler

My UDFs  _AuThread multithreading emulation for AutoIt · _ExtInputBox an inputbox with multiple inputs and more features · forceUTF8 fix strings encoding without knowing its original charset · JSONgen JSON generator · _TCPServer UDF multi-client and multi-task (run on background) event-based TCP server easy to do · _TCPClient_UDF multi-server and multi-task (runs on background) event-based TCP client easy to do · ParseURL and ParseStr functions ported from PHP · _CmdLine UDF easily parse command line parameters, keys or flags · AutoPHP Create documents (bills, incomes) from HTML by sending variables/arrays from AutoIt to PHP · (Un)Serialize Convert arrays and data into a storable string (PHP compatible) · RTTL Plays and exports to MP3 Nokia-format monophonic ringtones (for very old cellphones) · I18n library Simple and easy to use localization library · Scripting.Dictionary OOP and OOP-like approach · Buffer/stack limit arrays to N items by removing the last one once the limit is reached · NGBioAPI UDF to work with Nitgen fingerprint readers · Serial/Licensing system require license key based on unique machine ID from your users · HTTP a simple WinHTTP library that allows GET, POST and file uploads · Thread true AutoIt threads (under-dev) · RC4 RC4 encryption compatible with PHP and JS Classes _WKHtmlToX uses wkhtmlto* to convert HTML files and webpages into PDF or images (jpg, bmp, gif, png...) Snippets _Word_DocFindReplaceByLongText replace strings using Word UDF with strings longer than 255 characters (MSWord limit) rangeparser parser for printing-like pages interval (e.g.: "1,2,3-5") EnvParser parse strings/paths with environment variables and get full path Random stuff Super Mario beep sound your ears will hurt

 

Share this post


Link to post
Share on other sites

#3 ·  Posted (edited)

Are you actually replacing the original hex values with the literal "\New\String\"?   (Which would make each new record 5 characters longer than its original record.)

Or are you replacing the original hex values with a new set of hex values that would take up the same space in the new record?

Edited by cbruce

Share this post


Link to post
Share on other sites

#4 ·  Posted

I may have misunderstood what you are trying to do, but using a regular expression and changing all the values with a single pass rather than your current thousands of separate replace actions should be thousands of times faster.

Local $replacestring="\\New\\String\\"
    Local $string=FileRead("File.ext")
    Local $searchstring=StringSplit(FileRead("ListSearchString.txt"),"|")

    $string=StringRegExpReplace($string,"(X:\\f\\c\\g)\\[0-9a-f]{2}\\[0-9a-f]{2}\\(.+\.png)","\1" & $replacestring & "\2")


    FileWrite("NewFileReplaced.ext",$string)

 gives this result on your test data

X:\f\c\g\New\String\a815f6f11e281e378d310393894b9da9.png
X:\f\c\g\New\String\cc89671b7dcf1adad300145d27dd3f83.png
X:\f\c\g\New\String\30459a904e7bce61a744e92772edcba0.png
X:\f\c\g\New\String\efdb9d0d837858ce3a209775b8e07d66.png
X:\f\c\g\New\String\b60b11095b84935cc069a1f997a7e5ad.png
X:\f\c\g\New\String\2401d3834b4119790b76ff44d7397aab.png
X:\f\c\g\New\String\890b5b9b02e5ec56ef6b8d6a542cb7fd.png
X:\f\c\g\New\String\906f44428cc33593f008be47afb2288b.png
X:\f\c\g\New\String\6bb5c9d6ed6e48db41863675b6b0937b.png
X:\f\c\g\New\String\c39c341ad690cf76a090989f5b0e3e5b.png
X:\f\c\g\New\String\dceac588a90e67d65f0b6e2fcb43b7ae.png
X:\f\c\g\New\String\63affc4b4f13707042b45a2ccdf8882a.png
X:\f\c\g\New\String\11b6ac483fb453ecbfeed25903740cc5.png
X:\f\c\g\New\String\2b100ea9addad0cfc9193927ca8d211a.png
X:\f\c\g\New\String\3726bd8892826dbc1c2fa1d859504ac7.png

 

1 person likes this

"Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the universe trying to build bigger and better idiots. So far, the universe is winning."- Rick Cook

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now