Jump to content

How to replace faster?


Recommended Posts

EG:

List search string (>5000 value):

\00\1b\|\00\8b\|\00\8b\|\00\95\|\00\ab\|\00\dd\|\00\f0\|\01\27\|\01\37\|\01\4f\|\01\92\|\01\95\|\01\b5\|\01\bc\|\02\0a\|\02\12\|\02\1d\|\02\a3\|

String  (It is one big file >30MB!):

X:\f\c\g\a0\e3\a0e307c6659c7a66836e2981ea9ee6d8.png
X:\f\c\g\a8\15\a815f6f11e281e378d310393894b9da9.png
X:\f\c\g\cc\89\cc89671b7dcf1adad300145d27dd3f83.png
X:\f\c\g\30\45\30459a904e7bce61a744e92772edcba0.png
X:\f\c\g\ef\db\efdb9d0d837858ce3a209775b8e07d66.png
X:\f\c\g\b6\0b\b60b11095b84935cc069a1f997a7e5ad.png
X:\f\c\g\24\01\2401d3834b4119790b76ff44d7397aab.png
X:\f\c\g\89\0b\890b5b9b02e5ec56ef6b8d6a542cb7fd.png
X:\f\c\g\90\6f\906f44428cc33593f008be47afb2288b.png
X:\f\c\g\6b\b5\6bb5c9d6ed6e48db41863675b6b0937b.png
X:\f\c\g\c3\9c\c39c341ad690cf76a090989f5b0e3e5b.png
X:\f\c\g\dc\ea\dceac588a90e67d65f0b6e2fcb43b7ae.png
X:\f\c\g\63\af\63affc4b4f13707042b45a2ccdf8882a.png
X:\f\c\g\11\b6\11b6ac483fb453ecbfeed25903740cc5.png
X:\f\c\g\2b\10\2b100ea9addad0cfc9193927ca8d211a.png
X:\f\c\g\37\26\3726bd8892826dbc1c2fa1d859504ac7.png

Replace string:

\New\String\

Autoitscript:

Local $replacestring="\New\String\"
    Local $string=FileRead("File.ext")
    Local $searchstring=StringSplit(FileRead("ListSearchString.txt"),"|")
    
    For $i=1 to $searchstring[0]
        ConsoleWrite($searchstring&@CRLF)
        $string=StringReplace($string,$searchstring[$i],$replacestring)
    Next
    FileWrite("NewFileReplaced.ext",$string)

It works very long on my PC CPU 16-Core-i7

Edited by You

Regards,
 

Link to comment
Share on other sites

Well you have plenty of data to manage.

utoIt is interpreted. This means there is a C++ interpreter reading each line and executing it in runtime. So don't expect great performance from interpreted languages (expect, instead, ease of coding).

You could try using C/C++ or some other compiled language, but I'd really rather feeding a database engine with this data and using SQL to manage it...

My stuff

Spoiler

My UDFs  _AuThread multithreading emulation for AutoIt · _ExtInputBox an inputbox with multiple inputs and more features · forceUTF8 fix strings encoding without knowing its original charset · JSONgen JSON generator · _TCPServer UDF multi-client and multi-task (run on background) event-based TCP server easy to do · _TCPClient_UDF multi-server and multi-task (runs on background) event-based TCP client easy to do · ParseURL and ParseStr functions ported from PHP · _CmdLine UDF easily parse command line parameters, keys or flags · AutoPHP Create documents (bills, incomes) from HTML by sending variables/arrays from AutoIt to PHP · (Un)Serialize Convert arrays and data into a storable string (PHP compatible) · RTTL Plays and exports to MP3 Nokia-format monophonic ringtones (for very old cellphones) · I18n library Simple and easy to use localization library · Scripting.Dictionary OOP and OOP-like approach · Buffer/stack limit arrays to N items by removing the last one once the limit is reached · NGBioAPI UDF to work with Nitgen fingerprint readers · Serial/Licensing system require license key based on unique machine ID from your users · HTTP a simple WinHTTP library that allows GET, POST and file uploads · Thread true AutoIt threads (under-dev) · RC4 RC4 encryption compatible with PHP and JS ·  storage.au3 localStorage and sessionStorage for AutoIt Classes _WKHtmlToX uses wkhtmlto* to convert HTML files and webpages into PDF or images (jpg, bmp, gif, png...) Snippets _Word_DocFindReplaceByLongText replace strings using Word UDF with strings longer than 255 characters (MSWord limit) rangeparser parser for printing-like pages interval (e.g.: "1,2,3-5") EnvParser parse strings/paths with environment variables and get full path GUICtrlStaticMarquee static text scrolling Random stuff Super Mario beep sound your ears will hurt

 

Link to comment
Share on other sites

Are you actually replacing the original hex values with the literal "\New\String\"?   (Which would make each new record 5 characters longer than its original record.)

Or are you replacing the original hex values with a new set of hex values that would take up the same space in the new record?

Edited by cbruce
Link to comment
Share on other sites

I may have misunderstood what you are trying to do, but using a regular expression and changing all the values with a single pass rather than your current thousands of separate replace actions should be thousands of times faster.

Local $replacestring="\\New\\String\\"
    Local $string=FileRead("File.ext")
    Local $searchstring=StringSplit(FileRead("ListSearchString.txt"),"|")

    $string=StringRegExpReplace($string,"(X:\\f\\c\\g)\\[0-9a-f]{2}\\[0-9a-f]{2}\\(.+\.png)","\1" & $replacestring & "\2")


    FileWrite("NewFileReplaced.ext",$string)

 gives this result on your test data

X:\f\c\g\New\String\a815f6f11e281e378d310393894b9da9.png
X:\f\c\g\New\String\cc89671b7dcf1adad300145d27dd3f83.png
X:\f\c\g\New\String\30459a904e7bce61a744e92772edcba0.png
X:\f\c\g\New\String\efdb9d0d837858ce3a209775b8e07d66.png
X:\f\c\g\New\String\b60b11095b84935cc069a1f997a7e5ad.png
X:\f\c\g\New\String\2401d3834b4119790b76ff44d7397aab.png
X:\f\c\g\New\String\890b5b9b02e5ec56ef6b8d6a542cb7fd.png
X:\f\c\g\New\String\906f44428cc33593f008be47afb2288b.png
X:\f\c\g\New\String\6bb5c9d6ed6e48db41863675b6b0937b.png
X:\f\c\g\New\String\c39c341ad690cf76a090989f5b0e3e5b.png
X:\f\c\g\New\String\dceac588a90e67d65f0b6e2fcb43b7ae.png
X:\f\c\g\New\String\63affc4b4f13707042b45a2ccdf8882a.png
X:\f\c\g\New\String\11b6ac483fb453ecbfeed25903740cc5.png
X:\f\c\g\New\String\2b100ea9addad0cfc9193927ca8d211a.png
X:\f\c\g\New\String\3726bd8892826dbc1c2fa1d859504ac7.png

 

"Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the universe trying to build bigger and better idiots. So far, the universe is winning."- Rick Cook

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...