BigDaddyO

Which Language to Learn?

17 posts in this topic

I have been dubbed the File Validation guy at work recently which is fine with me, I kinda like it and AutoIT has been great to do this with except for 1 big thing.  it's super slow.

I'm validating files with millions of rows and some almost 4 thousand characters long in each line and it's taking 4 to 6 hours to finish validations.

I'm using regular expressions, String length, and if X then Y to validate the files.

I'm wondering if I should try and learn C++ or C# or something else in an attempt to speed up the process as the project teams are rather cranky on the time it takes even though it would never get fully validated without it.

 

right now, my focus is on speeding up my file validations but I'd also like to ensure the time spent learning the language will be useful in the future on other apps.

 

Thanks,

Mike

 


Share this post


Link to post
Share on other sites



Perhaps if you post your script forum members could offer suggestions on optimization. Beyond that, if you search the forum there are several threads going regarding using FreeBasic to overcome AutoIt's lack of speed with graphics; perhaps the same can be done with file parsing. Would be easier than jumping into a whole new language.


√-1 2^3 ∑ π, and it was delicious!

Share this post


Link to post
Share on other sites

Just wanted to post back here as I have had some more time to research and work on this.  I decided on C++ and re-wrote my script as a console application.

The AutoIT script was running almost exactly 6 hours.

The C++ console app which performs all the same actions runs in 36 minutes.  

Only downside is that I don't have a nice GUI or File>open to pick the file so I have to type the file into the console at run-time.

 

it was a lot more work to get CodeBlocks, mingw32, and boost setup and functioning properly than it was to figure out and write the C++ code as I was able to copy the majority of my Regular Expressions from my script and paste into the C++ code.

AutoIT will certainly stay my go-to language but it's nice to have C++ for the heavy lifting when needed.


Share this post


Link to post
Share on other sites

It must have been exceedingly well written AutoIt code. Only about 12 times slower than compiled C++ code. Usually this kind of AutoIt code is 100 - 1000 times slower than compiled C++ code.

Share this post


Link to post
Share on other sites

JLogan3o13:  I tried adding the OpenFileDialog many different ways but it's not recognized by the gcc mingw32 compiler.  I read that there is a switch I can add to the compiler to recognize it properly but I barely got the compiler working as-is.

 

LarsJ:  As for the AutoIT script, I did a lot of work testing each piece of the validation code with timers and switching code out to save microseconds where ever I could.  I really don't think it was possible to make it any faster.  as for my C++ code only being 12 times faster, it may just be that my C++ code sucks as it's my first attempt with that language.  Though, if I could figure out OpenMP so I could use more than one CPU core I could speed it up even more as it maxes out one core the entire time it's running.

 

I have lot's to learn, but I think I need to push my company to get me VisualStudio as this CodeBlocks and gcc mingw32 is just tough!

 


Share this post


Link to post
Share on other sites

#7 ·  Posted (edited)

My current favorite is FreeBasic, easy to learn and very fast!

My latest code in FB -> moving 69.696 pixels in 30fps on my notebook. :D Just move your mouse over the image to see the effect.

 

PS: I know that you are not looking for such kind of things...

Edited by UEZ
1 person likes this

Please don't send me any personal message and ask for support! I will not reply!

Selection of finest graphical examples at Codepen.io

The own fart smells best!
Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!
¯\_(ツ)_/¯  ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ

Share this post


Link to post
Share on other sites

waw.. wonderful effect :thumbsup:
very nice @UEZ!

here a similar effect in Javascript:

http://fabricjs.com/particles

p.s.
sorry for the little hijack...

1 person likes this

small minds discuss people average minds discuss events great minds discuss ideas.... and use AutoIt....

Share this post


Link to post
Share on other sites

Hello. Here is a CodeBlock+GCC OpenFileDialog  Example.

 

OpenDialogSample.zip

Saludos

Share this post


Link to post
Share on other sites

#10 ·  Posted (edited)

BigDaddyO, Do not take my comment too seriously. It was meant as a joke. I know that it's much harder to learn a language like C++ than AutoIt. If on the other hand, you have a good understanding of a language like AutoIt you can learn any language. It just takes more time and requires more reading. And a better language than C++ does not exist.

You can get a free Visual Studio here as you can use for learning.

Edited by LarsJ

Share this post


Link to post
Share on other sites

Don't want to start a huge discussion of which language is better, but I'm pretty sure that there are a number of better languages out there than C++.

Of course "better" is a relative term whose definition would change depending on what it is you're trying to do with the language. Such as, HTML is a far better language than C++ for writing web pages. But I wouldn't want to write high level programs in HTA. I'd bet that there are quite a few people that would argue that C# is a better language than C++ depending on what you're doing in it. Delphi would probably be as good as, if not better than, C++.


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites

#12 ·  Posted (edited)

16 hours ago, Chimp said:

waw.. wonderful effect :thumbsup:
very nice @UEZ!

here a similar effect in Javascript:

http://fabricjs.com/particles

p.s.
sorry for the little hijack...

Thx for the link. Interesting stuff. ;)

Edited by UEZ

Please don't send me any personal message and ask for support! I will not reply!

Selection of finest graphical examples at Codepen.io

The own fart smells best!
Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!
¯\_(ツ)_/¯  ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ

Share this post


Link to post
Share on other sites
21 hours ago, BigDaddyO said:

 As for the AutoIT script, I did a lot of work testing each piece of the validation code with timers and switching code out to save microseconds where ever I could.  I really don't think it was possible to make it any faster.

For 99.9% of all applications, AutoIt is fast enough, just when it comes to GUI-related stuff! Not to mention that the creation of the GUI´s is very fast in AutoIt!

If you need SPEED, in the case of many calculations probably mostly in (several) nested loops, those calculations could be done in a "faster" language like C(++), FreeBasic or ANY other compiling language. Profile your script to find the "inner loop", extract these handful lines and put them into a DLL. Call this DLL from AutoIt and you have the advantage of a fast (GUI)-development and a fast calculation by compiled code.

It is NOT necessary to be a C(++)-Crack to do some calculations in C(++), and it is NOT necessary to use OOP! Replace "C(++)" with any other compiling language you want...

 

I prefer Assembler to speed up code, because mostly the runtime of the code can be done faster compared to a compiled code. Yes, the time to write (draw up/create) ASM-code is MUCH longer than typing some lines in C(++) or any other compiling language, but if i want my program REALLY fast, i analyze the code made by a compiler and "see" if the expected acceleration by ASM is worth the time. Most of the time i can speed up compiled code with handcrafted ASM by factor of 2 to 4 (and up). With the use of SSE/SIMD the improvement can be much higher! I have never seen compiled SSE/SIMD code which uses 4x32Bit in a XMM-Register to calculate for example 4 Pixels of a image-filter at the same time....

Most of the code in HLL´s is poorly programmed, and so it is not a surprise that the compiled result is not turned into gold by flipping some compiler-switches...

 

Share this post


Link to post
Share on other sites

#14 ·  Posted (edited)

21 hours ago, Danyfirex said:

Hello. Here is a CodeBlock+GCC OpenFileDialog  Example.

 

OpenDialogSample.zip

Saludos

Thanks for the sample.  It works great within that sample project, but...

That code doesn't work with mine.  I even copied the contents of the main.cpp into a new blank projects main.cpp and it didn't work there either.

undefined reference to 'GetOpenFileNameA@4'

There must be some configuration setting somewhere that I can't find that allows it to work within your sample project.

Mike

 

EDIT:  Found the setting.

Projects > Build options...

Debug > Linker settings tab > add Comdlg32

 

I'm assuming I'll also need to do that for the Release item as well.

 

ScreenCap_58.JPG

Edited by BigDaddyO
Figured out the setting

Share this post


Link to post
Share on other sites

Yes you need to add link libraries to Release version too.

 

Saludos

Share this post


Link to post
Share on other sites

so...  just to make sure everyone knows i'm a total newbe with C++, I realized when adding that library link "thanks Danyfirex" that I was only compiling the debug version of my validation program.

After compiling the Release version and running it from my local drive instead of a network drive it runs in just over 10 minutes.  so, I am now running 36 times faster with the C++ code vs my original AutoIT code.

 

Mike


Share this post


Link to post
Share on other sites

Hi,

I am a guy who started learning programming with vb6. I have stopped it after 3 months. Then a few years later i have started again with VB.Net. Things went cool. I learned the basics of variables , control flow and databases. But not learned about functions and classes and loops. At that time i heard about python. I have read that it's easy. I started learning python. It helped me to learn loops and functions.  There is web tool called Python Visualizer. It teach me how to use loops. But OOP in python was horrible for me. So i came back to VB.Net. Then i started learning OOP  in VB.Net and so many books helped me. After that, i need to learn the low level programming of windows. I met Win32 API but failed to learn. Then i learned AutoIt. It helped me to learn win32 API easily. it points me to inner secrets of windows. And the book of Charles Petzold helped me a lot. Then i learned the basics of FreeBasic, ThinBasic, FBSL. 

 


My Contributions

UDF Link Viewer   --- A tool to visit the links of some most important UDFs 

 Includer_2  ----- A tool to type the #include statement automatically 

 Digits To Date  ----- date from 3 integer values

PrintList ----- prints arrays into console for testing.

 Alert  ------ An alternative for MsgBox 

 MousePosition ------- A simple tooltip display of mouse position

GRM Helper -------- A littile tool to help writing code with GUIRegisterMsg function

Access_UDF  -------- An UDF for working with access database files. (.*accdb only)

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now