multiprocess script: running batch jobs in parallel

[jftuga]

I have created a program that runs a command-line executable in parallel on a group of similar files. I am sure something like this has been posted before; but I wanted to have the satisfaction of creating my own flavor and it has also been awhile since I have contributed a script back to the community. I am very happy with the way it turned out.

I was able to run the ImageMagick convert.exe program to convert 1440 jpg files (avg size: 135kb) to tiff files. On a 16 CPU system this created 1440 tiff files in 74 seconds, which is just under 20 image conversions per second. The program figures out the number of CPUs your system has and runs that many subprocesses concurrently.

My program is called mp, short for multi-process. From a command line, you would use:

mp.exe convert “_input_ _base_.tiff” -- *.jpg

Anything inside of underscores are macros that get substituted with file names. See the website for more information and examples.

1) https://github.com/jftuga/Windows/tree/master/mp

2) click on the mp.exe link

3) click on Raw to download the actual mp.exe exectuable

4) usage: https://github.com/jftuga/Windows/raw/master/mp/usage.png

5) AutoIt source code: https://github.com/jftuga/Windows/blob/master/mp/mp.au3

If you decide to try this out, please reply as I would love to get some feedback. I'd also love to hear about similar AutoIt scripts, too. Maybe I can learn from those as well.

Thanks,

-John

[stormbreaker]

Please post your program at another hosting site (for eg. mediafire or any other), unable to load Github

EDIT:

GitHub links working now. Tested your program with 7za.exe and works fine.

Edited May 3, 2012 by MKISH