Eigen4AutoIt - Matrix computing with Eigen

76 posts in this topic

#1 ·  Posted (edited)

The Eigen C++ template library is a great environment for matrix computing; it is fast, reliable, extensive, and well-documented. It is also completely free, and does not rely on any external dependencies. Unfortunately for AutoIt users, the term “template library” implies that any functions you call are only instantiated upon compilation (in C++). That means there's nothing to hook into. :think:

To make Eigen ’s most important functionality directly accessible from within AutoIt scripts (version 3.3.12+, download it here), I developed the Eigen4AutoIt environment. :graduated: It runs on Windows and under Wine also on Linux.

>Download the latest version


It consists of:

1) Eigen4AutoIt.au3

  • an AutoIt library of wrapper functions that contain extensive bounds checks, matrix management, file I/O, type conversion, and precision control, and two-way data exchange with files and native AutoIt arrays;

2) Eigen-wrapper dlls (EigenDense.dll, EigenDense_x64.dll)

  • re-maps matrices created in AutoIt as Eigen Matrix objects, then calls Eigen’s powerful core functions; the single-precision part of the C++ source code I wrote is included in the bundle (the double-precision,, integer, and complex alternatives just duplicate this with different memory mappings and variable types). The basic functions consist of a single Eigen call; decompositions and statistics are more involved.

3) Additional materials:

  • the user-interactive, animated Function Selector and MatrixViewer tools
  • the MatrixFileConverter to read/write E4A matrices from/to .csv ASCII, Excel, and Xbase files.
  • three libraries of scientific and mathematical constants
  • online Help
  • Large (.chm) Help file
  • Quickstart Manual (11-page pdf, updated)
  • Test suite (24 scripts)
  • Five Tutorials, from Basics to Advanced

Please note:

  • none of this is part of Eigen's own distribution
  • you only need this bundle; you do not need to install Eigen.

How it works:
No matrix content is ever transferred, only memory pointers, meaning computations in AutoIt are virtually as fast as in Eigen’s native environment, with the added advantage of not having to be compiled first. The drawback is that Eigen's native ad-hoc expression templates (and their internal optimisations) cannot be exploited here; you have to construct your operations with the basic building blocks. These provide matrix creation, I/O, cellwise operations, reduction, multiplication, transformation, decomposition (LU, Householder, Choleski, and Jacobi SVD; these include general linear solvers) and a small statistics module by yours truly.


IMPORTANT: Posting Rules for this thread:
1) :naughty: Do not post or PM me any matrix-, maths-, or Eigen-related questions. Eigen has its own User Forum for that (or try math.stackExchange.com). I am not your maths guru! If you post such questions, I will either ignore your post or remind you of this rule.

2) :naughty::naughty: Do not post or PM me your data sets and/or non-working Eigen4AutoIt scripts; I will not analyse your data or fix your scripts for you! There are many reasons why a linear algebra procedure might fail to produce the answer you expect. You are wielding great mathematical power here, so exploit the fantastic internet resources at your fingertips and learn how to use it. To get you started, I've listed a few video tutorials and other helpful materials in the header remarks of Eigen4AutoIt.au3. Also check out the test scripts, the Tutorials, and the Help file.

3) :thumbsup: I do warmly welcome all of the following:

  • remarks, advice, suggestions for improvements, encouragement, cash;
  • bug reports re. the Eigen4AutoIt interface (individual functions that don't work the way they should) and/or the associated dll code (ditto);
  • your own working Eigen4AutoIt templates of general use that you'd like to see implemented in a future release.

Regarding that last item, have a look at my PCA tutorial. After the step-by-step stage, I summarise the entire procedure in a "mini script" of Eigen4AutoIt calls. Then I introduce the two internal PCA functions I developed, which replace that script with two simple calls. You can do the same thing, and submit your own functional Eigen4AutoIt script in this thread. If I consider it of general use and can implement it, it may become a new Eigen4AutoIt function in the next release (with source acknowledgement, of course). This means that you'd get a precompiled dll version of your script that will likely run faster and be simpler to call. Thereby this thread would become an Eigen4AutoIt Example Scripts mini forum. It's just a thought. :mellow:

>Download the latest version (uncompressed size: 26 MB)


How to Install

You need AutoIt version 3.3.12 or later. Extraction (with 7-zip) creates subdirectory Eigen4AutoIt, where you'll find Eigen4AutoIt.ini. Open it, find global variable $EIGEN_DLLPATH, and copy/paste the full absolute path (without trailing backslash) where the Eigen4AutoIt dlls are located on your machine. Save the ini file, open the first tutorial ("intro") in Scite, and start it. This shows basic matrix I/O and mode switching (single versus double precision). If that runs, you're in business.

Edited by RTFC
2 people like this

Share this post

Link to post
Share on other sites

It starts running for me with following function _CallStack_Push($procname) in file Eigen4AutoIt.au3:

Func _CallStack_Push($procname)

    If $procname="" Then Return
;   _ArrayAdd($Callstack,$procname,0,Chr(0))


and with true path for $EIGEN_DLLPATH - folder with file Eigen4AutoIt.au3.


The point of world view

Share this post

Link to post
Share on other sites

#3 ·  Posted (edited)

@ValeryVal: Thanks for your response.

Perhaps I should have emphasised that this library is written for AutoIt version 3.3.12; anything array-related will probably fail in earlier versions, because of the recent script-breaking changes in Array.au3. I'll stress this in the first post. Thanks a lot.

You will likely encounter other issues later on if you attempt to run in earlier AutoIt versions, even if the first tutorial runs normally. Please upgrade to AutoIt 3.3.12 before testing. See also the remarks in the "How to Install" section in the QuickStart Manual. Best of luck!


Edit: Okay, I've added an AutoIt version check to alert people to this. Thanks again for flagging this, ValeryVal.

Edited by RTFC

Share this post

Link to post
Share on other sites


Can you die from laughter? I have your very long test scripts completed with success.


The only note (apart from noted above) in MatrixDisplay.au3 is for me:

;   Local $asHeader = StringSplit($sHeader, $sCurr_Separator, $STR_NOCOUNT) ; No count element
    Local $asHeader = StringSplit($sHeader, $sCurr_Separator, 2) ; No count element

Would you like to add source of Eigen4AutoIt-QuickStart_Manual.pdf in any form (.doc, html, sdt) to simplify the translation it into Russian?

Thank you.

The point of world view

Share this post

Link to post
Share on other sites

#5 ·  Posted (edited)

Hi ValeryVal,

That's fast work! Glad you enjoyed the ride. :bike:

Regarding the MatrixDisplay.au3 script, the current trend in AutoIt is to move away from "magic numbers" and use sensibly-named constants (this one is in StringConstants.au3 in AutoIt version 3.3.12), and as Eigen4AutoIt is supposed to be run in that environment (or future versions) I'll keep using that constant, to make the code more transparent and legible to future users. You are of course completely free to edit it as you see fit for your own environment; whatever works for you. ;)

I'll PM you the powerpoint source of the manual; a Russian translation would be great! :thumbsup:  I hope you'll post a link to the result in this thread so others may benefit from it too. I'm not adding the powerpoint source to the package because I've almost run out of forum file storage space myself, and most users won't need it. Anyone else wishing to provide a translation, just PM me. ^_^

Edited by RTFC

Share this post

Link to post
Share on other sites

Beta Version 0.8 released. :)

Statistics module: several bug fixes plus new functions for skewness and kurtosis (biased and unbiased).

Share this post

Link to post
Share on other sites

#8 ·  Posted (edited)

Fantastic, ValeryVal, it looks great! :)

I hope you can advertise it at autoit-script.ru too (if you're a member there).

Your contribution should definitely help to make this environment more accessible to Russian-speakers.

Thank you very much for your hard work! :thumbsup:



Edit: in a PM to ValeryVal I mentioned that he missed translating my favourite Spoon Boy Matrix quotation, hence the following response...

Edited by RTFC

Share this post

Link to post
Share on other sites


Quick Tutorial of EigenAutoIt has been fixed (see above). Now spoon is nowhere in accordance to Spoon boy.





1 person likes this

The point of world view

Share this post

Link to post
Share on other sites

#11 ·  Posted (edited)

Beta version 0.9 released. :)

Aside from bug fixes in the <alternative operator> lookup, and, following a suggestion by ValeryVal, an extra manual page detailing what outputs are actually produced by each decomposition (and associated with which matrix letter), several new functions are added:

Conditional Cwise Operators:


These are Cwise operations acting on the entire matrix that are performed only on cells that satisfy a condtion (equal/unequal/smaller/larger/smaller-or-equal/larger-or-equal ) with respect to a separately parsed scalar value. See new test script _EIgenTest_12_ConditionalCwise.au3 for details.

Conditional evaluation:

_Eigen_ConditAll         ; returns True if all cells satisfy condition, otherwise False
_Eigen_ConditAny       ; returns True if any cell satisfies condition, otherwise False
_Eigen_ConditCount   ; returns number of cells that satisfy condition

See new test script _EIgenTest_12_ConditionalCwise.au3 for details.

These act either on the entire matrix or on a specific Col or Row.

In this context, I should mention that MatrixSpecs ID 16 (nonZeros) does not return the number of cells currently containing the value zero, :huh:  but the number of cells that have ever been assigned any value. So you'll need _Eigen_ConditCount to obtain the number of cells containing zero (or any other value).

Data retrieval from $MatrixList:

_Eigen_GetMatrixCols     ; return specified dimension for parsed matrix ID
_Eigen_GetMatrixRows   ; return specified dimension for parsed matrix ID
_Eigen_GetMatrixType     ; return matrix type string for parsed matrix ID


Unlike _Eigen_Transpose, this function leaves data storage unchanged, but changes the (vector or) matrix shape. So one could use it to change a rowvector to a colvector (without transposition). Note that this only works if the total number of cells (given the new user-defined dimensions) remains the same. Example:

Original 3 x 2 matrix

1  4

2  5

3  6

Transposed 2 x 3 matrix:

1  2  3

4  5  6

new ReDim  2 x 3 matrix:

1  3  5

2  4  6

I am currently starting work on a separate Import/export utility, to transfer data between matrix files and other file types, but this will take a while yet to finish.

Finally, I am happy to announce that Eigen now has a link to the Eigen4AutoIt thread here. :party::cheer:

Edited by RTFC

Share this post

Link to post
Share on other sites

Quick Tutorial of EigenAutoIt (Russian - v.0.9) has been fixed (see above).


The point of world view

Share this post

Link to post
Share on other sites

#13 ·  Posted (edited)

Version 1.0 released, now with separate MatrixFileConverter utility. :party:



Various minor issues fixed (bounds checks, matrix type conversion).

New UDF: _Eigen_RedimExistingMatrixFile(), works as _Eigen_RedimExistingMatrix(), but acting directly on a matrix file (editing its header), so no need to load the full matrix into memory first.

The MatrixFileConverter is a GUI that allows you to transfer and convert numerical content between binary matrix files as generated by Eigen4AutoIt on the one hand, and three external file formats: 1) ASCII delimited (.txt, .csv, .sdf, ...) with a choice of delimiters, 2) Excel Worksheets (provided you have Excel installed), or 3) Xbase files (dbf database files). Usage is pretty intuitive: define a matrix file and a target file, select the direction of transfer, and press the big Convert button. If a filename is not found, it is created in the specified directory.

Some formatting options (scientific notation, binary storage) are available from the GUI, others (such as the Xbase signature byte) can be altered by editing the auto-generated MatrixFileConverter.ini file.

If you need to convert lots of files, you may find the cmdline options useful (see Remarks in script header for a list of switches).

Options involving complex numbers are not yet functional in this release.

Note that unlike ASCII and Xbase conversions (which transfer directly between files), the Excel option uses an AutoIt array as intermediate buffer, and is thereby restricted to a maximum of 16 MB elements. If you need to import/export larger Excel files, use the CSV (comma-separated values) ASCII format as external intermediate instead. This is also the way to handle data in Matlab matrix files; like Excel, Matlab has its own sophisticated file im/export utility for that.

Finally, Xbase aficionados will be familiar with the limitation on the number of database fields (127 in dBase III / IV, 2047 in Xbase). To circumvent this, I've added a custom feature to store wider matrices (2D) as vectors (1D, ColMajor order); when converting such output back to a matrix file later on (with MatrixFileConverter!), the 2D structure (temporarily squeezed in a few rarely-used header fields) is rebuilt. For details on the Xbase specification, see the new include Xbase.au3 (link in signature).

Edited by RTFC

Share this post

Link to post
Share on other sites

#14 ·  Posted (edited)

Version 1.1 released. ^_^

Mainly a housekeeping update; there were still several inconsistencies between the AU3 wrappers and the dll wrappers.

There are several undocumented experimental functions involving complex datatypes in the dll for which no AU3 wrapper yet exists.

The MatrixFileConverter had a display issue with verylongpathnameswithverylongfilenames that were truncated incorrectly in the display.

The most significant internal change is a substantial expansion in the number of alias wrappers for many UDFs, making it much easier if you sort of approximately remember what a function was called, but not exactly (as happens to me a lot). :ermm::idiot: This also addresses several function naming convention issues (re. section splitting) that are now increasingly standardised (at least the alias is). All original UDFs names will remain functional.

A few (very minor) UDFs were also added:
_Eigen_ShowUnaryOperators            ; list the available options

Edited by RTFC

Share this post

Link to post
Share on other sites

#15 ·  Posted (edited)

Version 1.2 released, with all new, 1.2 MB Help File! o:):sorcerer:

See the Changelog in the Help file for all the latest changes, including a few new functions (mainly for display and MatrixList admin). The most significant change is a major overhaul of the function naming conventions. Writing the Help file made me aware of a number of inconsistencies which I tried to address. However, the old names still work as well (and will do indefinitely), thanks to additional alias wrappers. All test scripts have been edited to reflect the various changes.

The three Tutorials are also in the Help file, with lots of screenshots and some graphics where appropriate.

The >download link has been moved to Miscellaneous downloads, because the package is now far too big for me to host myself.

There are two script-breaking changes in version 1.2:

  • Changed: _EigenBinaryOp and all its subfunctions had their parameter order changed so the last two are now always: $operator, $matB. THIS IS A SCRIPT BREAKING CHANGE.
  • Changed: the large majority of functions now all behave in the same way in returning either (non-zero) target matrix ID or True upon success and False + nonzero @error flag upon failure. This affects for example all decompositions, which formerly returned an Eigen status code (now incorporated into error message upon failure). Notable exceptions are: all work environment functions (no return value), Show_<some list> functions (no return value), and functions that return a single value directly. THIS IS A SCRIPT BREAKING CHANGE.


I am grateful to everyone who contributed to the AutoIt help pages, which I took as my style template, to make it as easy as possible for AutoIt users to delve into Eigen4AutoIt. Having never before written a Help file myself, I never realised how much effort it takes to build one. Mine is of course still in its infancy, and doubtless still contains lots of typos, broken links, etc. Anyway, it's a start. :sweating:

Edited by RTFC

Share this post

Link to post
Share on other sites

#16 ·  Posted (edited)

Forgot to mention that I've upgraded Eigen (and my dlls) to its latest release 3.2.2.

As you may have noticed, this causes an issue with Eigen's own assert messages no longer being displayed in some environments. I'm currently fixing this, but it should only affect the TestAssert function I call in the first Tutorial, and any malformed function calls that get past Eigen4AutoIt's own bounds checks (which is not that easy to do). Valid code is not affected.

Edit: Dlls updated, asserts should work now (in Debug mode). :)

Another detail I forgot to mention yesterday is that none of the Help file topics yet has an Example snippet. These will be added in the near future.

Stay tuned...

Edited by RTFC

Share this post

Link to post
Share on other sites

#17 ·  Posted (edited)

Well, the future is now, so all Example code snippets have been added to the Help file, in the latest version: 1.3. :)

Still very much struggling with html-encoding, as you can see from the ridiculous text colours (it looks fine until I generate the final output :ranting:) and countless extra links that I never consciously inserted. :huh2:  Oh well, it's better than nothing, I suppose. :unsure: I'll be taking a little break from this Hellp file now...

Some handy new CreateMatrix functions with predefined contents have been added to the mix, plus various minor edits and improvements in code and Help file, and a nasty type-conversion bug was "retired" in the dll (Eigen version 3.2.2). Full details in the Changelog in the Help file.


Edit: two link-pruning sessions later (no, I couldn't let it lie), the large majority of stray links have now been expunged from the Help file. :sweating: The ugly colours we'll have to live with for now. The package has been updated under the same version number 1.3.

Edited by RTFC

Share this post

Link to post
Share on other sites

@ValeryVal: Wow, :o  are you sure you want to do this? I mean, it would be fantastic, :thumbsup:  but it's a huge job. I'm very impressed that you'd even consider such an undertaking. One thing though, this afternoon I uploaded a cleaned-up version 1.3, from which all stray hyperlinks in the Example code snippets have been removed (and there were many, many hundreds of them). :sweating:  You should preferably use this latest version (uploaded today) as your template, because many of those links I removed directed the user to the wrong destination pages.  There were also a few broken internet links I fixed.

The Help file won't undergo such large revisions in future (I hope), only individual functions being added, and maybe another Tutorial or so. First, I'll be focussing on other new parts of this environment...

Again, thanks for the vote of confidence, ValeryVal. :bye:  Perhaps other Russian-speaking forum members can be found to lend you a hand?

Share this post

Link to post
Share on other sites

>but it's a huge job

AutoIt Help (Russian edition) was also a huge job (since 1998, thanx Jon!), I think.

>The Help file won't undergo such large revisions in future (I hope)

It will be translated slowly and thoughtfully, I repeat

>Perhaps other Russian-speaking forum members can be found to lend you a hand?

We have here In Russia a motto: John Smith makes his work alone...

Thank you for your hard work and kindly words


The point of world view

Share this post

Link to post
Share on other sites

#20 ·  Posted (edited)

:cheer: :cheer: :cheer:

Celebrating 100 downloads since moving v1.2 to section Misc, and releasing version 1.4, a major upgrade. :dance:

Two new Tutorials, new advanced statistics, a significant number of new matrix management functions for cutting and pasting, more control over error handling, a complete overhaul of the alias wrappers (which now parse any error back up the calling chain), many new Help file topics, and the MatrixFileConverter utility can now handle free-format space-delimited ASCII, and add new Excel worksheets to store new data. More details in the Changelog in the Help file.

All this adds up to another megabyte of content (total package (uncompressed) ca. 7.5 MB now), and over 260 non-internal _Eigen_ functions. :)

This release was mainly driven by my first attempt to use this environment for actual work, which brought to light some glaring omissions in basic matrix management (mostly re. creating new matrices from data subsets in other matrices). If you identify similar omissions in basic functionality, please let me know; I may be able to implement a fix fairly quickly.


@ValeryVal:great work on the Help file translation into Russian!

Edited by RTFC

Share this post

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Similar Content

    • RTFC
      By RTFC
      The Eigen4AutoIt environment (thread is here:
      online Help is here) allows you to do matrix I/O (memory & files), matrix arithmetic, transformation, and decomposition, solve systems of linear equations, and perform advanced statistics, all at either single or double precision. Most functions can act on real or complex matrices (or the latter's real/imaginary parts separately). Much of the actual complexity of using Eigen in its native C++ environment has been hidden for AutoIt users, through extensive bounds and error checks, an intuitive function-naming convention, a large help file, and detailed tutorials and test examples.
      This library allows you to perform fast matrix operations on large numerical data sets, using special matrix variables and simple AutoIt wrapper functions. These wrappers call C++ wrappers in the dlls (also written by RTFC, source included). The dlls in turn re-map AutoIt memory and call one or more Eigen functions. All operations are memory pointer-based and act directly on the matrices created in your AutoIt script, obviating the need for data transfers. Both 32-bit and 64-bit environments are supported; in x64, matrices can be any size that fits into virtual memory.
    • AlecSadler
      By AlecSadler
      Hello friends! I have been working on an encryption algorithm in autoit as a proof of concept for some time now. Basically the algorithm uses a progressive recursion to encode data inside a matrix using a key that changes according to the date-time of the system, which is extracted from a larger key array. Recently after a drive failure, I lost the source and had to start from scratch, now I can't quite get it working the way it was before, and I can't see what I'm doing wrong, if anyone who understands matrix math or encryption could help I would much appreciate it. The problem is that the values returned by the decryption (extraction) process are way too big.
      I have figured out the solution to my problem, it was a typo, please disregard this thread.
      I will post my project into example scripts when it's ready.
    • Edano
      By Edano
      i searched the forum, but didn't find anything.
      has anyone an idea or ever done it with autoit ? simple image transforming, like sharpness, brightness, blurring etc. ?
      maybe, does gdiplus.dll feature that ?
      here is a tutorial on how to do it the hard (mathematical) way. http://lodev.org/cgtutor/filtering.html before i try that, i wanted to know if there is already an existing project or an idea or anyone has experiences.
    • logmein
      By logmein
      Hi there!

      I have been learning linear algebra in my university for months. The subject was rather hard, I did't understand it much so I sent a email to my teacher to ask him. But I realized that typing a matrix by text was extremely hard. I don't want to make a LaTex function, save to a file, attach it to the mail, bla bla bla!!! It's complicated! So I spent 2 hours yesterday to make this tool. Just type the number of lines and columns and matrix values, it will generate a quite nice matrix in ASCII characters
      Notes: It will rearrage your numbers to straight columns.
      1x1 will cause error.

      ConsoleWrite (@CRLF & '------------------------------------MATRIX TEST-----------------------------------------' &@CRLF) Dim $c = 6;column Dim $l = 4;line Dim $n = '1,6,2,4,b,a,7,6,6,0,44,4,6,3,6,2,6,8,9,6,6,1,2,logmein' $split = StringSplit ($n,',') If $c * $l <> $split[0] Then ConsoleWrite ('! Matrix values number do not match the columns and lines !' & @CRLF) Exit EndIf Dim $char[$l][$c], $len[$l][$c], $maxlen[$c][3] For $i = 0 To $l-1 For $u = 0 To $c -1 $v = $i*($c) + $u +1 $char[$i][$u] = $split[$v] $len[$i][$u] = StringLen ($split[$v]) If $len[$i][$u] > $maxlen[$u][0] Then $maxlen[$u][0] = $len[$i][$u] $maxlen[$u][1] = $i $maxlen[$u][2] = $u EndIf Next Next Local $finallen For $u = 0 To $c - 1 $finallen += $maxlen[$u][0] Next ConsoleWrite (' _' & _Repeat(' ',$finallen + $c) & '_' & @CRLF) For $i = 0 To $l-1;write lines For $u = 0 To $c-1;write columns $v = $i*($c) + $u +1 $char[$i][$u] = $split[$v] Switch $u Case 0; the first column If StringLen ($char[$i][$u])<$maxlen[$u][0] Then ConsoleWrite ('| ' & $char[$i][$u] & _Repeat(' ', $maxlen[$u][0]-StringLen ($char[$i][$u])+1)) Else ConsoleWrite ('| ' & $char[$i][$u] & ' ') EndIf Case 1 To $c-2 ;middle columns If $i > 0 Then If StringLen ($char[$i][$u]) < $maxlen[$u][0] Then ConsoleWrite ($char[$i][$u] & _Repeat(' ', $maxlen[$u][0]-StringLen ($char[$i][$u])+1)) Else ConsoleWrite ($char[$i][$u] & ' ' ) EndIf Else If StringLen ($char[$i][$u]) < $maxlen[$u][0] Then ConsoleWrite ($char[$i][$u] & _Repeat(' ', $maxlen[$u][0]-StringLen ($char[$i][$u])+1)) Else ConsoleWrite ($char[$i][$u] & ' ' ) EndIf EndIf Case Else ; the last column If StringLen ($char[$i][$u])<$maxlen[$u][0] Then ConsoleWrite ($char[$i][$u] & _Repeat(' ', $maxlen[$u][0]-StringLen ($char[$i][$u])+1) & ' |' & @CRLF) Else ConsoleWrite ($char[$i][$u] & ' |' & @CRLF) EndIf EndSwitch Next Next ConsoleWrite ('|_' & _Repeat(' ',$finallen + $c ) & '_|' & @CRLF) Func _Repeat($chars, $times);repeat a character in a specified time Local $rChar For $a = 1 To $times $rChar &= $chars Next Return $rChar EndFunc ;==>_Repeat