The Eigen C++ template library is a great environment for matrix computing; it is fast, reliable, extensive, and well-documented. It is also completely free, and does not rely on any external dependencies. Unfortunately for AutoIt users, the term “template library” implies that any functions you call are only instantiated upon compilation (in C++). That means there's nothing to hook into.
To make Eigen ’s most important functionality directly accessible from within AutoIt scripts (version 3.3.12+, download it here), I developed the Eigen4AutoIt environment. It runs on Windows and under Wine also on Linux and Mac (Ubuntu, Debian, Fedora, and MacOS supported by WineHQ), and SUSE, Slackware, and FreeBSD supported by the distros).
>Download the latest version
It consists of:
an AutoIt library of wrapper functions that contain extensive bounds checks, matrix management, file I/O, type conversion, and precision control, and two-way data exchange with files and native AutoIt arrays;
2) Eigen-wrapper dlls (EigenDense.dll, EigenDense_x64.dll)
re-maps matrices created in AutoIt as Eigen Matrix objects, then calls Eigen’s powerful core functions; the single-precision part of the C++ source code I wrote is included in the bundle (the double-precision, integer, and complex alternatives just duplicate this with different memory mappings and variable types). The basic functions consist of a single Eigen call; decompositions and statistics are more involved. 3) Additional materials:
the user-interactive, animated Function Selector and MatrixViewer tools the MatrixFileConverter to read/write E4A matrices from/to .csv ASCII, Excel, and Xbase files. three libraries of scientific and mathematical constants online Help Large (.chm) Help file Quickstart Manual (11-page pdf, updated) Test suite Tutorials from Basics to Advanced Please note:
none of this is part of Eigen's own distribution you only need this bundle; you do not need to install Eigen. How it works:
No matrix content is ever transferred, only memory pointers, meaning computations in AutoIt are virtually as fast as in Eigen’s native environment, with the added advantage of not having to be compiled first. The drawback is that Eigen's native ad-hoc expression templates (and their internal optimisations) cannot be exploited here; you have to construct your operations with the basic building blocks. These provide matrix creation, I/O, cellwise operations, reduction, multiplication, transformation, decomposition (LU, Householder, Choleski, and Jacobi SVD; these include general linear solvers) and a small statistics module by yours truly.
IMPORTANT: Posting Rules for this thread:
1) Do not post or PM me any matrix-, maths-, or Eigen-related questions. Eigen has its own User Forum for that (or try math.stackExchange.com). I am not your maths guru! If you post such questions, I will either ignore your post or remind you of this rule.
2) Do not post or PM me your data sets and/or non-working Eigen4AutoIt scripts; I will not analyse your data or fix your scripts for you! There are many reasons why a linear algebra procedure might fail to produce the answer you expect. You are wielding great mathematical power here, so exploit the fantastic internet resources at your fingertips and learn how to use it. To get you started, I've listed a few video tutorials and other helpful materials in the header remarks of Eigen4AutoIt.au3. Also check out the test scripts, the Tutorials, and the Help file.
3) I do warmly welcome all of the following:
remarks, advice, suggestions for improvements, encouragement, cash; bug reports re. the Eigen4AutoIt interface (individual functions that don't work the way they should) and/or the associated dll code (ditto); your own working Eigen4AutoIt templates of general use that you'd like to see implemented in a future release. Regarding that last item, have a look at my PCA tutorial. After the step-by-step stage, I summarise the entire procedure in a "mini script" of Eigen4AutoIt calls. Then I introduce the two internal PCA functions I developed, which replace that script with two simple calls. You can do the same thing, and submit your own functional Eigen4AutoIt script in this thread. If I consider it of general use and can implement it, it may become a new Eigen4AutoIt function in the next release (with source acknowledgement, of course). This means that you'd get a precompiled dll version of your script that will likely run faster and be simpler to call. Thereby this thread would become an Eigen4AutoIt Example Scripts mini forum. It's just a thought.
>Download the latest version (uncompressed size: 31.4 MB)
How to Install
You need AutoIt version 3.3.12 or later. Extraction (with 7-zip) creates subdirectory Eigen4AutoIt, where you'll find Eigen4AutoIt.ini. Open it, find global variable $EIGEN_DLLPATH, and copy/paste the full absolute path (without trailing backslash) where the Eigen4AutoIt dlls are located on your machine. Save the ini file, open the first tutorial ("intro") in Scite, and start it. This shows basic matrix I/O and mode switching (single versus double precision). If that runs, you're in business.
I am working on a GUI program to update Google's Dynamic DNS (API at https://support.google.com/domains/answer/6147083?authuser=1&hl=en if you scroll to bottom). I am not a programmer by any means - just a sysadmin who has picked up on some things along the way. I am sure that there's better ways to do a lot of things in this script; I'm just going with what I know.
My challenge right now is that I'd like a better way to store the credentials both in memory as well as in system registry or INI file (not sure which way I want to go for local storage). How should I convert the passwords to a secure string in a manner that can't be easily reversed, yet is still accessible to the script? Is that even an option in AutoIt?
Can anybody provide me with links to good reference posts, or coding suggestions for how best to achieve this in the script below? I am using the WinHTTP UDF (https://github.com/dragana-r/autoit-winhttp/releases) to make my API calls.
#include<WinHTTP.au3> #include<GUIConstantsEx.au3> #include<EditConstants.au3> #include<iNet.au3> #include<Array.au3> DIM $aDomainList $aDomainList = 0 $gMainGUI = GUICreate("Overkill's Google DNS Updater",800,800) $gDomainLabel = GUICtrlCreateLabel("FQDN",21,8) $gDomainInput = GUICtrlCreateInput("",60,5,300) $gUserLabel = GUICtrlCreateLabel("Username",5,36) $gUserInput = GUICtrlCreateInput("",60,32,130,Default,BitOR($GUI_SS_DEFAULT_INPUT,$ES_PASSWORD)) $gPasswordLabel = GUICtrlCreateLabel("Password",6,64) $gPassInput = GUICtrlCreateInput("",60,60,130,Default,BitOR($GUI_SS_DEFAULT_INPUT,$ES_PASSWORD)) $gAddButton = GUICtrlCreateButton("ADD DOMAIN",200,31,160,52) $gCurrentIP = GUICtrlCreateLabel("Current IP: " & _CheckIP(),5,780) $gDomainList = GUICtrlCreateListView("Domain | Resolved IP | Update Status",5,120,600,600) GUISetState(@SW_SHOW,$gMainGUI) while 1 $m = GUIGetMsg() IF $M = $GUI_EVENT_CLOSE then Exit IF $M = $gAddButton Then $sAddDomain = GUICtrlRead($gDomainInput) $sAddUser = GUICtrlRead($gUserInput) $sAddPass = GUICtrlRead($gPassInput) $sResolveIP = _DNSCheck($sAddDomain) ;Google wants you to avoid sending updates when there are no changes If StringCompare($sResolveIP,_CheckIP()) = 0 Then $sStatus = "No change, not sending update" Else $sStatus = _DNSUpdate($sAddDomain,$sAddUser,$sAddPass) EndIf ;Check to make sure all fields are completed before continuing IF StringLen($sAddDomain) = 0 OR StringLen($sAddUser) = 0 OR StringLen($sAddPass) = 0 Then MsgBox(0,"","Please complete all fields") Else ; If the fields all have data, then continue ;Check to see if the entry exists in the array already $iSanity = _ArraySearch($aDomainList,$sAddDomain) IF $iSanity = 0 Then _ArrayAdd($aDomainList,$sAddDomain & "|" & $sAddUser & "|" & $sAddPass ) If @error = 0 Then $aDomainList += 1 $aDomainList[$aDomainList] = GUICtrlCreateListViewItem($sAddDomain & "|" & $sResolveIP & "|" & $sStatus,$gDomainList) Else MsgBox(0,"","Error adding input to list") EndIf Else ; If $iSanity <> 0 ; Update existing info in array and listviewitem $aDomainList[$iSanity] = $sAddDomain $aDomainList[$iSanity] = $sAddUser $aDomainList[$iSanity] = $sAddPass GUICtrlSetData($aDomainList[$iSanity],$sAddDomain & "|" & $sResolveIP & "|" & $sStatus) EndIf ; If $iSanity = 0 EndIf ; If StringLen... EndIf ; If $m = $gaddbutton WEnd ;---------------------------------------------------------------------------------------- Func _DNSCheck($sFQDN) $sJSON = _INetGetSource("https://dns.google.com/resolve?name=" & $sFQDN & "&cd=1") ConsoleWrite($sJSON & @CRLF) $sIPAddress = StringRegExpReplace($sJSON,'^.*data": "(.*?)".*?$',"\1") Return $sIPAddress EndFunc ;---------------------------------------------------------------------------------------- Func _DNSUpdate($sFQDN,$sUser,$sPass) Local $sGoogleAPIURI = "https://domains.google.com" Local $hOpen = _WinHttpOpen() Local $hConnect = _WinHttpConnect($hOpen, $sGoogleAPIURI) Local $sHeader = _ 'Authorization: Basic ' & _Base64Encode($sUser & ":" & $sPass) & @CRLF & _ 'Accept: */*' & @CRLF & _ 'User-Agent: AutoITScript/' & @AutoItVersion & @CRLF & _ 'Content-Type: application/x-www-form-urlencoded' Local $aHTTPResponse = _WinHttpSimpleSSLRequest($hConnect, "POST", "/nic/update", Default, "hostname=" & $sFQDN, $sHeader, True, Default, Default, Default, True) _WinHttpCloseHandle($hConnect) _WinHttpCloseHandle($hOpen) If IsArray($aHTTPResponse) Then $sHTTPResponse = "Header:" & @CRLF & $aHTTPResponse & @CRLF & "Data:" & @CRLF & $aHTTPResponse & @CRLF & @CRLF & @CRLF Return $aHTTPResponse Else $sHTTPResponse = "NO REPLY" Return "No reply from " & $sGoogleAPIURI EndIf EndFunc ;---------------------------------------------------------------------------------------- Func _Base64Encode($sData) Local $oXml = ObjCreate("Msxml2.DOMDocument") If Not IsObj($oXml) Then SetError(1, 1, 0) EndIf Local $oElement = $oXml.createElement("b64") If Not IsObj($oElement) Then SetError(2, 2, 0) EndIf $oElement.dataType = "bin.base64" $oElement.nodeTypedValue = Binary($sData) Local $sReturn = $oElement.Text If StringLen($sReturn) = 0 Then SetError(3, 3, 0) EndIf Return $sReturn EndFunc ;---------------------------------------------------------------------------------------- Func _CheckIP() Return _INetGetSource("https://domains.google.com/checkip") EndFunc ;----------------------------------------------------------------------------------------
CodeCrypter enables you to encrypt scripts without placing the key inside the script.
This is because this key is extracted from the user environment at runtime by, for example:
password user query any macro (e.g., @username) any AutoIt function call any UDF call some permanent environment variable on a specific machine (and not created by your script) a server response a device response anything else you can think of, as long as it's not stored in the script any combination of the above You need several scripts to get this to work, and they are scattered over several threads, so here's a single bundle that contains them all (including a patched version of Ward's AES.au3; with many thanks to Ward for allowing me to include this script here):
Latest version: 2.3, (18 Jan 2018): CodeScannerCrypter.bundle.v2.3.7z
Note: if you experience issues under Win8/8.1 (as some users have reported), please upgrade to Win10 (or use Win7) if you can; as far as I can tell, the scripts in the bundle all work under Win7 & Win10 (and XP). Moreover, I have no access to a Win8 box, so these issues will not be fixed, at least not by yours truly.
How the bits and pieces fit together:
CodeCrypter is a front-end for the MCF UDF library (you need version 1.3 or later). Its thread is here:
'?do=embed' frameborder='0' data-embedContent>>
The MCF package (also contained in the CodeScannerCrypter bundle) contains MCF.au3 (the library itself) plus a little include file called MCFinclude.au3. The latter you have to include in any script you wish to encrypt. Any code preceding it will not be encrypted, any code following it will be encrypted. You define the dynamic key inside MCFinclude.au3, in the UDF: _MCFCC_Init().
From the same post you can download an MCF Tutorial which I heartily recommend, because encrypting a script requires a number of steps in the right order, namely:
In MCFinclude.au3, define and/or choose your dynamic key(s) (skip this step = use default setting) include MCFinclude.au3 in your target script Run CodeScanner (version 2.3+) on your target script, with setting WriteMetaCode=True (see '?do=embed' frameborder='0' data-embedContent>>), then close CodeScanner. Start CodeCrypter press the Source button to load your target file enable Write MCF0 (tick the first option in Main Settings) Enable "Encrypt" (last option in the Main Settings) Go to the Tab Encrypt and set up the encryption the way you want (skip this = use default settings) Return to Main Tab and press "Run" if all goes well, a new script called MCF0test.au3 is created in the same directory as your target. It has no includes and no redundant parts. Please check that it works as normal. (see Remarks if not) It all sounds far more complicated than it is, really.
Not convinced? Check out this updated and extended Q & A pdf (FAQ, also included in the bundle) to help you get started:
For additional explanations/examples in response to specific questions by forum members (how it works, what it can/cannot do), see elsewhere in this thread, notably:
Simple analogy of how it works: post #53, second part General Explanation and HowTo: post #9, 51, 75, 185/187, 196, 207, 270, 280 (this gets a bit repetitive) BackTranslation: post #179 Obfuscation: post #36 (general), 49 (selective obfuscation) Specific features and fixes: post #3 (security), 84 (redefining the expected runtime response), 169 (Curl Enum fix), 185/187 (using license keys), 194 (replacing Ward's AES UDF with different encryption/decryption calls), 251 (AV detection issue), 262 (extract key contents to USB on different target machine prior to encryption) Limitations: post #26 (@error/@extended), 149 (FileInstall), 191 (AES.au3 on x64) Not recommended: post #46/249 (static encryption), 102 (programme logic error), 237 (parsing password via cmdline)
BackTranslation is a test to check that the MetaCode translation worked. Skip it at your peril. It also turns your multi-include composite script into a single portable file without redundant parts (you can opt to leave the redundant parts in, if you want).
CodeCrypter can also obfuscate (vars and UDF names) and replace strings, variable names and UDF names with anything else you provide, for example, for language translation). After CodeScanner separates your target's structure from its contents, CodeCrypter (actually MCF, under the hood) can change any part, and then generate a new script from whichever pieces you define. See the MCF Tutorial for more explanation and examples.
Encryption currently relies on Ward excellent AES UDF, but you can replace this with any other algorithm you like; just edit MCFinclude.au3 UDF _MCFCC(), and MCF.au3 UDF _EncryptEntry(), see post #194 in this thread.
AES.au3, by Ward is now also included in bundle (with thanks to Ward), see '?do=embed' frameborder='0' data-embedContent>>
Going to lie down now...
MetaCode offers a way to:
separate a script's structure from its content remove all redundant definitions (globals and UDFs) change any content (and some structure) combine (new) structure and (new) content into a new script The most useful applications implemented so far are:
Fast language translation (not just text strings, also variable names and UDF names) Obfuscation (vars and/or UDFs) Script Encryption (conditionals, calls, and macros) Encryption is powerful because the key is not stored anywhere; you can define it to be a user password, macro, environment spec/variable, server response, something you define yourself, or a combination thereof; anything goes, as long as it's not a fixed string or fixed value. More info in the CodeCrypter thread: ?do=embed' frameborder='0' data-embedContent>'?do=embed' frameborder='0' data-embedContent>>
?do=embed' frameborder='0' data-embedContent>
But MetaCode has more potential than that; it allows you to tinker with any type of content separately, then rebuild a new version. So for example, you can have a single script structure and numerous different language modules you just plug in to create a new version in a different language.
A brief Tutorial is here:
The MCF library itself can be found in the CodeScannerCrypter bundle:
And a little example how to use it for translating your GUI into a different language:
UI_Translator.7z (new version that should work with the new version of Google Translate, see post #13 below)
MCF.au3 is just the library plus the MCFinclude.au3 file you need to include in any script you wish to encrypt.
There is no GUI here. However, I did write a separate front-end for it called CodeCrypter, which you can find here:
?do=embed' frameborder='0' data-embedContent>'?do=embed' frameborder='0' data-embedContent>>
?do=embed' frameborder='0' data-embedContent>
MCF uses output generated by my CodeScanner version 2.8+, which you can find here:
'?do=embed' frameborder='0' data-embedContent>>
CodeScanner also depends on MCF.au3 now, as it can now call a few of its functions.
I should also mention Ward's excellent AES.au3 UDFs used for the encryption and decryption calls, which is now included in the CodeScannerCrypter bundle (thanks to Ward for allowing to include it). You can find the original (unpatched) version here:
'?do=embed' frameborder='0' data-embedContent>>
Note: you can replace the encryption/decryption calls with whatever algorithm you like (hint: the native <Crypt.au3> library is too slow for most purposes, better stick to machine code routines)
So just to be clear:
CodeScanner (v2.8+) needs MCF (earlier versions won't work!) CodeCrypter needs MCF (plus anything that MCF needs) MCF itself needs MCFinclude (part of MCF zip) MCF also needs readCSdatadump (part of the CodeScanner package, you need the latest version packaged with CodeScanner v2.8; earlier versions won't work!) both MCF and MCFinclude currently rely on AES.au3 by Ward So you basically need to download the whole bundle for any of it to work.
If you have any questions, please start by reading the MCF Tutorial and the CodeCrypter FAQ (you can download the latter separately from the CodeCrypter thread).
Next, read the extensive Remarks sections in MCF.au3, MCFinclude.au3, and CodeCrypter.au3
If still no joy, then please post. However, I'm not online that often, and logged in to the forum even less, so response may take a while).
The Eigen4AutoIt environment (thread is here:
online Help is here) allows you to do matrix I/O (memory & files), matrix arithmetic, transformation, and decomposition, solve systems of linear equations, and perform advanced statistics, all at either single or double precision. Most functions can act on real or complex matrices (or the latter's real/imaginary parts separately). Much of the actual complexity of using Eigen in its native C++ environment has been hidden for AutoIt users, through extensive bounds and error checks, an intuitive function-naming convention, a large help file, and detailed tutorials and test examples.
This library allows you to perform fast matrix operations on large numerical data sets, using special matrix variables and simple AutoIt wrapper functions. These wrappers call C++ wrappers in the dlls (also written by RTFC, source included). The dlls in turn re-map AutoIt memory and call one or more Eigen functions. All operations are memory pointer-based and act directly on the matrices created in your AutoIt script, obviating the need for data transfers. Both 32-bit and 64-bit environments are supported; in x64, matrices can be any size that fits into virtual memory.