array in array

[RTFC]

One thing you're doing wrong is calling CloneMatrix with 2nd param $copydata=False (so the new matrix is empty), and then immediately using that empty result in a multiplication. Otherwise, provided the adjacent dimensions match (that is, if A.cols()==B.transpose().rows(), so if A.cols()==B.cols()),

_Eigen_Multiply_ABt

should produce valid output. The default setting of $copydata is True, because most of the time when you clone, you'll want the same content as well as the same shape.

Edited May 2, 2018 by RTFC

[kylomas]

RTFC, FMS, jchd, 

Thank you for the insight and links!

Kylomas

[FMS]

Is there by any chanse an function to put all row's of a matrix inside a vector's cel in E4A?
like (code) from (image) :

Local $temp = 0

For $k = 0 To UBound($M_bias_h2,1) -1
    For $j = 0 To UBound($M_grad_h2,2) -1
        $M_bias_h2[$k] += $M_grad_h2[$k][$j]
    Next
Next

I've tried a lot but couldn't get any good results

 

 

[RTFC]

_Eigen_MatrixSpecs_Rowwise_Single ( $matA, $specID) with specID=2 (sum) does what you want.

NB not clear from your example whether vector M_bias_h2 is empty (zero) before, but if not, you'd collect the result in a temporary vector which you then add to M_bias_h2 with CwiseBinaryOp.

Edited May 6, 2018 by RTFC
typo

[junkew]

You can get your array back and forth into excel in 1 call directly no need to do it per cell (which is terribly slow)
Example below demonstrates the concept

$oExcel=objcreate("excel.application")
$oExcel.visible=1

$oWB=$oExcel.workbooks.add()
$oWS=$oWB.worksheets.add()

;~ putting it In
Local $aArray[2][2]
$aArray[0][0] = "A1"
$aArray[1][0] = "A2"
$aArray[0][1] = "B1"
$aArray[1][1] = "B2" 

$oWS.Range("A1:B2").value=$aArray

;~ getting it out, see you get a different range back including empty cells
$oVarArr=$oWS.Range("A1:C2").value

for $i=0 to ubound($oVarArr,1)-1
    for $j=0 to ubound($oVarArr,2)-1
        consolewrite("<" &  $i & ";" & $j & ";" & $oVarArr[$i][$j] & ">")
    Next
Next

 

[RTFC]

@junkew: Come again? Maybe you misunderstood that last question (about E4A, not Excel), or answered in the wrong thread? Or is this a glitch in the matrix?

Doing neural nets in Excel would be like building a fusion reactor with Lego bricks.

[junkew]

yep, was reading E4A as xls (>ლ)

yep, like also lego anyway so also excel is an option (i would switch for neural programming more to a compiled language anyway)

https://quantmacro.wordpress.com/2015/08/13/artificial-neural-network-with-backpropagation-training-in-vba/

The E4A library looks nice to do matrix calculations 

If the OP is still on arrays look into this thread

 

 

[RTFC]

I love Lego too.

37 minutes ago, junkew said:

a compiled language

Yeah, that's (part of) why E4A uses a compiled C++ dll. A major limitation of AutoIt arrays (regardless of access time) is the meagre size limit, whereas E4A matrices can be any size that fits into virtual memory (in x64 mode, that is, otherwise only <2GB).

[FMS]

Quote

Doing neural nets in Excel would be like building a fusion reactor with Lego bricks. 

Also mine riaction in a simelar line

I'm more in to it too understand the math/code behind the AI.

Also whit E4A the code looks more simpler i must say and mush better performanse than only "array-based"

A big plus

Is there by anyway a posibility to find the id of the higest in a matrix?
e.g. -> largest output in the "output layer -vector" so i can store it?

At this point I'm try to do something like this in vectors :

#include <Array.au3> ; also for multiping

;~ _Eigen_StartUp()

Global $aArray[10][1]
Global $bArray[10][1]


For $i = 0 to 9
   $aArray[$i][0] = 0
   $bArray[$i][0] = 0
Next

$aArray[Random(1 , UBound($aArray,1) - 1 , 1)][0] = 1
$bArray[Random(1 , UBound($bArray,1) - 1 , 1)][0] = 1

Global $correct = False
Global $guessed_id = -1
Global $target_id = -1

For $i = 0 to 9
   If $aArray[$i][0] = 1 Then $guessed_id = $i
   If $bArray[$i][0] = 1 Then $target_id = $i
   If $aArray[$i][0] = 1 And $bArray[$i][0] = 0 Then
      $correct = True
   EndIf
Next


_ArrayDisplay ($aArray)
_ArrayDisplay ($bArray)
MsgBox($MB_SYSTEMMODAL, "Title", "$guessed_id -> " & $guessed_id & " $target_id " & $target_id , 10)
If $correct Then MsgBox($MB_SYSTEMMODAL, "Title", "guessed -> correct!!"  , 10)

ofcource the outputs aren't 0 and 1 and i use _ArraySort($outputs_os, 1, 1, 0, 1) for this
($outputs_os[$i][2] is an array whit and id and a output of -1 to 1)

Edited May 4, 2018 by FMS

[RTFC]

Like your earlier question, you're taking a matrix to extract something dimensionally smaller from it, e.g., a vector (rowwise sum, or maybe a list of the frequencies of the unique values in the matrix) or in this case, a scalar (the maximum value). These are examples of so-called matrix reduction. In the E4A Help, go to the chapter heading "Reduction" (not the actual functions underneath it). At the bottom of the page you'll find a table with all scalar specs you can extract (sum, product, mean, and so forth). There you'll also find minimum and maximum value, plus their row/col coordinates. Note that not all matrix part function variants support all specs. But if it's just the max value you're after, you don't need to sort the matrix first (for sorting, see functions under the heading "Transformation").

All clear?

[RTFC]

@FMS:NB instead of using CwiseScalarOp with params "*",-1 (in the sigmoid function), using CwiseUnaryOp with param "-" (minus) should be slightly faster.

Furthermore, if you're serious about image recognition, your programme would likely benefit from a PCA preprocessing pass instead of feeding it raw image data. This is for example used in facial recognition software, google "eigenface" to learn more. Just a thought. Check out E4A's PCA tutorial if you're interested.

Edited May 6, 2018 by RTFC

[FMS]

Nice,

checked , tested and implemented (thanks @RTFC )

$quessed_id = _Eigen_MatrixSpecs_Single ( $M_outputs_o, 14 )
$quessed_value = _Eigen_MatrixSpecs_Single ( $M_outputs_o, 11 )
$target_id = _Eigen_MatrixSpecs_Single ( $M_target_o, 14 )

MsgBox($MB_SYSTEMMODAL, "Title", "guessed -> " & $quessed_id & @CRLF & "value -> " & $quessed_value  , 10)
MsgBox($MB_SYSTEMMODAL, "Title", "target -> " & $target_id  , 10)

also whit :

_Eigen_CwiseUnaryOp_InPlace ( $M_outputs_h1, "-" )
_Eigen_CwiseUnaryOp_InPlace($M_outputs_h1,"exp")
_Eigen_CwiseScalarOp_InPlace($M_outputs_h1,"+",1)
_Eigen_CwiseUnaryOp_InPlace($M_outputs_h1,"inverse")

Quote

Check out E4A's PCA tutorial if you're interested.

I've looked into it but I'm not shure what u mean I must say and also how I can benefit from " Principal Components Analysis " ? (maybe i don't undestand the math )
I'm inplementing the raw data into array's and afther "normelizing" it I put it in a Matrix whit :

Func normelize_input()
   Local $input_min = 0
   Local $input_max = 255
   For $i = 0 To UBound($A_inputs,1) -1
      $A_Ninputs[$i] = round(($A_inputs[$i] - $input_min) / ($input_max - $input_min),4)
   Next
   $M_Ninputs = _Eigen_CreateMatrix_FromArray ( $A_Ninputs, True )
EndFunc

I'm not realy aiming for "image recognition" , at first I want to make the math working .
The files (t10k-images.idx3-ubyte / t10k-labels.idx1-ubyte) are just files I found to test the math.
 

 

Edited May 7, 2018 by FMS

[RTFC]

The advantage of a precursor PCA pass is that your inputs will consist of uncorrelated contributions to data variability, which means any algorithm dealing with distinguishing between different inputs (to produce some desired different behaviours/responses) does not have to deal with covariant inputs, making the decision-making process much easier (because it does not have to disentangle raw parameters that mutually affect one another in some way). Facial recognition was just an example; in the linked page, you cansee lots of different pics of the same face; PCA can reduce each one to their common denominator,i.e., the basic face underlying all different expressions and lighting conditions. If you have lots of different people as inputs, PCA can define the common face underlying all, then define each person as how it departs from the common face. (Those two approaches can also becombined).

But if you're focussed on getting the math working, just forget about PCA for now (but remember it for later on, maybe). Think of it as a noise filter for the (artificial) mind.

[FMS]

Thanks for the info @RTFC , I'll look into it.

Seems intresting but I'm not shure how to implement this

Do you have an idea how to do the folowing?
I hope the code say's enough

I tried multiple functions but couldn't get it to work (most of it I get the error message complaining about mismatch in dimensions)

I'm sure that what I'm trying to do is a function in E4A but couldn't find it.
(still a little unsure about the naming but got the math down I think )

#include "Eigen4AutoIt.au3"
#include <Array.au3> 

_Eigen_StartUp()
Local $aArray[3][2] = [[1, 2], [3, 4], [5, 6]]
Local $bArray[3][1] = [[1], [2], [3]]
Local $cArray[3][1] = [[0], [0], [0]]
Global $matA
Global $matAT
Global $matB
Global $matC

;$cArray[0] = (($aArray[0][0] * $bArray[0]) + ($aArray[0][1] * $bArray[1] ) + ...)
;$cArray[1] = (($aArray[1][0] * $bArray[0]) + ($aArray[1][1] * $bArray[1] ) + ...)
;$cArray[2] = (($aArray[2][0] * $bArray[0]) + ($aArray[2][1] * $bArray[1] ) + ...)
;....

$matA = _Eigen_CreateMatrix_FromArray ( $aArray, True )
$matAT = _Eigen_Transpose ( $matA )
$matB = _Eigen_CreateMatrix_FromArray ( $bArray, True )
$matC = _Eigen_CreateMatrix_FromArray ( $cArray, True )

$matC = _Eigen_Multiply_AtB  ($matA,$matB)

_MatrixDisplay ( $matA, "$matA" )
_MatrixDisplay ( $matAT, "$matAT" )
_MatrixDisplay ( $matB, "$matB" )
_MatrixDisplay ( $matC, "$matC" )

_Eigen_CleanUp()

 

[RTFC]

Oh dear. Where to begin? Some clean E4A code maybe.

#include "..\Eigen4AutoIt.au3"

_Eigen_StartUp()

Global $matA = _Eigen_CreateMatrix_LinSpaced_RowMajor(3,2,1,6)
_MatrixDisplay ( $matA, "$matA" )

Global $matB = _Eigen_CreateMatrix_LinSpaced_ColMajor(3,1,1,3)
_MatrixDisplay ( $matB, "$matB" )

Global $matC = _Eigen_Multiply_AtB ( $matA, $matB )
_MatrixDisplay ( $matC, "$matC" )

_Eigen_CleanUp()

Some pointers:

• #include <Array.au3> is superfluous, because E4A already includes it.
• use Linspaced matrix creation functions to fill a matrix with regularly-spaced values (rows, cols, first value, last value)
• it's not wrong to predeclare all globals at the top of the script, but I wouldn't use 4 lines for 4 globals; in such a small test script, I'd declare the globals as they are created/filled with values
• you're using an internal-transpose function (little "t" in function name: _AtB); that means you don't (need to) create a separate transposed matrix A; the function instead takes the original A, and accesses it transposed. So no need to create $matAT at all.
• results container C is automatically created for you (unless you explicitly presupply it in the multiply call) with the correct dimensions. From your array definition, you seem to expect a matrix multiplication of [2,3] . [3,1] to produce a [3,1]- shaped output matrix. That's plain wrong. The multiplication itself is valid because the adjacent ("inner") dimensions of At and B are the same (3): [2,3] . [3,1]. The final result accumulates in a container defined by the outer dimensions, so [2,1], not [3,1], viz.: [2,3].[3,1] Maybe review the matrix multiplication chapter introduction in the Help file?
• If instead you wish to perform a cell-wise vector-multiply per row or column, look into _Eigen_CwiseBinaryOp_Colwise/Rowwise. Note that your vector B needs to have the right shape, that is, being either a Colvector [N,1]  or a RowVector [1,N], with N matching the corresponding dimension in matrix A. You already know how to obtain the Rowwise/Colwise sum (see MatrixSpecs_Row/Colwise_Single)
• If you're unsure how some function you're trying out is supposed to work, start by copying the example script in the E4A Help, run it (to see how it works), then edit it to suit your needs.

Hope it helps.

Edited May 9, 2018 by RTFC
typos

[FMS]

O, sorry @RTFC I see now that I left some testing code inside the test-script

The question i tried to ask was how to calculate :

$vectorB[0] = (($matrix[0][0]*$vectorA[0])+($matrix[1][0]*$vectorA[1])+($matrix[2][0]*$vectorA[2])+...)
$vectorB[1] = (($matrix[0][1]*$vectorA[0])+($matrix[1][1]*$vectorA[1])+($matrix[2][1]*$vectorA[2])+...)
$vectorB[2] = (($matrix[0][2]*$vectorA[0])+($matrix[1][2]*$vectorA[1])+($matrix[2][2]*$vectorA[2])+...)
...

sorry for the confusion.

Could you shine a light on the folowing?
I'm not shure how to release the matricies and thought all the used matricies where released in the code below.
Also is there a "best-habit" in making and releasing matricies in loops?

expandcollapse popup

#include "Eigen4AutoIt.au3"
Global $mat[3][3]

_Eigen_StartUp()

   For $x = 0 To 2
      $mat[$x][0] = _Eigen_CreateMatrix (3,2)
      $mat[$x][1] = _Eigen_CreateMatrix (3,1)
      $mat[$x][2] = _Eigen_CreateMatrix (2,1)
   Next
;~ _Eigen_Show_MatrixList()
feed()


_Eigen_Show_MatrixList()
   For $x = 0 To 2
      _Eigen_ReleaseMatrix ( $mat[$x][0] )
      _Eigen_ReleaseMatrix ( $mat[$x][1] )
      _Eigen_ReleaseMatrix ( $mat[$x][2] )
   Next
_Eigen_Show_MatrixList()

_Eigen_CleanUp()

Func feed()
      _Eigen_SetListMarker()
   For $x = 0 To 2
;~    _Eigen_SetListMarker()
      $mat[$x][0] = _Eigen_SetLinSpaced_RowMajor ($mat[$x][0],1,6)
;~    _MatrixDisplay ( $mat[$x][0], "$mat[$x][0]" )

      $mat[$x][1] = _Eigen_SetLinSpaced_RowMajor ($mat[$x][1],1,3)
;~    _MatrixDisplay ( $mat[$x][1], "$mat[$x][1]" )

      $mat[$x][2] = _Eigen_Multiply_AtB ( $mat[$x][0], $mat[$x][1] )
;~    _MatrixDisplay ( $mat[$x][2], "$mat[$x][2]" )
;~    _Eigen_ReleaseFromMarker()
;~    _Eigen_ReleaseMatrix ( $mat[$x][0] )
;~    _Eigen_ReleaseMatrix ( $mat[$x][1] )
;~    _Eigen_ReleaseMatrix ( $mat[$x][2] )
   Next
      _Eigen_ReleaseFromMarker()

;~    For $x = 0 To 2
;~    _Eigen_ReleaseMatrix ( $mat[$x][0] )
;~    _Eigen_ReleaseMatrix ( $mat[$x][1] )
;~    _Eigen_ReleaseMatrix ( $mat[$x][2] )
;~    Next

EndFunc

 

[RTFC]

Although E4A has various built-in safeguards to minimise the risk of memory leaks upon (unexpected) Exits, it's good practice to release matrices as soon as you no longer need them. What you do in your code snippets looks fine, using SetListMarker before creating some work spaces, and using ReleaseFromMarker immediately afterwards to clear them (I'm just a bit confused by storing these IDs in a 2D array, but that's up to you). Note that a variant of this function, ReleaseFromMarker_Except(), allows you to exclude up to 10 specified matrix IDs from that release action (e.g., to keep a final result while discarding all temporary buffers). Of course, releasing matrix allocations individually with _Eigen_ReleaseMatrix(<matID>) is also fine. Note that calling _Eigen_CleanUp() at the end of a session will automatically release all remaining used entries in the $matrixList for you. You can do this yourself at other times by calling _Eigen_ReleaseMatrix_All(). To review these functions, study Help sections "Work Environment" and "Matrix Management."

Since matrix IDs are sequentially assigned and slots in the list are not re-used during a session, you can also use _Eigen_ReleaseFromIndex_ToIndex to clear a specific block of IDs in a single call. If you need more control, remember that matrix IDs are just the index numbers of the entries in the $MatrixList array. You can use internal function _Eigen_GetNewMatrixID() to figure out ahead of time what the next free ID to be assigned to a newly created matrix will be.

The one thing you should NOT do is edit the $matrixList array directly yourself (with _ArrayDelete, _ArrayAdd, etc), as E4A needs to keep track of how much space to dynamically allocate to keep all the matrix admin correct. This is especially important in x64 (64-bit mode), where part of this admin is off-loaded to my HighMem UDF (which enables matrices of unlimited size (>2GB, up to whatever fits into your virtual memory (which can greatly exceed your physical RAM) that can be accessed simultaneously by multiple processes, which can hugely speed up calculations; but that's another story). So whatever you do, please use only the provided functions to manipulate the list. Speaking of x64, if you find that you're constantly running out of memory due to allocating too many large matrices, switch to x64 mode (if you have x64 hardware running a 64-bit OS, that is) by adding directive "#AutoIt3Wrapper_UseX64=Y" at the top of your script. This may also speed up some computations a bit.

Edited May 9, 2018 by RTFC

[RTFC]

1 hour ago, FMS said:

question i tried to ask

See the penultimate point in my previous answer:

If you wish to perform a cell-wise vector-multiply per row or column, look into _Eigen_CwiseBinaryOp_Colwise/Rowwise. Note that your vector B needs to have the right shape, that is, being either a Colvector [N,1]  or a RowVector [1,N], with N matching the corresponding dimension in matrix A. You already know how to obtain the Rowwise/Colwise sum (see MatrixSpecs_Row/Colwise_Single).

Edited May 9, 2018 by RTFC

[FMS]

I must say, now I'm getting to the end of a working "feed forward" and "back propagration" E4A is a usefull tool. 
Its a little work on getting to know all the correct functions for the matricies and vectors you have.
(naming wise I'm not getting the hang of it but know how it works )

On 9-5-2018 at 3:23 PM, RTFC said:

(I'm just a bit confused by storing these IDs in a 2D array, but that's up to you).

@RTFC - I'm using at this point 2d array's because in the futher I want to make 1 network (matricies id's ) in a row of the 2d array (weights  , inputs , outputs etc.)

At this point I'm looking at the floor and ciel functions.
I'm not shure if I'm having the right function for this but I want to set al the intigers in a matrix (or a vector) to 0 if it's below 0.
Or 1 if it's higer then 1.
(in the futher I think I want to use other max and min digits)

Is this possible in E4A ? (I think it does but couldn't find the right function)

 

[RTFC]

The perfect functions for bit-masking would be _Eigen_ConditMask/_Eigen_ConditMask_InPlace. However, there's a serious issue in the way Eigen implements the underlying .select() operator that I've only just fixed yesterday (well, circumvented, more like) for the next release (v4.3); ConditMask in version 4.2 should work, but ConditMask_InPlace in v4.2 is unstable (it will likely crash, or hang). So until 4.3 is ready for release, you'd best use a slightly more cumbersome double call of conditional replace, using _Eigen_ConditScalarOp/_InPlace (with "replace" operator).

Edit: actually, if you wish to retain true values in the [0,1] interval, then you shouldn't be using bit-masking, so stick with ConditScalarOp/_InPlace.

Edited May 16, 2018 by RTFC