AutoIt Package Manager

[genius257 110]

Hello.

So for some time now, I've been thinking about the possibilities with a package manager for AutoIt, like npm for UDF's and such.

I am curious if anyone else would be interested in such a thing, or if it would be useless/problematic in practice?

My personal problem with the idea so far, is the hurdle of legality and hosting options.

Your thought(s) on this matter would be greatly appreciated.

 

Edit:

Project: https://github.com/genius257/au3pm

Latest release: https://github.com/genius257/au3pm/releases/tag/0.2.1

Download link: https://github.com/genius257/au3pm/releases/download/0.2.1/au3pm.exe

Edited January 20 by genius257
Relevant information in first post

[Danp2 1,033]

Did you see this recent thread?

 

[genius257 110]

4 minutes ago, Danp2 said:

Did you see this recent thread?

Hi @Danp2.

Yeah, sadly right after i posted this topic.
Story of my life

[Deye 113]

Hi genius257.

I say give it a go if you are ready & willing,  it will surely be interesting to see what you can come up with for this
I was starting it about over here  and then got caught up doing other stuff
personally it can help with the house keeping .. 

There can be 2 versions or ways to go about it :

1. - Doing as I described at the  previous Topic  @ coping the files to @scriptDir

2. - would be to edit the paths for the includes pointing to 1 folder that will be holding all the extras ..

Deye

[genius257 110]

Hi @Deye.

Thanks for the nice feedback

I guess I’ll get started today when I get home from work

About the files themselves I considered the "global" installation would download the module to the "Standard library" folder, in most cases that would be the /Include folder in the installation directory. The local installed modules would be in a folder maybe called au3_modules, and would work by adding a relative path to the registry key "Include" in the path "HKEY_CURRENT_USER\Software\AutoIt v3\AutoIt\".

With conflicting versions I see 2 possibilities:

• Report error and do nothing with the conflict.
• Parse the package and rename all functions with a version prefix.

I also plan on using this "guide" to get me started on the basic workings of the version management and structure.

[Deye 113]

Hi genius257,

I think the idea of grouping some modules into 7z packages can just make it easy to pull out single specific files as needed
* having something that makes a listing of files of @Scriptdir into an ini - listing all production modules and versions

* Anything that cannot be retrieved by this pack system remains siding the UDF along with its file versions stored in the ini .. 
Then maybe  if there can be some kind of switch for instance #AutoIt3Wrapper_module_Override = y : causing anything in @ScriptDir to gain priority over the installed autoit's lib file modules including the autoit.exe version ..  then it ca make thins even easier ..

One example of what this can be good for : For instance someone posts a UDF with its in_production ini file and you need to quickly test the examples attached without going through the hurdle of inspecting all its script breaking changes future-wise ..
another example is @ offering quick fixes or ways to update older UDFs for instance: https://www.autoitscript.com/forum/topic/167184-winapivhd-udf-winapiex-udf-extension/ ..

btw 
your StringRegExp() doesn't do:

#AutoIt3Wrapper_Res_File_Add=C:\Users\me\Desktop\88x31.jpg, RT_ICON ..

@ what i got so far:

expandcollapse popup

#NoTrayIcon
;Place it inside any module dir you want, Update its path from any other module dir paths by executing it stand alone ..
#pragma compile(Out, Get_au3_modules_Cmd.exe)
#AutoIt3Wrapper_Change2CUI=y
#include <File.au3>
If Not @Compiled Then Exit

If $CmdLine[0] = 0 Then
    $address = 'HKEY_LOCAL_MACHINE\SOFTWARE\Classes\AutoIt3Script\Shell\__Get_module_Copies\Command'
    If Not StringInStr(RegRead($address, ""), @ScriptFullPath) Then
        If Not IsAdmin() Then Exit ShellExecute(@ScriptFullPath, "", @ScriptDir, "runas")
        RegWrite($address, '', 'REG_SZ', '"' & @ScriptFullPath & '" "%1"')
        ConsoleWrite(' a new option called: "__Get_module_Copies)" has been added to the explorer *.au3 RC Context Menu ..' & @CRLF)
        Sleep(5000)
    EndIf
    Exit
EndIf

#include <WinAPIShPath.au3>
$sFile = FileRead($CmdLine[1])
$pattern = '(?(DEFINE) (?<string>(["''''])((?!\2).)*\2))(?:#include ((?&string))|FileInstall\(((?&string))|#AutoIt3Wrapper_Res_[a-zA-Z]+_Add=([^\n]+))'
$aResults = StringRegExp($sFile, $pattern, 3)
For $i = 0 To UBound($aResults) - 1
    If StringLen($aResults[$i]) > 0 Then
        $s = StringRegExpReplace($aResults[$i], '^(["''''])(.*)\1', "$2")
        $sPath = @WorkingDir & "\" & $s
        $sCopyDest = @ScriptDir & "\" & $s
        If FileExists($sCopyDest) Then
            FileCopy($sCopyDest, $sPath, 8)
            If Not @error Then
                ConsoleWrite(_WinAPI_PathCompactPathEx($sPath, 65) & "   -Copied" & @CRLF)
                $sFile = FileRead($sPath)
                $aResults1 = StringRegExp($sFile, $pattern, 3)
                For $j = 0 To UBound($aResults1) - 1
                    If StringLen($aResults1[$j]) > 0 Then
                        $s = StringRegExpReplace($aResults1[$j], '^(["''''])(.*)\1', "$2")
                        $sPath1 = StringLeft($sPath, StringInStr($sPath, "\", "", -1))
                        $sCopyDest = @ScriptDir & StringRight($sPath1, StringInStr($sPath1, "\", "", -2))
                        If FileExists($sCopyDest) Then
                            FileCopy($sCopyDest & $s, $sPath1 & $s, 8)
                            If Not @error Then
                                ConsoleWrite(_WinAPI_PathCompactPathEx($sPath1 & $s, 65) & "   -Copied" & @CRLF)
                            Else
                                ConsoleWrite($sFile & "   not found" & @CRLF)
                            EndIf
                        EndIf
                    EndIf
                Next
            Else
                ConsoleWrite($sFile & "   not found" & @CRLF)
            EndIf
        EndIf
    EndIf
Next

Sleep(2500)

Deye

Edited July 13, 2018 by Deye

[genius257 110]

Status update.

I've been stuck on a host for the package registry, and finally found a solution (at least for now), thanks to the introduction of GitHub Actions.

Currently the plan is only to support GitHub hosted repositories (represent packages).

The package manager solution I've started to implement is heavily inspired by NPM-cli and yarn package manager.

I plan on finishing initial code for the install command today, and pusing it to the au3pm repository, if anyone is interested in giving feedback or just for looking into it .

Initial install command will only support getting:

• custom github repo/commit
• au3pm package with semantic versioning support
• local packages

I will be happy to explain anything I've done so far, but i remind you that this is in the alpha stage so nothing is set in stone.

Links:

au3pm repository

au3pm package registry (currently does nothing)

au3pm dev package registry (currently used for the alpha testing with au3pm)

[seadoggie01 200]

I love this idea, I keep needing to update TaskScheduler.au3 from water at home and work and I'm tired of downloading the same thing over and over

Even if GitHub actions doesn't work out or goes away, you could still use releases from the GitHub API... I have some code at home for pulling released binaries from the latest version. I was going to use it for keeping programs up to date.

[genius257 110]

Hi @seadoggie01

Thank you

The GitHub Actions are intended only for the "official" au3pm package registry. It holds only the references to the GitHub repo and SHA of the release. GitHub Actions are needed to automate the registering of package releases on the package registry. This way if the au3pm "butler" script works as intended, members of the au3pm GitHub Organisation with Admin repository permissions will only need to make changes to the package registry in rare cases.

Ideally a proper package registry should be made, but as au3pm is in it's infancy and the number of expected users actually using it is unknown, this current solution is intended as a starting point. The package registry may very likely need to be hosted on a server environment, more akin to the NPM, should needed features and number of users using au3pm turn out to be of that scale.

I looked into using GitHub releases, but GitHub releases are finicky, as some releases will be listed on the releases endpoint and some will be listed in tags. Take into account there is no way to make sure a user does not adhere to expected version naming or renaming of releases. I would love to see your code to pull the binaries, i may simply not understand the GitHub releases API enough.

[seadoggie01 200]

TBH, I don't get all of what you just said, but I figured there was a good reason to use Actions besides it being shiny

I'm not great at programming with APIs, but I found the GitHub API and managed to find this: https://developer.github.com/v3/repos/releases/#get-the-latest-release. I started testing it with some code like this:

Spoiler

#include <CustomDebugging.au3> ; Debug($sMsg) -> Prints to console and log.txt
#include <Array.au3>

Global Const $fileName = @TempDir & "\releases.json"
Global Const $apiUrl = 'https://api.github.com/repos/'

_GitRel_AssetLatestGet('seadoggie01', 'Test')

Func _GitRel_AssetLatestGet($userName, $repository)

    ; build url
    Local $downloadUrl = $apiUrl & $userName & "/" & $repository & "/" & "releases/latest"
    Debug($downloadUrl)

    ; Download the json
    If InetGet($downloadUrl, $fileName, 1, 0) = 0 Then
        Debug("No file get")
        Return SetError(1, 0, False)
    EndIf

    Local $json = FileRead($fileName)
    
    ; Display all downloads
    _ArrayDisplay(StringRegExp($json, '(?i)(?U)"browser_download_url":\s*"([^]]*?)"', 3))

    ; Delete the json file
    ;~FileDelete($fileName)

EndFunc

 

It gets an array of the url of each asset that is added to the release. You can also get the zip/tar of the source code of the release as well, but I didn't write that... I thought I did more, but apparently not. (The repository in the code is real, see it here: https://github.com/seadoggie01/Test/releases)

[genius257 110]

Yeah i tend to ramble, sorry ^^'

Thank you for the code yeah it seems to be working just fine

I might look into using releases/tags with au3pm again at a later time, then

[genius257 110]

Small update post, to let you know I am still very much working on this.

Further development of au3pm, beside bug fixes are on hold, until i develop a better way of working with command line arguments (currently in progress).

pre-release version is available via Github (see link(s) in top post)

initial documentation is available, thanks to Github pages.

Currently the package manager bot is broken after the official launch of GitHub Actions, but i will be more than happy to add packages manually if you create an issue here.

Let me know if something is confusing about the implementation or usage, as i try to keep it as simple and user friendly as possible

Edited September 8, 2020 by genius257

[TheDcoder 1,334]

The project looks interesting, I once thought of making a package manager myself

However I have to say that I dislike npm, and the way it managed packages per folder... I like my packages to be in a central location, being shared with other scripts as needed. I guess everyone has their own preferences.

What I do like is how a GitHub repository can be served as a package (if I am understanding it correctly), it will be interesting to see how this project grows. Best of luck @genius257