Setting up CI on GitHub for autoit scripts?

[rcmaehl]

Added solution here: 

 

 

Hi all,

I'm attempting to setup CI on github for several of my projects. I've encountered a few issues I've been able to solve (namely tools exiting before they're complete), but my current issue is that I can't seem to get #AutoIt3Wrapper headers to be parsed. Unlike AU3Check and Aut2exe, using the same parameters SciTE shows being used does not seem to work. I'm wondering if I'm doing something wrong? Any advice on the matter is appreciated, and my CI yml is below.

name: lili

on: 
 push:
   paths-ignore:
     - '.github/**'
     - '!.github/workflows/**'
     - '*.md'

jobs:
  build:

    runs-on: windows-latest
    steps:
    - name: Install Autoit
      run: |
           Invoke-WebRequest -Uri https://www.autoitscript.com/cgi-bin/getfile.pl?autoit3/autoit-v3-setup.exe -OutFile autoit-v3-setup.exe
           ./autoit-v3-setup.exe /S
    - name: Checkout repo
      run: |
           cd C:\"Program Files (x86)"\AutoIt3\Aut2Exe
           git clone https://github.com/rcmaehl/LinuxLiveUSBCreator
           cd LinuxLiveUSBCreator
           Move-Item * ..
    - name: Install Autoit Code Stripper
      run: |
           Invoke-WebRequest -Uri https://www.autoitscript.com/autoit3/scite/download/Au3Stripper.zip -OutFile Au3Stripper.zip
           Expand-Archive -Force .\Au3Stripper.zip C:\"Program Files (x86)"\AutoIt3\Aut2Exe\
    - name: Wrap Code
      run: |
           cd C:\"Program Files (x86)"\AutoIt3
           ./AutoIt3.exe "C:\Program Files (x86)\AutoIt3\SciTE\AutoIt3Wrapper\AutoIt3Wrapper.au3" /NoStatus /prod /in "C:\Program Files (x86)\AutoIt3\Aut2Exe\sources\LiLi\LiLi USB Creator.au3"
           Start-Sleep -s 10
    - name: Compile
      run: |
           cd C:\"Program Files (x86)"\AutoIt3\Aut2Exe
           cd LinuxLiveUSBCreator
           $ver= git rev-list --count HEAD
           cd ..
           mkdir build
           ./Aut2exe.exe /in "sources\LiLi\LiLi USB Creator.au3" /out build\LiLiUSBCreator-2.10alpha$ver.exe /nopack /icon tools\img\lili.ico /comp 4 /x64
           Start-Sleep -s 10
           Move-Item tools build
    - name: Upload
      uses: actions/upload-artifact@v2
      with:
       name: LinuxLiveUSBCreator
       path: C:\Program Files (x86)\AutoIt3\Aut2Exe\build
       if-no-files-found: error

TIA!

Edited July 3, 2021 by rcmaehl

[TheDcoder]

Cool, GitHub now has Windows VMs for CI?

[rcmaehl]

1 minute ago, TheDcoder said:

Cool, GitHub now has Windows VMs for CI?

Yep

Edited April 27, 2021 by rcmaehl

[TheDcoder]

Very nice. Sorry for going a bit off-topic, another interesting thing is that LiveLinuxUSB is the reason why I discovered AutoIt, I have no idea where I would be now if it weren't for it

Thanks for forking the project and improving it

As for your actual question, tagging @Jos so that he can advice you on AutoIt3Wrapper.

[rcmaehl]

Just now, TheDcoder said:

Very nice. Sorry for going a bit off-topic, another interesting thing is that LiveLinuxUSB is the reason why I discovered AutoIt, I have no idea where I would be now if it weren't for it

Thanks for forking the project and improving it

As for your actual question, tagging @Jos so that he can advice you on AutoIt3Wrapper.

Yeah I remember using LiLi wayyyyy back in the day because it was really easy persistent storage. Sad to see the original dev got busy, but it's understandable. Regardless, some of the original translators saw the thread on r/Linux so hopefully I'll have a 2.10 with updated translations out within a month or so. The original dev's coding style is quite a bit different than how I would code and I'm still getting used to it.

[TheDcoder]

23 minutes ago, rcmaehl said:

Yeah I remember using LiLi wayyyyy back in the day because it was really easy persistent storage.

That's cool, I never used that feature personally... I was just trying out various Linux distros.

1 hour ago, rcmaehl said:

The original dev's coding style is quite a bit different than how I would code and I'm still getting used to it.

Yeah, he also doesn't use the usual Hungarian notation to denote the type of the variable

[Jos]

4 hours ago, rcmaehl said:

I can't seem to get #AutoIt3Wrapper headers to be parsed. Unlike AU3Check and Aut2exe, using the same parameters SciTE shows being used does not seem to work.

Not sure I understand the question as I only see the input (CI)  which I guess I can interpret, but have no idea what you mean with "headers to be parsed". 
Could you explain please and maybe add the log of "what you do see" ?

... and arn't you missing the download for AutoIt3Wrapper from that Script?  (Maybe you could start explaining what that CI exactly should accomplish for this noob when it comes to Github CI's) 

Jos 

Edited April 27, 2021 by Jos

[rcmaehl]

5 hours ago, Jos said:

... and arn't you missing the download for AutoIt3Wrapper from that Script?  (Maybe you could start explaining what that CI exactly should accomplish for this noob when it comes to Github CI's) 

Jos 

I'm not seeing in under https://www.autoitscript.com/site/autoit/downloads/ or https://www.autoitscript.com/site/autoit-script-editor/downloads/ so I thought it came by default. I must have installed it a while ago on my own system without realizing. Let me see if I can add that

Edited April 27, 2021 by rcmaehl

[TheDcoder]

5 hours ago, Jos said:

(Maybe you could start explaining what that CI exactly should accomplish for this noob when it comes to Github CI's) 

It's just basically a task which is done for each commit/revision to the code, people use it to run syntax checks, build errors etc. These checks are ran in a VM in the cloud, GitHub provides this feature for free (for public repositories).

10 minutes ago, rcmaehl said:

I'm not seeing

It's included in the full version of the SciTE4AutoIt3 editor I think

[rcmaehl]

I was able to get it to work. I didn't realize running the wrapper actually compiled the script instead of having to manually run Aut2Exe. I'll post my new CI yml in case anyone else ever experiences this issue (Yes, the sleeps ARE needed).

name: lili

on: 
 push:
   paths-ignore:
     - '.github/**'
     - '!.github/workflows/**'
     - '*.md'

jobs:
  build:

    runs-on: windows-latest
    steps:
    - name: Install Autoit
      run: |
           Invoke-WebRequest -Uri https://www.autoitscript.com/cgi-bin/getfile.pl?autoit3/autoit-v3-setup.exe -OutFile autoit-v3-setup.exe
           ./autoit-v3-setup.exe /S
           Start-Sleep -s 10
    - name: Checkout repo
      run: |
           cd C:\"Program Files (x86)"\AutoIt3\Aut2Exe
           git clone https://github.com/rcmaehl/LinuxLiveUSBCreator
    - name: Install Full Editor tools
      run: |
           Invoke-WebRequest -Uri https://www.autoitscript.com/cgi-bin/getfile.pl?../autoit3/scite/download/SciTE4AutoIt3.exe -OutFile SciTE4AutoIt3.exe
           ./SciTE4AutoIt3.exe /S
           Start-Sleep -s 10 
    - name: Wrap Code
      run: |
           cd C:\"Program Files (x86)"\AutoIt3\Aut2Exe\LinuxLiveUSBCreator\sources\LiLi\
           ../../../../AutoIt3.exe "C:\Program Files (x86)\AutoIt3\SciTE\AutoIt3Wrapper\AutoIt3Wrapper.au3" /NoStatus /prod /in "C:\Program Files (x86)\AutoIt3\Aut2Exe\LinuxLiveUSBCreator\sources\LiLi\LiLi USB Creator.au3"
           Start-Sleep -s 10
    - name: Rename and Aggregate Files
      run: |
           cd C:\"Program Files (x86)"\AutoIt3\Aut2Exe
           mkdir build
           cd C:\"Program Files (x86)"\AutoIt3\Aut2Exe\LinuxLiveUSBCreator
           $ver= git rev-list --count HEAD
           cd C:\"Program Files (x86)"\AutoIt3\Aut2Exe\LinuxLiveUSBCreator\Sources\LiLi
           Move-Item "LiLi USB Creator.exe" C:\"Program Files (x86)"\AutoIt3\Aut2Exe\build
           cd C:\"Program Files (x86)"\AutoIt3\Aut2Exe
           Move-Item LinuxLiveUSBCreator\tools build
           cd build
           Rename-Item -Path "LiLi USB Creator.exe" -NewName LiLiUSBCreator-2.10alpha$ver.exe
    - name: Upload
      uses: actions/upload-artifact@v2
      with:
       name: LinuxLiveUSBCreator
       path: C:\Program Files (x86)\AutoIt3\Aut2Exe\build
       if-no-files-found: error

 

Edited April 27, 2021 by rcmaehl

[micwoj92]

Hello, I'm the guy who made initial CI and I think using chocolatey to install autoit3 could be better, would look much cleaner. Has anyone here worked with autoit3 installing from chocolatey?

I don't use Windows so every change that was needed to do was (almost) a blind guess and I'm honestly surprised that it works (more or less). Rcmaehl has been a big help with his Windows expertise (and it's his project after all).

[mLipok]

@rcmaehl 
Please consider to change Title
from:

Quote

Setting up CI for autoit scripts?

to:

Quote

Setting up CI on GitHub for autoit scripts?

 

And add tag  GitHub to yours OP

Edited July 3, 2021 by mLipok

[micwoj92]

Also, the CI has been improved dramatically since WNW11, so it might be worth posting updated version here.

Edited July 3, 2021 by micwoj92

[rcmaehl]

As requested, the current, efficient CI setup for AutoIt is as such:

expandcollapse popup

name: CompileScript

on: [push, pull_request, workflow_dispatch]

jobs:
  build:
    runs-on: windows-latest
    steps:
      - uses: actions/checkout@v2
      - name: Cache tools
        uses: actions/cache@v2
        id: cache
        with:
          path: |
            autoit-v3-setup.exe
            SciTE4AutoIt3.exe
            C:\Program Files (x86)\AutoIt3\SciTE\Au3Stripper
          key: v2
      - name: Download tools
        if: steps.cache.outputs.cache-hit != 'true'
        run: |
          curl -sSfLO https://www.autoitscript.com/cgi-bin/getfile.pl?autoit3/autoit-v3-setup.exe `
            -sSfLO https://www.autoitscript.com/cgi-bin/getfile.pl?../autoit3/scite/download/SciTE4AutoIt3.exe `
            -sSfLO https://www.autoitscript.com/autoit3/scite/download/Au3Stripper.zip
          Expand-Archive Au3Stripper.zip "${env:ProgramFiles(x86)}\AutoIt3\SciTE\Au3Stripper"
      - name: Install tools
        run: |
          Start-Process autoit-v3-setup.exe -ArgumentList /S -NoNewWindow -Wait
          Start-Process SciTE4AutoIt3.exe -ArgumentList /S -NoNewWindow -Wait
      - name: Submodule UDF update
        run: |
          git submodule init
          git submodule update --remote --merge
      - name: Compile Script
        run: |
          Start-Process "${env:ProgramFiles(x86)}\AutoIt3\AutoIt3.exe" "`"${env:ProgramFiles(x86)}\AutoIt3\SciTE\AutoIt3Wrapper\AutoIt3Wrapper.au3`" /NoStatus /prod /in SCRIPT_NAME_CHANGE_ME.au3" -NoNewWindow -Wait
          sha256sum -b SCRIPT_NAME_CHANGE_ME*.exe > checksums.sha256
      - uses: actions/upload-artifact@v2
        with:
          name: Compiled_Script
          path: |
            SCRIPT_NAME_CHANGE_ME*.exe
            checksums.sha256
          if-no-files-found: error
      - name: Zip package
        if: startsWith(github.ref, 'refs/tags/')
        run: 7z a SCRIPT_NAME_CHANGE_ME.zip SCRIPT_NAME_CHANGE_ME*.exe checksums.sha256
      - name: Release
        uses: softprops/action-gh-release@v1
        if: startsWith(github.ref, 'refs/tags/')
        with:
          files: |
            SCRIPT_NAME_CHANGE_ME*.exe
            SCRIPT_NAME_CHANGE_ME.zip
            checksums.sha256
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

 
It only needs slight modification for your au3 input, output exe, and output zip file names

Edited October 18, 2021 by rcmaehl

[mLipok]

Today I ran into a problem with TIDY implementation in your CI workflows exactly here:
https://github.com/rcmaehl/WhyNotWin11/pull/411

I was trying to use:

#AutoIt3Wrapper_Run_Tidy=Y
#Tidy_Parameters=/reel /portable /showconsoleinfo=9 /proper=1

@rcmaehl could you explain how this CI works on GitHub ?

Question 1:
Do this CI workflows each time download:

Quote

          curl -sSfLO https://www.autoitscript.com/cgi-bin/getfile.pl?autoit3/autoit-v3-setup.exe `
            -sSfLO https://www.autoitscript.com/cgi-bin/getfile.pl?../autoit3/scite/download/SciTE4AutoIt3.exe `
            -sSfLO https://www.autoitscript.com/autoit3/scite/download/Au3Stripper.zip

?

Question 2:
How they are unpaced/installed ?

Question 3:
Could you try to use:
https://www.autoitscript.com/cgi-bin/getfile.pl?../autoit3/scite/download/SciTE4AutoIt3_Portable.zip

instead:
https://www.autoitscript.com/cgi-bin/getfile.pl?../autoit3/scite/download/SciTE4AutoIt3.exe

?

Edited July 5, 2021 by mLipok

[micwoj92]

@mLipok

Quote

Question 1:
Do this CI workflows each time download:

No, there is this function before in CI which caches the files. If you want to download again, you'd have to change the key. for example v3 or something. Then I think that you can also go back to old caches when you change number to old one.

- name: Cache tools
        uses: actions/cache@v2
        id: cache
        with:
          path: |
            autoit-v3-setup.exe
            SciTE4AutoIt3.exe
            C:\Program Files (x86)\AutoIt3\SciTE\Au3Stripper
          key: v2

Quote

Question 2:
How they are unpaced/installed ?

This is done in this step, just a standard install as you would do in command line on Windows.

- name: Install tools
        run: |
          Start-Process autoit-v3-setup.exe -ArgumentList /S -NoNewWindow -Wait
          Start-Process SciTE4AutoIt3.exe -ArgumentList /S -NoNewWindow -Wait

Au3stripper is unpacked in previous step

`Expand-Archive Au3Stripper.zip "${env:ProgramFiles(x86)}\AutoIt3\SciTE\Au3Stripper"`

 

Quote

Question 3:
Could you try to use:

I mean, it can be done, but what would be the benefit of using it instead of the installer?

 

 

I will post this code with comments, what's necessary and not etc. here soon, so will be easier to understand.

[mLipok]

4 minutes ago, micwoj92 said:

I will post this code with comments, what's necessary and not etc. here soon, so will be easier to understand.

Thanks.

[micwoj92]

expandcollapse popup

name: wnw11 # just a name, not really important

on: [push, pull_request, workflow_dispatch] # runs workflow on: commit push, pull request and last options allows to trigger it manually

jobs:
  build:
    runs-on: windows-latest # specify windows version, windows-2019 and 2016 are available at the time of writing, latest uses latest available
    steps:
      - uses: actions/checkout@v2 # check outs repository under $GITHUB_WORKSPACE
      - name: Cache tools
        uses: actions/cache@v2 # cache paths specified below so they don't have to be redownloaded each time
        id: cache
        with:
          path: |
            autoit-v3-setup.exe
            SciTE4AutoIt3.exe
            C:\Program Files (x86)\AutoIt3\SciTE\Au3Stripper
          key: v2 # cache key, simple change (for example to v3) to redownload, can also be used to roll back to older versions of downloaded programs
      - name: Download tools
        if: steps.cache.outputs.cache-hit != 'true' # downloads tools if they are not in cache
        run: |
          curl -sSfLO https://www.autoitscript.com/cgi-bin/getfile.pl?autoit3/autoit-v3-setup.exe `
            -sSfLO https://www.autoitscript.com/cgi-bin/getfile.pl?../autoit3/scite/download/SciTE4AutoIt3.exe `
            -sSfLO https://www.autoitscript.com/autoit3/scite/download/Au3Stripper.zip
          Expand-Archive Au3Stripper.zip "${env:ProgramFiles(x86)}\AutoIt3\SciTE\Au3Stripper"
      - name: Install tools # self explanatory, installs tools
        run: |
          Start-Process autoit-v3-setup.exe -ArgumentList /S -NoNewWindow -Wait
          Start-Process SciTE4AutoIt3.exe -ArgumentList /S -NoNewWindow -Wait
      - name: Compile # this just compiles, Start-Process is used because AutoIt returns to CMD before finishing. Definitely much better solution than start-sleep used before.
        run: |
          Start-Process "${env:ProgramFiles(x86)}\AutoIt3\AutoIt3.exe" "`"${env:ProgramFiles(x86)}\AutoIt3\SciTE\AutoIt3Wrapper\AutoIt3Wrapper.au3`" /NoStatus /prod /in WhyNotWin11.au3" -NoNewWindow -Wait
          sha256sum -b WhyNotWin*.exe > checksums.sha256 # output shasums of file so people are sure that the files are genuine 
      - uses: actions/upload-artifact@v2 # uploads artifacts so they can be accessed to download
        with:
          name: WNW11
          path: |
            WhyNotWin11*.exe
            checksums.sha256
          if-no-files-found: error # error if no files found, default is warn, error also sends you email by default
      - name: Zip package # this step creates one zip with all files
        if: startsWith(github.ref, 'refs/tags/') # runs setp only on release (github tag)
        run: 7z a WhyNotWin11.zip WhyNotWin11*.exe checksums.sha256 # this uses 7z to make the zip
      - name: Release # uploads files specified below on release
        uses: softprops/action-gh-release@v1
        if: startsWith(github.ref, 'refs/tags/')
        with:
          files: |
            WhyNotWin11*.exe
            WhyNotWin11.zip
            checksums.sha256
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

GitHub runners are just windows server datacenter virtual machines

All the "uses:" things are just github repositories, for for example if you want to check out what "softprops/action-gh-release@v1" is and read more documentation on it , then you just go to "https://github.com/softprops/action-gh-release"

Bare minimum for this to work is the download + install tools step and compile step, it will simply test if your code compiles

Official GitHub actions documentations is available at https://docs.github.com/en/actions

Edited August 2, 2021 by micwoj92
highlighting

[rcmaehl]

Hi all,

I've recently started moving UDFs I use between multiple projects into their own Github repo. Since I'm lazy and don't want to have to manually merge UDF changes across repos, I've added in basic submodule support to the CI script marked as the answer. 

You can find documentation on submodules here: https://github.blog/2016-02-01-working-with-submodules/

You can find an example submodule UDF here: https://github.com/rcmaehl/_GetSteam