CONCEPT: UDF Updater allowing to update the UDFs between releases of the AutoIt3 installer

[Jos]

23 hours ago, WildByDesign said:

would like to present (and discuss as a community) the following ideas:

• Repo with which to store updated UDF files that have been officially accepted (eg. code changes approved)
• Decoupling the UDF files from the AutoIt installer and having its own source/installer
• Updater with which to check for these updated UDF files

My 2 cents: I haven't searched, but am pretty sure this has been discussed before. The setup is the easy part, as far as I can tell, just create a github repo with the UDFs in there and maintain all UDF's in it. To use them would require a simple "git clone repo"  and update a simple "git pull".

The hard part is how to govern this repo and ensure changes/pull requests aren't simply merged but will be properly tested and reviewed. Wonder who has the time available for this effort, and not to forget "the balls" to say No to pull requests that aren't up-to-spec. 

[argumentum]

4 minutes ago, Jos said:

The hard part is how to govern this repo and ensure changes/pull requests aren't simply merged but will be properly tested and reviewed. Wonder who has the time available for this effort, and not to forget "the balls" to say No to pull requests that aren't up-to-spec. 

12 hours ago, argumentum said:

**We are all working independently without a central coordination. We take "something" and work on it when we have the time. All we'd have to do (MVPs), is to make a post in the MVP area and ask for status, and if the status is "am good" for everyone, then that is a good moment to distribute the \includes\ folder ( with a newly compiled help file ). But, that is my opinion and not anything that the MVPs are aware of because so far, am the only one on this thread, and there is no thread discussing what we are cooking here on this thread elsewhere.

Since the GitHub would require a single person to attend to it ( I guess, I don't have much experience with it ), packing the UDFs when they are not "in between" updates, and that is mainly you, JPM, mLipok, water, etc., making a post announcing a, semi-annual update or whenever there are changes, would be better than just clone to GitHub. The idea of using GitHub is good, but not as a clone, rather as "here is a zipped update", would avoid the control issues that arise from people screwing the UDFs.
This site is just as good to collaborate without going all GitHub. What do you think @Jos ?

[Jos]

12 minutes ago, argumentum said:

Since the GitHub would require a single person to attend to it ( I guess, I don't have much experience

That is clear so maybe don't respond  with those lengthy brainwave dumps and first "Think-Research-Think again" before sharing those thoughts! 😉 

I for one haven't read the whole thread and only rrad the post i responded to, as I am not the one that will be part of this effort. 

Github indeed has one owner for the repo but multiple people can be given the rights to approve pull requests and merge them into the repo.

 

[argumentum]

2 minutes ago, Jos said:

..., as I am not the one that will be part of this effort.

Ok, power to the people !  
, but, who is going to make the bridge between SVN and GitHub ( not you, ok ). But has to be an already MVP to access the SVN.
All in all, we are brainstorming a proposal for Jon to say yay or nay.
On the other hand, going all GitHub would open up for expanding the standard UDFs, but it would certainly be more hectic, ..and would need approval 🤔 ?

12 minutes ago, Jos said:

..maybe don't respond  with those lengthy brainwave dumps..

I guess I did it again. But is not to bug anyone. Is to bring ideas to the "think tank".  Since I can not state what we are going to do, how things are going to work from now on, and am afraid of unintended consequences, ...bringing a bunch of doubt and assumed limitations, clarifies things. Like your "we can GitHub it" was a surprise for me.

[Jos]

Why would a bridge be required? Just do your thing as user community and ignore what we do in the autoit3 svn. Keep life simple and maintain your own. 

I for one will root for you and wish you all a lot of wisdom in getting this workable. 

Edited 21 hours ago by Jos

[argumentum]

Ok, @WildByDesign, long time a go, there was a post called WinAPIEx UDF that was later integrated in the standard UDFs.
How about we do our community effort of an unofficial UDFs ( like "WinAPIEx UDF" was at one point ) but hosted in GitHub, and take it from there ?.

[WildByDesign]

58 minutes ago, argumentum said:

How about we do our community effort of an unofficial UDFs ( like "WinAPIEx UDF" was at one point ) but hosted in GitHub, and take it from there ?.

That is certainly a possibility. That opens up some more questions:

• Would we be still be using the UDF files from <AutoIt-install-dir>\AutoIt3\Include as a starting point?
• Would we be replacing/updating the official UDF files from that same directory?

         If not, we would have to rename the functions to avoid duplicates. So I assume that we would be overwriting/updating them.

• Would we have the ability to cherry-pick changes from SVN?

[SOLVE-SMART]

I don't want to promote GitHub even more, but at least a few things to understand (maybe as clarification):

"GitHub organizations" are concipated for exactly such actions - working in teams (communities).
In that organizations you can create (multiple) teams, but you don't have to. You can add "Members" to the org., but you also can add "Outside collaborators".
You can set permission on "Owner", "Member" and "Outside collaborators" levels with several additional custom roles, for the whole org. but also for seperate repositories.

This allows you to add multiple Members, even Owners to a single repository with read, write or extended permissions.
Usually quality gates (approval strategies) will be added to the repositories. This avoids force pushing code or wrong code via pull requests.
Two or three "reviewers" would have to check each pull request. Thousands of professional teams use GitHub with this approach and for exactly this purpose.

💡The funny thing is, Jon already uses two GitHub organizations for AutoIt. Not very active, at least for what is seen publicly. Maybe he uses several private repositories, maybe not. So it would be easy to do the start with his "official" org.

• https://github.com/AutoIt
• https://github.com/AutoItConsulting

I have experience creating and managing such GitHub orgs.

Example for the user added UDFs (not the official ones):
One of the reasons I haven't yet moved the UDFs from https://www.autoitscript.com/wiki/User_Defined_Functions to GitHub and keeping them up-to-date is the fact of legal issues. Not all scripts do have the some license or agreement etc. Once something is on GitHub, it should be subject to one or more licenses or at least to consents.

I would love to with such an approach with the suggested baby steps and the WinAPIEx UDF , but I would be know how to deal with the licenses and could we use the official GitHub orgs. by Jon? This would make things more easy.

Best regards
Sven

FYI: @argumentum, @Jos, @WildByDesign, @genius257

Edited 19 hours ago by SOLVE-SMART

[SOLVE-SMART]

27 minutes ago, WildByDesign said:

Would we have the ability to cherry-pick changes from SVN?

Is the SVN which was also mentioned by @Jos publicly available (readable)? Where?

If yes, then why a cherry-pick should be hard @WildByDesign 🤔 ?
Do you mean the the process to cherry-pick from SVN and add that commit to Git?
This is feasable, but I don't know where to find the sources and I doubt that we will get read permission to the SVN or how it is managed?

Fascinating. All of this makes me very curious 😁 .

Best regards
Sven

[argumentum]

29 minutes ago, SOLVE-SMART said:

Jon already uses two GitHub organizations for AutoIt. Not very active, at least for what is seen publicly.

..that's the problem. Jon is mostly unavailable. Jos is presenting an option that does not include him.
The whole issue is with Jon's lack of delegation, and if you follow the history of AutoIt's development, that lack of delegation is perfectly understandable.
The SVN is not publicly readable, hence my requesting of permission to even think of any changes. Mind you that am a new addition and haven't been here long enough to know exactly how to go about changing the way it is been all this time. But for me to "man up" and ask Jon for something it will have to be clear and concise and, not solely dependent on one forum user, because that user may die ( or something ). So it should be something that does not risk degradation due to so-and-so is not here. Something that can be automated as much as possible and the team that runs the GitHub not be able to screw with this site's repo ( SVN ). All this with the assumption that Jon does not have the time with the hectic pace of GitHub. On the other hand, he does not code UDFs, he's just for AutoIt3 binary, the UDFs is MVP area, so far.

I don't know what "picture of reality" will better serve the community, and that is something that Jon would have to thinker with ( along a drink and soft music I guess  ) 

And this is as far as I can contribute as far as ideas, I think.
Know nothing of GitHub and don't have the time to take it on my shoulders ( upon myself ) to run it and be the sole maintainer.

@Jon, have a read of this thread and whenever you find the time, share your 2 cents.

[SOLVE-SMART]

I really appriciate your statements and clarification @argumentum, thanks.

14 minutes ago, argumentum said:

Know nothing of GitHub and don't have the time to take it on my shoulders ( upon myself ) to run it and be the sole maintainer.

Me neither, but I would if I could.

My motivation regarding these topics is similar to yours: simply to prevent this wonderful language from dying, because it is dying more and more. I would love to see a more community-driven approach to ensure the language and the ecosystem around it, will increase and improve continuously.

You've been around much longer than I have. I hardly know the inner workings and processes at all, so thank you for the insight. It's difficult to offer the community a way to take work off their plate without transparency. But if that's not what's wanted, then that's just how it is. What a shame 😔 .

Best regards
Sven

[Jos]

1 hour ago, argumentum said:

The SVN is not publicly readable

Wrong assumption again (partly) ! 😉   

There are a couple of Dev's that have full access to the SVN like JPM&Myself, and there are a couple of members/MVPs/Mods that have access to the UDF part of the SVN library and can make the UDF modifications that get included in each release. 
You all have to figure out what it is you like to strive for like @WildByDesign mentioned. 
So define what it really is that you want without having the dependency on Jon or the core SVN library.

Looking at the posts so far I see a lot of questions and not too many ideas on "process" so curious to see what the result will be of this exercise. 🙂  

Edited 17 hours ago by Jos

[WildByDesign]

41 minutes ago, Jos said:

You all have to figure out what it is you like to strive for

The following is just my opinion. Realistically, it would be great if we could somehow have access to the UDF files whenever new changes are made. For example, one or more of the UDF files gets updated as per what the current process already is. Now, if that single SVN submission could then automatically trigger the creation of UDF-r123456.zip (based on revision number) and upload that archive somewhere on the server that is accessible to anyone.

Is that something that could be a possibility?

[Jos]

So the only wish is to have access to the latest version of all standard UDFs to allow you to load the UDF's between releases, nothing else?
UDF Helpfile updates maybe?
I was under the impression it was more than that, but that could be me not reading the whole thing properly. 

Edited 4 hours ago by Jos

[genius257]

Here is a suggestion for how this could be done, if GitHub is chosen:
 

• Create a new GitHub organization or use an existing if applicable.
• Setup global repository ruleset(s). At least one ruleset should require pull requests to merge to master, and at least one pull request review, to merge.
• Invite relevant users and setup permissions.
• Create a new repository for the organization, as a proof of concept.
• Create the license file.
• Decide on a versioning standard. I would suggest semver, but that is just my preference.
• Decide on how to document dependency version requirements, preferable in a format for a script to use.
• Optionally have a maintained changelog file. my personal prefence is the Keep a Changelog standard.
• Create a code of conduct. More information here, if needed.
• Create guidelines for contributors.
• Optionally there are issue and pr templates in GitHub, if relevant.
• I would also suggest setting up GitHub workflows for automated tests, like linting, maybe testing functionality. This will also help in pull requests, before any reviewer needs to be involved.

After the first release (which could simply be the current available code), a update script can be made.

[Jos]

2 minutes ago, genius257 said:

Here is a suggestion for how this could be done, if GitHub is chosen:

Great list of stuff to do, but I guess the "What" question needs to be answered before the "How". 🙂  

[SOLVE-SMART]

34 minutes ago, Jos said:

Great list of stuff to do, but I guess the "What" question needs to be answered before the "How". 🙂  

I agree @Jos.

Your list @genius257 is really good - thanks a lot - and matches almost all of my expectations and experiences in dev teams, but the scope should be clarified before. I still do not understand the "what" fully, because I have to think about it more and the current "what" is constantly updating 😅 .

42 minutes ago, Jos said:

I was under the impression it was more than that, but that could be me not reading the whole thing properly. 

I can not answer this yet, but I try like others surely do too.

Best regards
Sven

[jpm]

I understand you want an update of the UDFS part in Sync with SVN

So I will work on a autoitudfs-v3.3.19.0-alpha1.zip delivery

which can be installed for Include and Examples delivery also the corresponding Autoit.chm

in installation AutoIt3 dir you want

I will upload it in an Autoit\Aplha dir similar to the Beta one

Like that I will be the @Jon deliverer  of the alpha update in sync with SVN

[WildByDesign]

I just wanted to point out a foolish mistake on my part pertaining to the thread title "...& Decoupling UDF Files from AutoIt Installer".

Clearly, the UDF files need to be shipped with the installer. I'm not sure what I was thinking when I wrote that because that would indicate removing the UDF files from the installer which would be a problem.

Can the thread title be edited without causing any issues for the thread?

1 hour ago, Jos said:

So the only wish is to have access to the latest version of all standard UDFs to allow you to load the UDF's between releases, nothing else?
UDF Helpfile updates maybe?
I was under the impression it was more than that, but that could be me not reading the whole thing properly. 

This is essentially it, yes. And great point about the UDF Helpfile updates, I hadn't thought about that.

Now, the community can build some sort of updater from this and also ensure safeguards in place to notify about any script breaking changes (if any) and whatever else comes along.

Also, since it is all about the community, I will need to see, listen to, and respect the input of anyone else who wants to share their views on this.

[WildByDesign]

@genius257 and @SOLVE-SMART, you both have a good understanding of GitHub organizations and most importantly the access permissions that go along with that. I am pretty familiar with GitHub but I have no experience with the organizations aspect.

Since Jon likely doesn't have a lot of time to manage who has access to AutoIt's GH organization, we may have to make one. Although we haven't heard from Jon yet so maybe it is a possibility.

Licence type would still be a big question, as was already mentioned. Just like a lot of code snippets throughout the forum, there quite often isn't a specific licence tied to it. So I'm not sure about that.