NEW Test SYSTEM

[Xenobiologist]

Hi,

recently you created "two" new systems to do the bug-tracking and feature-tracking.

Maybe you can also create a TEST system.

What is it supposed to do?

I think about a testing-system like ... (Okay, no advertising) where you define test-sets.

The number of testsets grows with evolution and delevopment of Autoit. After having released a new version of Autoit, the users can sign up and do the tests and confirm that everything is fine.

This way, we get even higher quality and the devs needn't to test everything on their own.

Tests like:

• Install on Win98, 2000, XP, Vista
• Using func...
• ...

So long,

Mega

[Valik]

Well, we already have a rather extensive test suite that can be run unattended. I sometimes run it on a second machine after I've made big changes to make sure I didn't break anything. JP is pretty fanatical about ensuring we don't (accidentally) break AutoIt to the point his tests start failing. Earlier today I was pondering the idea of opening those tests up and letting the public add new tests and to make the existing tests better. I didn't think much on it, however. I think it would be a lot of work and the last thing any of us want is more work.

[Xenobiologist]

Well, we already have a rather extensive test suite that can be run unattended. I sometimes run it on a second machine after I've made big changes to make sure I didn't break anything. JP is pretty fanatical about ensuring we don't (accidentally) break AutoIt to the point his tests start failing. Earlier today I was pondering the idea of opening those tests up and letting the public add new tests and to make the existing tests better. I didn't think much on it, however. I think it would be a lot of work and the last thing any of us want is more work.

Hi,

yeah the initial effort to get it started maybe a few hours, but then you'll get a huge amount of test script which can also be used as example scripts for all the funcs in the helpfile. The helpfile will not be bloated and it is also a kind of tutorial for the users who search for further examples scripts than the ones provided with the helpfile. If something doesn't work, users can refer to test set XYZ, func ABC test Nr. X and everything is fine.

Anyway, it was just an idea.

So long,

Mega

Edited January 16, 2008 by Xenobiologist

[Valik]

We already have a huge amount of test scripts and the effort would take days. I'm pretty critical of the tests. I like them in theory and there are some good things about the way they are written, but I find them almost totally useless because they are so disorganized. It would take days and days to clean them up. For me, at least, trying to get the tests to run on a machine is an exercise in frustration. I bitch - a lot - when I try to get the tests working again after a lapse in running them.

[Xenobiologist]

We already have a huge amount of test scripts and the effort would take days. I'm pretty critical of the tests. I like them in theory and there are some good things about the way they are written, but I find them almost totally useless because they are so disorganized. It would take days and days to clean them up. For me, at least, trying to get the tests to run on a machine is an exercise in frustration. I bitch - a lot - when I try to get the tests working again after a lapse in running them.

Hi,

okay I understand. When creating a TEST System is to much work, maybe Jon can create a script collection page for examples.

I'd like the idea, cause then there a much more examples for all the specific functions in the helpfile. The helpfile won't get bigger and if people are interested in a special func the can go search for examples on the web (maybe WIKI).

Examples for e.g. all the _WinAPI_ functions.

So long,

Mega

[Valik]

Test scripts do not learning scripts make. I would not use the techniques used in a test script in a "normal" script.

[Xenobiologist]

Hi,

yes. Please see that as a different point. I mentioned that a TEST System will maybe help to get the community into the testing and so "developing" process.

+

• Transparency of functions and testing process for the users and devs
• Time saving for the devs, cause users can do XXXX tests parallel
• Higher quality cause errors can be found quicker
• Many new example scripts

-

• High initial effort
• additional effort is needed to keep it going

But the other point I'd like to mention was a kind of wiki with links for every func of the helpfile, where users can add example code. But this snippets are suppossed to be no real scripts. Just showing the use of the funcs in multiple situations and ways.

Currently, there are a lot of funcs in the helpfile without any piece of example. Maybe this can be a new addition to the helpfile (but not included in the chm - it is online or a different file). Also in the helpfile not every parameter of every func is used in the example code. This can also be explained by the community in wiki for the community.

(It is kind of what we do in the German forum. We got a link for every func of the helpfile and then we are going to translate the func into German for a German helpfile.)

And in the wiki we won't translate, we will add commented examples to show how to use the funcs.

So, there two points.

1. TEST SYSTEM - not wanted cause it is much work

2. Online Examples SYSTEM - ???

So long,

Mega

Edited January 16, 2008 by Xenobiologist

[James]

I think this is a clever idea. If users add scripts, why is that hard?

[Valik]

This goes back to something I've said a few times in the past. Less talk, more do. There's a Wiki already. If you feel it's a good idea, start working on it and try to get others to work rather than talk about how it would be a good idea. Everybody can sit around and nod their heads and talk about how great the idea sounds, but that does nothing. The wiki is there. Make use of it.

[Xenobiologist]

Hi,

okay.

You said - Less talk, more do.

My speech is: Let the community do ...(with the guidance of the devs and mods)

So long,

Mega