BETA: SciTE v5x & lua Dynamic_include and "Smart" AutoComplete for Vars/UDFs/Abbrevs

[Jos]

Please try the latest version of  SciTE_changes_Dynamic_Includes_LUA.zip:

• Fix double BS needed for CRLF 
• Fix avoid dropdown on BS in Comments and Literal strings.
• Delete Selected text on BS 

 

[donnyh13]

Works great Jos! Thank you very much for your hard work.

Edited March 13 by donnyh13

[donnyh13]

I was wondering one thing. If I freshly open scite, open a larger script, (7,000 lines), and press enter, the CRLF is instantly "written". However as soon  as I have pressed save, when I press enter, the insertion of a CRLF is more delayed. To make it instant again I have to reopen Scite, reopening the script alone wont change it. What I'm wondering is if there is something "activated" once save is pressed that would cause this? This isn't noticeable in small scripts.

Edited March 13 by donnyh13

[mLipok]

@Jos I start working with @donnyh13 UDF 

I have turned on some debuging stuff.

What I notice that when I type function names then for each key which I press the function lua do the their stuff int the background.

So I'm starting to wonder if the LUA function might not work until I write and take a break, e.g. 1 sec.
This would prevent LUA functions from being redundantly called multiple times in the background.

 

[Jos]

  On 9/24/2023 at 5:34 PM, donnyh13 said:

However as soon  as I have pressed save, when I press enter, the insertion of a CRLF is more delayed.

Expand  

I'll have a check to what the reason could be.

  On 9/24/2023 at 9:04 PM, mLipok said:

So I'm starting to wonder if the LUA function might not work until I write and take a break, e.g. 1 sec.

Expand  

So you want the drop-down not to update while hitting keys untill there is no key hit for 1sec? The issue is that the lua scripts are fired on each event and when you don't type, there is no event.

[mLipok]

  On 9/25/2023 at 8:34 AM, Jos said:

So you want the drop-down not to update while hitting keys untill there is no key hit for 1sec?

Expand  

At this moment yes, but of course the 1 sec delay should be easy to change for example to 0.5 sec or even 0.2 sec this need to be tested which value will fit best to user typing speed or other requirements.

 

  On 9/25/2023 at 8:34 AM, Jos said:

The issue is that the lua scripts are fired on each event and when you don't type, there is no event.

Expand  

exactly.

[Jos]

by the way: why does it take so much longer for you than on my laptop?:

> time: 0.04 secs Dropdown for $wm_a with 10 item(s) out of 13647

Still working on a USB drive?
Do you have Local Cache active?

[mLipok]

  On 9/25/2023 at 10:05 AM, Jos said:

Still working on a USB drive?

Expand  

yes.
I just did test at work I have new Dell with CPU i5-12400 
on Z drive (USB) and my C drive (NVME) 
On both C and Z drive I have 0.15 sec

Previous test I did on my laptop with i5-8XXX on the same Z drive

 

  On 9/25/2023 at 10:05 AM, Jos said:

Do you have Local Cache active?

Expand  

Not sure how to check ?

 

to be sure we had the same testing route: I use LibreOfficeWriter.au3 which contains 45841 lines of code.

 

[Jos]

  On 9/25/2023 at 10:21 AM, mLipok said:

to be sure we had the same testing route: I use LibreOfficeWriter.au3 which contains 45841 lines of code.

Expand  

mmm  ...   maybe that should be split into logical (udf) files as 46000 lines sounds quit big ! 

  On 9/25/2023 at 10:21 AM, mLipok said:

Not sure how to check ?

Expand  

  Quote

Added support for :

====================

#SciTE4AutoIt3_Dynamic_Include=y                         ;dynamic.include=y/n

#SciTE4AutoIt3_Dynamic_Include_whiletyping=y             ;dynamic.include.whiletyping=y/n

#SciTE4AutoIt3_Dynamic_Include_recursive_check=n         ;dynamic.include.recursive.check=n/y

#SciTE4AutoIt3_Dynamic_Include_use_local_cache=n         ;dynamic.include.use.local.cache=n/y

#SciTE4AutoIt3_Dynamic_Include_version=prod              ;dynamic.include.version=prod/beta

#SciTE4AutoIt3_Dynamic_Include_verboselevel=0            ;dynamic.include.verbose.level=0/1/2

#SciTE4AutoIt3_Dynamic_Include_Path=

 

#SciTE4AutoIt3_AutoItDynamicIncludes_debug=n             ;debug.autoitdynamicincludes=n/y

#SciTE4AutoIt3_AutoItAutocomplete_debug=n                ;debug.autoitautocomplet=n/y

#SciTE4AutoIt3_AutoItGotoDefinition_debug=n              ;debug.autoitgotodefinition=n/y

#SciTE4AutoIt3_AutoItTools_debug=n                       ;debug.autoIttools=n/y

#SciTE4AutoIt3_AutoItIndentFix_debug=n                   ;debug.autoitindentfix=n/y

]]

Expand  

 

[mLipok]

  On 9/25/2023 at 10:34 AM, Jos said:

#SciTE4AutoIt3_Dynamic_Include_use_local_cache=n         ;dynamic.include.use.local.cache=n/y

Expand  

 

in my SciTEUser.properties I have set it all the time

  Quote

dynamic.include.use.local.cache=y

Expand  

 

[mLipok]

I just notice that this happend only when I work with UDF FILE:  LibreOfficeWriter.au3

  Quote

> time: 0.16 secs Dropdown for _lowriter_cellb with 5 item(s) out of 5976
> time: 0.16 secs Dropdown for _lowriter_cellba with 1 item(s) out of 5976
> time: 0.15 secs Dropdown for _lowriter_cellbak with 0 item(s) out of 5976
> time: 0.15 secs Dropdown for _lowriter_cellba with 1 item(s) out of 5976
> time: 0.16 secs Dropdown for _lowriter_cellbac with 1 item(s) out of 5976
> time: 0.16 secs Dropdown for _lowriter_cellback with 1 item(s) out of 5976
 

Expand  

But using the same feature/testing route but with EXAMPLE FILE: _LOWriter_DocReplaceAll.au3

I get results

  Quote

> time: 0.02 secs Dropdown for _lowriter_cell with 14 item(s) out of 5896
> time: 0.02 secs Dropdown for _lowriter_cellb with 5 item(s) out of 5896
> time: 0.02 secs Dropdown for _lowriter_cellba with 1 item(s) out of 5896
> time: 0.02 secs Dropdown for _lowriter_cellbac with 1 item(s) out of 5896
> time: 0.02 secs Dropdown for _lowriter_cellback with 1 item(s) out of 5896
> time: 0.02 secs Dropdown for _lowriter_cellbackc with 1 item(s) out of 5896
> time: 0.02 secs Dropdown for _lowriter_cellbackco with 1 item(s) out of 5896
> time: 0.02 secs Dropdown for _lowriter_cellbackcol with 1 item(s) out of 5896
> time: 0.02 secs Dropdown for _lowriter_cellbackcolo with 1 item(s) out of 5896
> time: 0.02 secs Dropdown for _lowriter_cellbackcolor with 1 item(s) out of 5896
 

Expand  

 

Edited September 25, 2023 by mLipok

[mLipok]

is that mean that in this particular case:

working with 46k lines UDF it is parsed on the fly,

and working with examples the cached values are taking for the UDF function names and UDF is not processed on the fly, thus such differences in the speed.

I mean that my question is:
Do any currently edited file is parsed on the fly and not from cache ?

[Jos]

Makes total sense!
What happens is that when you type a character, the LUA will load the previously found UDFs/Vars tables and then scan the current file for all existing UDFs/Vars.
That last bit is taking more time when the current file is a BIG ASS LARGER SCRIPT!  

This has to happen as changes could have been made during editing ....right?

Edited September 25, 2023 by Jos

[mLipok]

Can you add feature that also currently edited file will be parsed from CACHE by adding directive (for currently processed file) or settings in SciTEUser.properties (for entire SciTE)

I know that this will force user to save file before changes will be taken.

But I think this can be game changer for such big UDF's  or big any other "Main Script".

 

[mLipok]

  On 9/25/2023 at 11:29 AM, Jos said:

This has to happen as changes could have been made during editing ....right?

Expand  

In some sense yes,

My point is:

When making changes, you must save the changes to use them.
Only saved changes should be taken into account.
From this we can say that we can start using CACHE in this case as well.

 

I don't see a situation where someone creates 30 new functions and immediately uses them without saving.

If I have created 1 function and I use it, I have it in memory or on the screen.
If this function is in a separate UDF, I still have to save it to use them in main script.
If the function is a large main file, I believe that the place of the function is the UDF, not the main program file.

If it is a small program file, I still say that we basically have three cases (maybe more):

1. the file is small and I know what and where I just changed it, so I don't have to remember the name of the function.

2. the function is visible on the screen or not, but I can always jump to it and see it

3. after modifying the function, I should save the file to get the desired effect

[Jos]

Ok, i have played a bit with the update of the local file and made the following changes:

• Added the following option which will determine whether the current file is scanned of each character typed (current Default is not to scan but that could change back to the old behavior): 
  Directive: #SciTE4AutoIt3_Dynamic_Include_Always_Update_Local_File=n
  Property: dynamic.include.always.update.local.file=n/y
• Re-enabled the logfile message to the console when debugging is on/off
• uploaded also the au3.autoit3wrapper.api & autoit3wrapper.keywords.properties with the added directive. 

 

  On 9/24/2023 at 5:34 PM, donnyh13 said:

If I freshly open scite, open a larger script, (7,000 lines), and press enter, the CRLF is instantly "written". However as soon  as I have pressed save, when I press enter, the insertion of a CRLF is more delayed.

Expand  

I am not sure what happens here, but it is expected that there is a small delay when Enter is pressed as a check is done for that line to see if any #include needs to be added.
I have done some testing with your big script and didn't see any difference between the initial Enter and after a Save.

[mLipok]

tested with:

#SciTE4AutoIt3_Dynamic_Include_Always_Update_Local_File=n

and

#SciTE4AutoIt3_Dynamic_Include_Always_Update_Local_File=y

both:

> time: 0.16 secs Dropdown for _lowriter_cellbackcolo with 1 item(s) out of 5977
> time: 0.15 secs Dropdown for _lowriter_cellbackcolor with 1 item(s) out of 5977
> time: 0.15 secs Dropdown for _lowriter_cellbackcolo with 1 item(s) out of 5977
> time: 0.15 secs Dropdown for _lowriter_cellbackcol with 1 item(s) out of 5977
> time: 0.16 secs Dropdown for _lowriter_cellbackco with 1 item(s) out of 5977

 

[Jos]

 

 

Show me the version info please: This is my result with the default

SciTE4AutoIt3_AutoItDynamicIncludes_debug=y
#SciTE4AutoIt3_AutoItAutocomplete_debug=y
#SciTE4AutoIt3_Dynamic_Include_verboselevel=2
_LOWriter_CellBorderWidth(

 

+-> (v20230925 1630) Get_FileINFO time:0.29 lCache:false IncludeFiles:2   UDFs:544  ConstVars:167  Debugging:true  Source:C:\Users\josva\Downloads\LibreOfficeWriterUDF\LibreOfficeWriter.au3
><- PerfDynUDFs  time: 0.36 Debugging:--  save   C:\Users\josva\Downloads\LibreOfficeWriterUDF\LibreOfficeWriter.au3
> time: 0.04 secs Dropdown for _lowrite with 544 item(s) out of 6428
--> AddMissingIncludes wordcheck _LOWriter_CellBorderWidth time: 0.07 secs Debugging:true

Log of dropdown bit:

!17:06:17 ->AutoCompleteDropdownNeeded ->ShowKeywordDropDown ! ===== Start ShowKeywordDropDown to current script funcdropdown C:\Users\josva\Downloads\LibreOfficeWriterUDF\LibreOfficeWriter.au3
>17:06:17 ->AutoCompleteDropdownNeeded ->ShowKeywordDropDown > time: 0.00  >>1 funcdropdown Word:280-288 _LOWrite 0 0 0 FullWord= _LOWrite
 17:06:17 ->AutoCompleteDropdownNeeded ->ShowKeywordDropDown @ Start make list of possible UDFs and in file words for _LOWrite
>17:06:17 ->ShowKeywordDropDown ->getlocalstuffandshowlist > time: 0.02  >> start getlocalstuf: n
>17:06:17 ->ShowKeywordDropDown ->getlocalstuffandshowlist > time: 0.02  >> step 2 sort getlocalstuf..  6428
>17:06:17 ->ShowKeywordDropDown ->getlocalstuffandshowlist > time: 0.02  >> step 3 builddropdown getlocalstuf..
>>>>>>>>>> Console:> time: 0.04 secs Dropdown for _lowrite with 544 item(s) out of 6428

 

This when enabled:

#SciTE4AutoIt3_AutoItDynamicIncludes_debug=y
#SciTE4AutoIt3_AutoItAutocomplete_debug=y
#SciTE4AutoIt3_Dynamic_Include_verboselevel=2
#SciTE4AutoIt3_Dynamic_Include_Always_Update_Local_File=y

_LOWriter_CellBorderStyle(

 

> time: 0.20 secs Dropdown for _lowri with 544 item(s) out of 6428
> time: 0.20 secs Dropdown for _lowrit with 544 item(s) out of 6428
> time: 0.20 secs Dropdown for _lowrite with 544 item(s) out of 6428
> time: 0.20 secs Dropdown for _lowriter with 544 item(s) out of 6428
--> AddMissingIncludes wordcheck _LOWriter_CellBorderStyle time: 0.08 secs Debugging:true
--> AddMissingIncludes linecheck  time: 0.08 secs Debugging:true

Log of dropdown bit:

!17:08:11 ->AutoCompleteDropdownNeeded ->ShowKeywordDropDown ! ===== Start ShowKeywordDropDown to current script funcdropdown C:\Users\josva\Downloads\LibreOfficeWriterUDF\LibreOfficeWriter.au3
>17:08:11 ->AutoCompleteDropdownNeeded ->ShowKeywordDropDown > time: 0.00  >>1 funcdropdown Word:341-350 _LOWriter 0 0 0 FullWord= _LOWriter
 17:08:11 ->AutoCompleteDropdownNeeded ->ShowKeywordDropDown @ Start make list of possible UDFs and in file words for _LOWriter
>17:08:11 ->ShowKeywordDropDown ->getlocalstuffandshowlist > time: 0.02  >> start getlocalstuf: y
>17:08:11 ->ShowKeywordDropDown ->getlocalstuffandshowlist > time: 0.09  >> cleaned doc getlocalstuf..
>17:08:12 ->ShowKeywordDropDown ->getlocalstuffandshowlist > time: 0.18  >> step 2 sort getlocalstuf..  8873
>17:08:12 ->ShowKeywordDropDown ->getlocalstuffandshowlist > time: 0.18  >> step 3 builddropdown getlocalstuf..
>>>>>>>>>> Console:> time: 0.20 secs Dropdown for _lowriter with 544 item(s) out of 6428

 

Edited September 25, 2023 by Jos

[mLipok]

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7
#SciTE4AutoIt3_Dynamic_Include_Always_Update_Local_File=n
#SciTE4AutoIt3_AutoItDynamicIncludes_debug=y
#SciTE4AutoIt3_AutoItAutocomplete_debug=y
#SciTE4AutoIt3_Dynamic_Include_verboselevel=2
_LOWriter_CellBorderWidth

  Quote

+-> Get_Std_INFO time:1.33 GlobalConstFiles:69 GlobalConstVars:12460 STDUDFIncludeFiles:82  STDUDFs:3179  USRUDFIncludeFiles:0  USRUDFs:0  Debugging:true
+-> (v20230923 1416) Get_FileINFO time:2.47 lCache:true IncludeFiles:2   UDFs:544  ConstVars:167  Debugging:true  Source:Z:\!!!_SVN_AU3\UDF_Forum\Other_Members\Official_AutoIt_Forum\LibreOfficeWriter\LibreOfficeWriter.au3
><- PerfDynUDFs  time: 3.90 Debugging:-0-0  open   Z:\!!!_SVN_AU3\UDF_Forum\Other_Members\Official_AutoIt_Forum\LibreOfficeWriter\LibreOfficeWriter.au3
> time: 0.16 secs Dropdown for _lowriter_cellborderwidt with 1 item(s) out of 5977
> time: 0.16 secs Dropdown for _lowriter_cellborderwidth with 1 item(s) out of 5977
 

Expand  

 

[mLipok]

found issue I copied only properties and forget lua files.

now works fine

sorry for confusion