Scite backup copies

[pseakins]

I have modified my copy of AutoItTools.lua so that backup files go to a \Backup subfolder.

To avoid having a DOS box flash when copying the file, I read and write with io.open and file:write.

Unfortunately, the timestamp info of the copied file gets lost in this process.

I have done a lot of googling and the best I have come up with is a lua library "LuaFileSystem" (LFS) which may help me with this issue.

 

My question is whether lua libraries can be easily added to the Scite environment, if so, how?  I know it's a bad idea to do this and wish to avoid it.

 

Or, is there another method of file copy in lua which avoids os.execute?

[Jos]

Couldn't you use my current logic already in place and modify that?

Jos 

[pseakins]

15 minutes ago, Jos said:

Couldn't you use my current logic already in place and modify that?

I am modifying your code Jos and the current logic is very useful. However, It occurred to me that moving the existing file from the working folder to the backup folder would cause some kind of trigger to Scite which it may not like. I now see that you do get away with effectively removing the current file in your code so I will revise.

Also, I didn't think that os.rename was capable of moving a file between directories but apparently it can, so I will test that.

You may have helped me answer my question, I tend to overthink stuff sometimes.

[Jos]

In case you need to resort to performing cmd commands, without the blackbox popping up,  you could add this function and use that:

function perform_oscommand(cmd)
   local handle=io.popen("" .. cmd)
   local consoleoutput = handle:read('*all')
   handle:close()
   print(cmd..">"..consoleoutput)
   return consoleoutput
end

Jos

[Jos]

This is a version for the function that allows to configure the target backup directory in the properties file  e.g.:

backup.files=5

# default -> $(FilePath)\backup
backup.directory=d:\temp\backup
-or-
backup.directory=$(FileDir)\mybackups

When backup.directory isn't provided, it will create a backup directory in the sourcefile's directory and copy them there: 

expandcollapse popup

--------------------------------------------------------------------------------
-- OnBeforeSave(filename)
--
-- keep the number of backups as defined by backup.files = ?    by Jos van der Zande (JdeB)
--
-- AutoItTools.OnBeforeSave $(au3) 2 Alt+Shift+I Open Beta Include
--------------------------------------------------------------------------------
function AutoItTools:OnBeforeSave(filename)
    local sbck = tonumber(props['backup.files'])
    -- backup specified and bigger than 0 ?
    if sbck == nil or sbck == 0 then
        return false
    end
    -- get file directory
    fdir = filename:match(".+[/\\]")
    fname = filename:match(".+[/\\](.+)")
    -- set backup directory
    if props['backup.directory'] == "" then
        bdir = fdir .. "backup"   -- no fixed backupdirectory location defined
    else
        bdir = props['backup.directory']    -- use the backupdirectory location defined for backup.directory
    end
    -- set targetbackupfilename
    tfile = bdir.."\\"..fname
    -- ensure the backup directory exists
    perform_oscommand('mkdir "'..bdir..'"')
    --
    local nbck = 1
    local bfil1 = ""
    if sbck > 1 then
        bfil1 = ".1"
    end
    -- check first file backup file format without number and rename it to *.1.bak
    if sbck > 1 and io.open (tfile.. "." .. ".bak", "r") == nil and io.open (tfile.. "." .. ".1.bak", "r") == nil then
        os.rename (tfile .. ".bak", tfile .. ".1.bak")
    end
    -- rename all files to +1
    while (sbck > nbck ) do
        local fn1 = sbck-nbck
        local fn2 = sbck-nbck+1
        os.remove (tfile.. "." .. fn2 .. ".bak")
        os.rename (tfile .. "." .. fn1 .. ".bak", tfile .. "." .. fn2 .. ".bak")
        nbck = nbck + 1
    end
    -- rename original to .1.bak or .bak depending on number of backups
    os.remove (tfile.. bfil1 .. ".bak")
    os.rename (filename, tfile .. bfil1 .. ".bak")
    return false
end

I probably will update the standard to this version .

Jos

Edited June 28, 2020 by Jos

[pseakins]

On 6/28/2020 at 7:47 PM, Jos said:

In case you need to resort to performing cmd commands, without the blackbox popping up,  you could add this function and use that:

Thanks Jos. On an initial test on my Windows 7 system I still get a box popup using  "function perform_oscommand(cmd)". It seems no different from "os.execute". I'll do more tests later to confirm. My box is a pretty blue with yellow text, the CMD window is one of the first things I configure on a new system after keyboard repeat delay.

There is a contradiction above (a typo I think) with the backup.directory default arg for SciTEUer.properties.

Using backup.directory=$(FilePath)\Backup I get a value of  "C:\Users\user\Documents\AutoIt\Snippet.au3\Backup" which would not be of much use.

Using backup.directory=$(FileDir)\Backup I get a more useful returned value of "C:\Users\user\Documents\AutoIt\Backup". The correct arg should be $(FileDir).

I'll post more about my changes to AutoItTool.lua later. Thanks again for your help.

 

[pseakins]

FYI. Checked again. On my Windows 7, 64bit system, I still get a DOS box popup flash when I call either of these functions.

-- Neither of these work without flashing a DOS box
function perform_oscommand(cmd)
  local handle=io.popen("" .. cmd)
  local consoleoutput = handle:read('*all')
  handle:close()
  print(cmd..">"..consoleoutput)
  return consoleoutput
end

function exec_silent(command)
  local p = assert(io.popen(command))
  local result = p:read("*all")
  p:close()
  return result
end

 

[Jos]

You are correct...   I saw there was going to be a patch for SciTE to allow to suppress the window popping up, so hopefully we can do something after that has come out.

Jos

[pseakins]

Jos, pending Scite patch noted. Here is my version for backups to go to a backup directory or per properties file. I prefer that the backups maintain their original filetype extension (.au3) as they are easier to view and compare. There is code to prevent backups being incremented if the source file is not dirty. I have an additional function (OnSave) to ensure a copy of the current state of the edited file (with all changes) is also copied to the backup directory. There is always the possibility that due to stupidity I may accidently delete or alter the working copy. By having a redundant copy in the backup folder I can feel a little more secure.

I would have liked the backup files to integrate with the backups created by the Tidy function but the Tidy backups decrement while the Scite backups increment . Also, I believe Tidy only makes a backup if it actually alters the source file, but if the file is dirty it does first trigger a Scite backup which is good.  So for the moment the two schemes are incompatible. I note that running a script (Go) with F5 will also trigger a Scite backup if the file is dirty, also good.

This part goes near the top of the file

AutoItTools = EventClass:new(Common)

-- OnBeforeSave() signals to OnSave() via eMyes that the file has changed.
-- Needed here because editor.Modify flag has been reset when we reach OnSave()
local eMyes

New and modified functions, OnSave and OnBeforeSave

expandcollapse popup

--------------------------------------------------------------------------------
-- OnSave(filename)
-- After the backup has been written by OnBeforeSave() we make a copy of the current
-- state of the edited file (with all recent changes) in the backup subfolder.
--------------------------------------------------------------------------------
function AutoItTools:OnSave(filename)
  local sbck = tonumber(props['backup.files'])
  if sbck == nil or sbck == 0 or eMyes == false then
    return false
  end
  local path, fname, basename, ftype = string.match(filename, "(.-)(([^\\/]-)%.?([^%.\\/]*))$") -- (no dot in ftype)
  ftype = "." .. ftype
  local bakpath = path .. "Backup\\"
  if props['backup.directory'] ~= "" then
    bakpath = props['backup.directory'] .. "\\"
  end
  local newname = bakpath .. basename .. ftype
  os.execute("copy /Y " .. filename .. " " .. newname) -- We will get a DOS box flash here
end -- OnSave()

--------------------------------------------------------------------------------
-- OnBeforeSave(filename)
--
-- keep the number of backups as defined by backup.files = ?    by Jos van der Zande (JdeB)
-- Modified to place backup files in ..\Backup folder or per SciTEUser properties
--------------------------------------------------------------------------------
function AutoItTools:OnBeforeSave(filename)
  eMyes = false -- signal OnSave() that the file has not changed
  local path, fname, basename, ftype = string.match(filename, "(.-)(([^\\/]-)%.?([^%.\\/]*))$") -- (no dot in ftype)
  ftype = "." .. ftype

  -- The corresponding entry for SciTEUser.properties would be "backup.files=nnn". This can be assigned with Scite\Config
  local sbck = tonumber(props['backup.files'])
  -- backup specified and bigger than 0 ?
  if sbck == nil or sbck == 0 or not editor.Modify then
    return false
  end
  eMyes = true -- signal OnSave() that the file has changed

  local bakpath = path .. "Backup\\"
  if props['backup.directory'] ~= "" then
    bakpath = props['backup.directory'] .. "\\"
  end
  local newname = bakpath .. basename

  -- create backup subfolder if it doesn't exist
  local ok, err, code = os.rename(bakpath, bakpath) -- ok = nil, code = 2 and err = "No such file or directory"
  if ok == nil and err == "No such file or directory" then
    -- This will flash up a DOS box. But it only occurs once on the first save after moving to a new folder
    os.execute('mkdir "' .. bakpath .. '"') -- mkdir doesn't care if the specified path has a trailing slash or not
  end

  -- move the current file from the working folder to the backup folder
  os.rename (filename, newname .. ".0" .. ftype)

  -- rename all backup files to +1. delete the highest backup if it matches the limit set by Scite\Config
  local nbck = 1
  while (sbck > nbck - 1) do
    local fn = sbck - nbck
    os.remove (newname .. "." .. fn + 1 .. ftype)
    os.rename (newname .. "." .. fn .. ftype, newname .. "." .. fn + 1 .. ftype)
    nbck = nbck + 1
  end
  if sbck == 1 then
    os.remove (newname .. ".bak")
    os.rename (newname .. ".1" .. ftype, newname .. ".bak")
  end
  return false
end

 

[Jos]

I have added the patch to a test version of SciTE on my Laptop and the popup now indeed seems to never show anymore. I will release it to Beta when it all performs stable over the next couple of days. 
Also added the statement I gave you a while back to test whether the file is modified or else simply skip making a backup.

function AutoItTools:OnBeforeSave(filename)
    -- skip making the backup when file isn't changed
    if not editor.Modify  then
        return true
    end

I am not sure what the extra OnSave() code adds for you as OnBeforeSave() i always ran first?

 

On 7/4/2020 at 3:17 AM, pseakins said:

I would have liked the backup files to integrate with the backups created by the Tidy function but the Tidy backups decrement while the Scite backups increment .

This is an updated version that contains the extra check for File-changed and also fixes the file rename to a similar logic as Tidy. The file rename is made configurable and can now do similar names as Tidy, but default will be:  Filename_bakx.Fileext.
Set "backup.renameto=_old"  to make it similar to Tidy's backups and thus only saving the number defined by either one.

expandcollapse popup

--------------------------------------------------------------------------------
-- run a cmd command and capture the output
--------------------------------------------------------------------------------
function perform_oscommand(cmd)
   local handle=io.popen("" .. cmd)
   local consoleoutput = handle:read('*all')
   handle:close()
   return consoleoutput
end

--------------------------------------------------------------------------------
-- OnBeforeSave(filename)
--
-- keep the number of backups as defined by backup.files = ?    by Jos van der Zande (Jos)
--
-- AutoItTools.OnBeforeSave $(au3) 2 Alt+Shift+I Open Beta Include
--------------------------------------------------------------------------------
function AutoItTools:OnBeforeSave(filename)
    -- skip making the backup when file isn't changed
    if not editor.Modify  then
        return false
    end
    local sbck = tonumber(props['backup.files'])
    -- backup specified and bigger than 0 ?
    if sbck == nil or sbck == 0 then
        return false
    end
    -- get file directory
    fdir = filename:match(".+[/\\]")
    fname = filename:match(".+[/\\](.+)")
    fext = fname:match(".+(%..+)") or ""
    if fext ~= "" then
        fname = fname:match("(.+)%..+")
    end
    -- set backup directory
    if props['backup.directory'] == "" then
        bdir = fdir .. "backup"             -- no fixed backupdirectory location defined
    else
        bdir = props['backup.directory']    -- use the backupdirectory location defined for backup.directory
    end
    -- set backup rename to extra string. default   Filename_bakx.Fileext
    if props['backup.renameto'] == "" then
        bmid = "_bak"                      -- default
    else
        bmid = props['backup.renameto']    -- use the backupdirectory location defined for backup.directory
    end
    -- set targetbackupfilename
    tfile = bdir.."\\"..fname..fext
    -- ensure the backup directory exists
    perform_oscommand('mkdir "'..bdir..'"')
    --
    local nbck = sbck
    local bfil1 = ""
    if sbck > 1 then
        bfil1 = ".1"
    end
    -- rename all files by shifting them to +1 and making the new backupfile 1
    while (nbck > 1 ) do
        nbck = nbck - 1
        local fn1 = bdir.."\\".. fname..bmid..nbck..fext
        local fn2 = bdir.."\\".. fname..bmid..(nbck+1)..fext
        os.remove (fn2)
        os.rename (fn1,fn2)
    end
    -- rename original to backupname 1
    local fn1 = bdir.."\\"..fname..bmid.."1"..fext
    os.remove (fn1)
    os.rename (filename, fn1)
    return false
end

Jos

Edited August 15, 2020 by Jos

[pseakins]

I don't think the file rename is similar to Tidy. Scite backup, per the script, increments all the existing backups and adds the new backup as #1. So, if the limit were set to 10, backup #10 gets deleted as it is replaced by #9. All the others get shuffled up one.

Meanwhile, Tidy (19.1127.1402.0) backups work the opposite way. The new backup goes in as #10 and all the backups below it get decremented with #1 being deleted.

The two schemes couldn't work together like this if they shared the same Filename_bakx.Fileext format as they would kind of meet in the middle. It would be weird, with each one deleting the other one's latest backup.

Maybe Beta Tidy no longer works this way, have you changed it?

The extra OnSave() code ensures that there is is a backup copy of the latest source file changes placed in the backup folder. As it stands, without my code, the only files placed in the backup folder are those of the file state prior to making the edit changes and hitting ^B. There is no saved copy of the latest source code modification other than to the source code working folder.

At the moment you hit ^B the only changes are in the editor memory buffer. The disk image in the source folder holds the old pre-edit image. The backup code then copies the old folder image. At this point in time there is no access to the new changes in the editor buffer. After the OnBeforeSave() function executes, control is passed back to Scite which then flushes the dirty buffer to the source file folder. With your code there is no further action taken. I have added function OnSave which is then called by Scite after it has updated the source folder. We then need a copy of the newly updated source file. However, Scite has now cleared the editor.modify flag so we can't use that to know if there has been a change to the source file. So I added the eMyes flag so that OnBeforeSave could semaphore OnSave that that it should make the required backup copy.

[pseakins]

Here's a filelist showing the chronology of Scite and Tidy backups. The very latest Scite backup #1 is at 12:16 while the latest Tidy backup #10 is also around that time.

You can see that the Scite backup numbering increments while the Tidy numbering decrements.

 

06/07/2020  12:17 PM               248 TidyTest.au3
06/07/2020  12:16 PM               242 TidyTest.1.au3
06/07/2020  12:14 PM               236 TidyTest.2.au3
06/07/2020  12:07 PM               232 TidyTest.3.au3
06/07/2020  12:05 PM               234 TidyTest.4.au3
06/07/2020  12:04 PM               236 TidyTest.5.au3
06/07/2020  12:03 PM               226 TidyTest.6.au3
06/07/2020  12:03 PM               220 TidyTest.7.au3
06/07/2020  12:02 PM               211 TidyTest.8.au3
06/07/2020  12:02 PM               206 TidyTest.9.au3
06/07/2020  12:02 PM               197 TidyTest.10.au3
06/07/2020  12:02 PM               181 TidyTest_old1.au3
06/07/2020  12:02 PM               188 TidyTest_old2.au3
06/07/2020  12:02 PM               191 TidyTest_old3.au3
06/07/2020  12:02 PM               200 TidyTest_old4.au3
06/07/2020  12:02 PM               203 TidyTest_old5.au3
06/07/2020  12:03 PM               212 TidyTest_old6.au3
06/07/2020  12:03 PM               216 TidyTest_old7.au3
06/07/2020  12:04 PM               226 TidyTest_old8.au3
06/07/2020  12:05 PM               226 TidyTest_old9.au3
06/07/2020  12:17 PM               248 TidyTest_old10.au3

 

[Jos]

6 hours ago, pseakins said:

Maybe Beta Tidy no longer works this way, have you changed it?

No you are right ...   back to the drawing-board.  

6 hours ago, pseakins said:

I have added function OnSave which is then called by Scite after it has updated the source folder

Ok... understand now. It is simply a copy of the original source and not a previous version. 

I will have a look at the LUA script to make it function the same as Tidy does. 

Jos

 

[pseakins]

2 minutes ago, Jos said:

I will have a look at the LUA script to make it function the same as Tidy does. 

It's easy enough to reverse the chronology of the Scite backup, but to me, this is not a desirable option. The Scite chronology already works properly. It is the Tidy chronology that needs to be fixed. In my opinion newer backups should have the lower numbers. But that is just my opinion, yours may differ and I can live with either.

[Dan_555]

Hi, i'm jumping into this with a request. Can you add leading 0 before the numbers ?

So that the sorting order, (in my case i'm sorting the files by the extension, not by the date), be correct.

currently the backup filenames are displayed as:

Quote

file1.au3

file10.au3

file11.au3

...

file2.au3

file20.au3

file21.au3

...

file3.au3

file4.au3

file5.au3

file6.au3

file7.au3

file8.au3

file9.au3

So at least the leading 0 would fix the ordering.

Edited July 6, 2020 by Dan_555

[Jos]

This version should do the trick (I hope) .  

expandcollapse popup

--------------------------------------------------------------------------------
-- run a cmd command and capture the output
--------------------------------------------------------------------------------
function perform_oscommand(cmd)
   local handle=io.popen("" .. cmd)
   local consoleoutput = handle:read('*all')
   handle:close()
   return consoleoutput
end

--------------------------------------------------------------------------------
-- OnBeforeSave(filename)
--
-- keep the number of backups as defined by backup.files = ?    by Jos van der Zande (Jos)
--
-- AutoItTools.OnBeforeSave
--------------------------------------------------------------------------------
function AutoItTools:OnBeforeSave(filename)
    -- skip making the backup when file isn't changed
    if not editor.Modify  then
        return false
    end
    print("--- start backup ---")
    local backupfiles = tonumber(props['backup.files'])
    -- backup specified and bigger than 0 ?
    if backupfiles == nil or backupfiles == 0 then
        return false
    end
    -- get file directory
    fdir = filename:match(".+[/\\]")
    fname = filename:match(".+[/\\](.+)")
    fext = fname:match(".+(%..+)") or ""
    if fext ~= "" then
        fname = fname:match("(.+)%..+")
    end
    -- set backup directory
    if props['backup.directory'] == "" then
        bdir = fdir .. "backup"             -- no fixed backupdirectory location defined
    else
        bdir = props['backup.directory']    -- use the backupdirectory location defined for backup.directory
    end
    -- set backup rename to extra string. default   Filename_bakx.Fileext
    if props['backup.renameto'] == "" then
        bmid = "_bak"                      -- default
    else
        bmid = props['backup.renameto']    -- use the backupdirectory location defined for backup.directory
    end
    -- ensure the backup directory exists
    perform_oscommand('mkdir "'..bdir..'"')
    --
    -- check for all current existing backup files
    local currentbackupfiles = 1
    local tfile = ""
    local fh=0
    while (1) do
        tfile = bdir.."\\".. fname..bmid..currentbackupfiles..fext
        fh = io.open (tfile, "r")
        if fh == nil then break end
        io.close(fh)
        currentbackupfiles = currentbackupfiles + 1
    end
    -- move old backupfiles to 1 to (backupfiles-1)
    if backupfiles > 0 and backupfiles < currentbackupfiles then
        for y=1,currentbackupfiles-1 do
            y2 = y + backupfiles - currentbackupfiles
            local fn1 = bdir.."\\".. fname..bmid..y..fext
            local fn2 = bdir.."\\".. fname..bmid..y2..fext
            if currentbackupfiles - y < backupfiles then
                os.remove (fn2)
                os.rename (fn1,fn2)
            else
                os.remove (fn1)
            end
        end
        currentbackupfiles = backupfiles
    end
    local fn1 = bdir.."\\"..fname..bmid..(currentbackupfiles)..fext
    os.remove (fn1)
    os.rename (filename, fn1)
    return false
end

On 7/6/2020 at 12:36 PM, Dan_555 said:

Hi, i'm jumping into this with a request. Can you add leading 0 before the numbers ?

That is of course possible, but requires a change in both this LUA backup script as in Tidy.   
Will have a look at that, but the complexity lies in de conversion from 1 digit to 2 digit filenames as they can both exist after we would make this change and is it worth that effort?

Jos

Edited August 15, 2020 by Jos

[Melba23]

Jos,

Quote

The Scite chronology already works properly [...] In my opinion newer backups should have the lower numbers.

May I add my support for this opinion - the latest backup should be the lowest number.

M23

[Jos]

Just now, Melba23 said:

May I add my support for this opinion - the latest backup should be the lowest number.

of course, but that would be even way more complex as the current logic for Tidy is the other way around, so I do not see any clean migration path for that...   
Do you?

Jos

 

[Melba23]

Jos,

Stick with the current SciTE backup naming - I have no problem with .bak extensions!

M23

[Jos]

31 minutes ago, Melba23 said:

Stick with the current SciTE backup naming - I have no problem with .bak extensions!

That in itself is no issue and simply leaves the old naming standard files alone after the change. My remark was more about changing the logic from  "1=oldest  10=newest"   to "1=newest  10=oldest" withing the same naming standard.  That would create an issue for all existing backups.

Jos  

Edited July 6, 2020 by Jos