XML - Simultaneous Access

[elpresident3]

Can Autoit + XML cope with simultanious updates from hundreds of users? If so, How?

I have made a program to Take the username, pc name and place it into an xml document for the purpose of reporting whether a Lotus Notes initialization script has run successfully. This is my first script, pulled together from bits and pieces of the Example scripts forum. So it's a bit of a mess, but it has been my first attempt at programming and I hope it can help other new coders out there understand XML communication a bit more.

I wish to deploy this model to around 600 users... at the same time and I am wondering how XML will cope with this.

My script will probably only write to the xml file from whoever out of 600 runs it first. Is there anyone out there who can help me work out how I can get the most amount of users to write to the xml file at the same time.

The XML file will be saved on a gigabit network share on a raid drive.

Be sure to change your computername in the xml document after running the script once for testing.

xmlupdater.au3

expandcollapse popup

#Include <String.au3>
#Include <_XMLDomWrapper.au3>
#include <Array.au3>
#include <Date.au3>
;Add in a check if @username = "Administrator" msgbox: you cannot run this script as administrator
Local $sXmlFile, $oOXml, $n, $s
Global $node, $snode, $b, $r, $XMLloc, $no, $arr, $narray[10][5], $found, $dc, $record, $ifound, $rarray_size, $as, $strr1, $iChildct

Dim $XMLclient[2]
Dim $XMLclientval[2]
Dim $XMLexe[5]
Dim $XMLexeval[5]
Dim $dc = ""
Dim $found = ""
Dim $ifound = ""
Dim $record = False
Dim $rarray_size = 0
Dim $as = 5
Dim $strr1 = ""

ReDim $XMLexe[$as]
ReDim $XMLexeval[$as]
ReDim $narray[10][$as]

$XMLclient[0] = "id"
$XMLclient[1] = "name"
$XMLclientval[0] = @UserName
$XMLclientval[1] = "Firstname Lastname"

$XMLexe[0] = "Time"
$XMLexe[1] = "Computer"
$XMLexe[2] = "OS"
$XMLexe[3] = "SP"
$XMLexe[4] = "BAD"

$XMLexeval[0] = _NowCalc()
$XMLexeval[1] = @ComputerName
$XMLexeval[2] = @OSVersion
$XMLexeval[3] = @OSServicePack
$XMLexeval[4] = "NO"
$yes = "YES"

$XMLname ="new"
$sXmlFile = "new.xml"
$n = "//Inst/Client"
$s = "/Exe"
$oOXml = ""

     
Func GetNode($node)
    Local $x, $y, $iNodeCt
    $iNodeCt = _XMLGetNodeCount($node, "", $NODE_ELEMENT)
        
    For $x = 1 To $iNodeCt 
        GetAlloc($x,$n,$s)
    Next
    Return $narray
EndFunc
 
Func GetAlloc($in,$node,$snode)
    Local $iNodeCt, $sState, $sState2, $aAttrN[1], $aAttrV[1], $ret
    Local $x, $y, $z, $a, $b, $str, $comp

    $iIndex = "[" & $in & "]"
    $sState = _XMLGetAttrib ($node & $iIndex, "id")

    IF $sState = @UserName Then 
        $record = True
        $ifound = String($iIndex)
        $sState2 = _XMLGetAttrib ($node & $iIndex, "name")
        $iChildct = _XMLGetNodeCount ($node & $iIndex & $snode, "", $NODE_ELEMENT)
        $rarray_size = $iChildct - 1
        If $iChildct > 0 Then 
            ReDim $narray[$iChildct][$as]
        EndIf
    
        For $y = 1 To $iChildct;$iChildct = The ammount of times installed under this client id;consolewrite($y & @LF)
        
            If $iChildct  = 0 Then
                MsgBox( 4096, "", "childcount 0: " & $iChildct )
                _XMLGetAllAttrib ($node & $iIndex & $snode, $aAttrN, $aAttrV)
            Else
                _XMLGetAllAttrib ($node & $iIndex & $snode, $aAttrN, $aAttrV, "[" & $y & "]")
            EndIf
            
            $str = String(@ComputerName)
            $comp = String($aAttrV[1])
    ;Consolewrite("<"& $str & "><" & $comp & ">"& @lf)
            $narray[$y -1][0] = $aAttrV[0]
            $narray[$y -1][1] = $aAttrV[1]
            $narray[$y -1][2] = $aAttrV[2]
            $narray[$y -1][3] = $aAttrV[3] 
            $narray[$y -1][4] = $aAttrV[4]
            
            For $r1 = 1 to $iChildct
                $r2 = $r1 - 1 
                $strr1 = ""
                $strr1 = $strr1 & $narray[$r2][1] & " had Lotus Configured on " & $narray[$r2][0]  & " while running " & $narray[$r2][2] & @CRLF
            Next
            
            IF $narray[$y -1][4] = "YES" Then 
                    MsgBox(0,'Program Terminating', "Please call the helpdesk on 4200 to have lotus configured manually." & @CRLF & @CRLF & @CRLF & "Lotus Configuration History: " &  @CRLF &  @CRLF &  $strr1)
                    Exit
            Else
                If $narray[$iChildct-1][1] = @ComputerName Then;check if the last installation was done on this computer, under this username.
                    $ianswer = MsgBox(4,'Warning - Workstation Previously Configured', "This PC ("& @ComputerName &") has been previously configured on: " & $aAttrV[0] & @CRLF & @CRLF & "Is this the PC where you last logged into Lotus Notes succesfully?")
                    Select
                        Case $ianswer = 6;Yes
                            MsgBox(0,'PC already configured', "Please try running Lotus Notes, If it is still not working then call the helpdesk on 4200")
                        Case $ianswer = 7;No
                            $cIndex = "[" & $iChildct & "]"
                            $cstring = "//Inst/Client"& $iIndex & "/Exe"& $cIndex
                            _XMLSetAttrib ($cstring, "BAD", "YES")
                            MsgBox(0,'Program Terminating', "Please call the helpdesk on 4200 to have lotus configured manually." & @CRLF & @CRLF & @CRLF & "Lotus Configuration History: " &  @CRLF &  @CRLF &  $strr1)
                    EndSelect
                    exit
                EndIf
            EndIf
        Next
    EndIf
 Return
EndFunc

Func StartNode()
    $iNodeCt = String("["&(_XMLGetNodeCount ("/Inst/Client")+1)&"]")
    _XMLCreateRootNodeWAttr ("Client", $XMLclient, $XMLclientval)
    If $iNodeCt = "[0]" Then $iNodeCt = ""
    _XMLCreateChildWAttr("/Inst/Client"&$iNodeCt,"Exe", $XMLexe, $XMLexeval)
;ConsoleWrite("/Inst/Client"&$iNodeCt)
EndFunc
 
Func UpdateNode()
    _XMLCreateChildWAttr("/Inst/Client"&$ifound,"Exe", $XMLexe, $XMLexeval)
EndFunc

$oOXml = _XMLFileOpen ($sXmlFile)
$rarray = GetNode($n)

If $record = True Then;Move the folder item and update the script with the lastest location
;copy from 
    $dpath = "\\" & $rarray[$iChildct -1 ][1] & "\c$\Documents and Settings\" & @username & "\Local Settings\Lotus\Notes"
    $mpath = "c:\Documents and Settings\" & @username & "\Local Settings\Lotus\Notes\"

;FileCopy ("w:\data\", $mpath, 1)
    MsgBox(0,"","Copy \Data\ From" & @CRLF & @CRLF & $dpath & @CRLF & @CRLF & "Paste to" & @CRLF & $mpath)
    UpdateNode()
Else; First time running the script
    StartNode()
EndIf


#cs
Conditions of reading/writing into the XML
a -----Install Script is run succesfully, Entry is made----\
1 does your username appear
2 what was the last place the script was run
3 if the last place was this computer EXIT
4 if the last place was another computer ask if the settings would be copied.
b -----Install Script has not run, There should be no record of @usernname
1 does username appear
2 Ask - run install script / update xml to add this computer (automate?) 
#ce

new.xml

<?xml version="1.0" encoding="utf-8"?>
<Inst>
    <Client id="demczuk" name="Firstname Lastname">
        <Exe Time="2008/03/13 02:16:17" Computer="UB1863-221" OS="WIN_XP" SP="Service Pack 2" BAD="NO"/>
        <Exe Time="2008/05/13 02:32:34" Computer="UB1863-222" OS="WIN_XP" SP="Service Pack 2" BAD="NO"/>
    </Client>
    <Client id="sam" name="Firstname Lastname">
        <Exe Time="2008/06/13 02:36:17" Computer="UB1863-223" OS="WIN_XP" SP="Service Pack 2" BAD="NO"/>
        <Exe Time="2008/09/13 02:38:34" Computer="UB1863-224" OS="WIN_XP" SP="Service Pack 2" BAD="NO"/>
    </Client>
</Inst>

Edited September 13, 2008 by elpresident3

[PsaltyDS]

Can Autoit + XML cope with simultanious updates from hundreds of users? If so, How?

I have made a program to Take the username, pc name and place it into an xml document for the purpose of reporting whether a Lotus Notes initialization script has run successfully. This is my first script, pulled together from bits and pieces of the Example scripts forum. So it's a bit of a mess, but it has been my first attempt at programming and I hope it can help other new coders out there understand XML communication a bit more.

I wish to deploy this model to around 600 users... at the same time and I am wondering how XML will cope with this.

My script will probably only write to the xml file from whoever out of 600 runs it first. Is there anyone out there who can help me work out how I can get the most amount of users to write to the xml file at the same time.

The XML file will be saved on a gigabit network share on a raid drive.

Be sure to change your computername in the xml document after running the script once for testing.

xmlupdater.au3

Well, first things first. There is no magic in an XML file. It's just a plain text file with a specific format for the lines.

Like any other text file, it cannot be written to by more than one process AT THE SAME TIME. If you include a little error handling so that your clients can detect when they fail to open the file for write, wait a short (preferably randomized) few seconds, and then try again, you can have multiple clients politely taking turns writing to the file. Six hundred ACTIVE clients would be pushing this model too far, but your description doesn't sound like there would be that many client writes. So this might be all you need.

If you really had to scale up to many clients writing at the same time, then you would want them connecting to a database server. The database file can only be written to by one process also, but that process belongs to the DB server software, which can accept updates from many clients at the same time and arrange for them to be written in an orderly fashion.

Hope that helps.

Edited September 13, 2008 by PsaltyDS

[elpresident3]

Thank you PsaltyDS for clearing my head on this. As this was my first real project it has been a learning experience trying to write and read data from a data source.

You are right by mentioning that all 600 users might not write to the file at once. This is basically an asset tracking script, but I have no idea how to get a client to wait until the file has been previously closed. Any help with this would be greatly appreciated.

[Bowmore]

One method that I have used in a similar situation is to have each client write it's data to a temporary file in a folder. Another script then polls the files in this folder and updates the master file with the data before deleting the temporary file. This avoids the problem of multiple clients trying to write to the same file at once. I've not used it with 600 clients but have run it with 50 clients for over a week with no glitches.

[elpresident3]

I'll try that. I've done a search for _XMLFileClose and a suggested option was to open a temporary file after you have opened and saved to the desired one so that it does not remain open for the duration of the program.

Global $sFile
Dim $sFile = "c:\temp\temp.xml"

;Close the XML by opening a temp file
_XMLCreateFile ($sFile, "menu", True)
_XMLFileOpen ($sFile)
FileDelete ( $sFile )

I'll add some code in later to check how long the file was open

Edited September 14, 2008 by elpresident3

[elpresident3]

One method that I have used in a similar situation is to have each client write it's data to a temporary file in a folder. Another script then polls the files in this folder and updates the master file with the data before deleting the temporary file. This avoids the problem of multiple clients trying to write to the same file at once. I've not used it with 600 clients but have run it with 50 clients for over a week with no glitches.

You were right, XML was not meant to be read/written to by many users at the same time.

After testing my script I found that read/writing to one file took 3 seconds, multiplied by 600 would be around 30mins of access time required to one file. Even if I had split the file into a section for the first half of the alphabet and the second l-z, it would've taken at least 120 seconds of access time.

The solution in my case was through logic. If each person required to run this application several times on several computers the one condition to prevent using a single master.xml is that each person would never run the application on two computers at once.

<Solution>

If I made an xml file for each username (which is unique) I could list the updates and each file would only be run by one person. I could combine the total results from all the files through a filelist-to array function later for reporting.

</Solution>

Thanks guys, I hope the newbies out there can use some of the code I created as it took me about 20 hours to get to that stage without much knowledge of programming.