extract all words from text files

[face]

how can i extract words from text files and save them as a list. 

program gets access to the folder that contains the text files (more than 2 text files)

reads all the text files contents

ignores special characters and numbers

extracts all words (no duplicates)

saves all extracted words in a .txt file as a list

textfilefolder.rar

[JohnOne]

Define "words"

Is bbfgytwxv a word?

[face]

words are sepearated by spaces, even made up words count

[JohnOne]

Then FileRead() and StringSplit() should get you started.

[jchd]

Adapt this to your context:

#include <File.au3>
#include <Array.au3>

Local $oDictionary = ObjCreate("Scripting.Dictionary")
Local $mypath = @ScriptDir
Local $aFiles = _FileListToArray($mypath, "*.txt", 1, 1)
Local $aWords

For $i = 1 To $aFiles[0]
    $aWords = StringRegExp(FileRead($aFiles[$i]), "\w+", 3)     ; change pattern to fit your definition of "word"
    For $Word In $aWords
        $oDictionary.ADD($Word, $Word)
    Next
Next

$aWords = $oDictionary.Items
FileWrite("words.txt", _ArrayToString($aWords, @CRLF))
ConsoleWrite($oDictionary.Count & " words found in " & $aFiles[0] & " files." & @LF)

[face]

 

Adapt this to your context:

#include <File.au3>
#include <Array.au3>

Local $oDictionary = ObjCreate("Scripting.Dictionary")
Local $mypath = @ScriptDir
Local $aFiles = _FileListToArray($mypath, "*.txt", 1, 1)
Local $aWords

For $i = 1 To $aFiles[0]
    $aWords = StringRegExp(FileRead($aFiles[$i]), "\w+", 3)     ; change pattern to fit your definition of "word"
    For $Word In $aWords
        $oDictionary.ADD($Word, $Word)
    Next
Next

$aWords = $oDictionary.Items
FileWrite("words.txt", _ArrayToString($aWords, @CRLF))
ConsoleWrite($oDictionary.Count & " words found in " & $aFiles[0] & " files." & @LF)

 

theres 1 error

i cant fix it

Edited December 30, 2013 by face

[Melba23]

face,

That syntax uses the new v3.3.10.# version of the function. If you have not yet updated (which from the error would seem to be the case) then change the line to read:

Local $aFiles = _FileListToArray($mypath, "*.txt", 1)

M23

[face]

i updated autoit version

now i get another error

 

[jchd]

As you can see I omitted every error checking. Most probably no file match your search.

[Melba23]

face,

Then we need to add some errorchecking code to see where the problem lies:

#include <File.au3>
#include <Array.au3>
#include <MsgBoxConstants.au3>

Local $oDictionary = ObjCreate("Scripting.Dictionary")
Local $mypath = @ScriptDir
Local $aFiles = _FileListToArray($mypath, "*.au3", 1, 1)
If @error Then
    MsgBox($MB_SYSTEMMODAL, "Error", "No files found")
    Exit
Else
    MsgBox($MB_SYSTEMMODAL, "Found", $aFiles[0] & " files")
EndIf

Local $aWords
For $i = 1 To $aFiles[0]
    $aWords = StringRegExp(FileRead($aFiles[$i]), "\w+", 3)     ; change pattern to fit your definition of "word"
    Local $iError = @error
    If $iError = 0 Then
        For $Word In $aWords
            $oDictionary.ADD($Word, $Word)
        Next
    Else
        MsgBox($MB_SYSTEMMODAL, "Error", $aFiles[$i] & " - " & $i & @CRLF & "error: " & $iError)
    EndIf
Next

$aWords = $oDictionary.Items
FileWrite("words.txt", _ArrayToString($aWords, @CRLF))

What error MsgBoxes do you get - if any?

M23

[face]

thank you so much, it works

[jdelaney]

Might want to look into only skipping spaces...(grabs special chars, that way...but then again, might not be what you expect...example: would return appostrophes )

$aWords = StringRegExp(FileRead($aFiles[$i]), "[^\s]+", 3)

or only including non spaces...

$aWords = StringRegExp(FileRead($aFiles[$i]), "\S+", 3)

Edited December 30, 2013 by jdelaney