Define Word Boundary - SOLVED

[czardas]

1 hour ago, alien4u said:

Hi @czardas
My code is really bad and it does not work properly but my point is with StringInStr() you will find where is the substring no matter what delimiter is there and base on that you could extract that word.

Regards
Alien.

Actually this did give me some ideas. StringIsAlpha() appears to be working for many alphabets and could be used in conjunction with your idea. It's a simple idea which is often best.

[czardas]

I believe I have solved this. I have never used (*UCP) and didn't know it would solve the problem so easily. Perhaps the RegExp syntax can be simplified or I might have overlooked something, but I'll mark the topic solved anyway. Thanks again - discussing this was useful for clarity.

Local $sString = '\E 師_green|house,деления<<<'
Local $sFind = 'деления'
MsgBox(0, "", StringRegExp(StringReplace($sString, '\E', ChrW(57344), 0, 1), '(*UCP)(?i)(\A|[^[:alpha:]])(\Q' & StringReplace($sFind, '\E', ChrW(57344), 0, 1) & '\E)(\z|[^[:alpha:]])'))

 

Edited April 25, 2016 by czardas

[jchd]

The definition of word is pretty fuzzy and mostly context-dependant. For instance would you consider COP21 as one word (granted it's an acronym) or only COP?

Nonetheless PCRE with UCP opens some possibilities ... and some pitfalls. The last "word" in the first returned array doesn't dissociate joined words from distinct scripts, but looking for "letter words" in specific scripts does:

Local $sString = ' 師_green|house,деления<<<Δελτα___ՅԴգէկՔիթטאףצװהםлениΔελτα'
Local $aWords = StringRegExp($sString, '(*UCP)(?<!\pL)\pL+', 3)
_ArrayDisplay($aWords, "Words")
$aWords = StringRegExp($sString, '(*UCP)(?<!\p{Armenian})\p{Armenian}+', 3)
_ArrayDisplay($aWords, "Armenian")
$aWords = StringRegExp($sString, '(*UCP)(?<!\p{Hebrew})\p{Hebrew}+', 3)
_ArrayDisplay($aWords, "Hebrew")
$aWords = StringRegExp($sString, '(*UCP)(?<!\p{Latin})\p{Latin}+', 3)
_ArrayDisplay($aWords, "Common latin")
$aWords = StringRegExp($sString, '(*UCP)(?<!\p{Han})\p{Han}+', 3)
_ArrayDisplay($aWords, "Chinese")
$aWords = StringRegExp($sString, '(*UCP)(?<!\p{Cyrillic})\p{Cyrillic}+', 3)
_ArrayDisplay($aWords, "Cyrillic")
$aWords = StringRegExp($sString, '(*UCP)(?<!\p{Greek})\p{Greek}+', 3)
_ArrayDisplay($aWords, "Greek")

Even in UCP mode, \b may not be what you want. Try to replace (?<!\p{Armenian}) by \b for an easy example. Also consider that many scripts have dedicated Unicode symbols for decimal digits. Thus \p{Devanagari}+ for instance will match both Devanagari letters and digits.

Disclaimer: don't expect real words here, I just concatenated random letters from some scripts I can't read. So if something here reads as a horrible offence to you, persuade yourself it's God's hand drawing a smilley for you.

[czardas]

I have 3 algorithms:

1. Find an exact string
2. Find a string within another string
3. Find a word within a string

It is assumed that the user knows how to write their own language and that they are using a keyboard which enables these characters to be typed. The language can contain any characters (mixed character sets or languages) and may not even have been invented yet, but it is assumed that some kind of non-alpha character delimiter exists between words. When such a delimiter is absent from the language, the first two alternative algorithms would be more suitable option to conduct the search. All bases are reasonably covered.

If the user wants to search for the word  COP21, who am I to argue.

Edited April 25, 2016 by czardas

[jchd]

My remark was generic, not precisely aimed at something/some code/some situation in particular. Take it as a sidenote.

[czardas]

1 hour ago, jchd said:

My remark was generic, not precisely aimed at something/some code/some situation in particular. Take it as a sidenote.

I really appreciate the knowledge you share here. I'm trying to avoid the Swiss Army Knife approach and cast the net wide instead. The level of detail, your tips and tricks, will surely be welcome by many. So thank you! 

Edit: After further consideration [^[alpha]] in my previous solution would be better if it were replaced by [^[alnum]], so searching for COP21 would not return a result for COP2175430. 

Edited April 25, 2016 by czardas