Jump to content
Sign in to follow this  
darkleton

Accented characters in translation

Recommended Posts

darkleton

Hi,

I am using the googletranslate code that was posted on here a while back to create a little gui that asks for the text, which language to translate to and then will display a message box with the result.

the problem is that google returns accented characters for all languages and the msgbox can't display them.

for example the msgbox returns this for "summer is the best season" - L\u0026#39;t est la meilleure saison

Is there any way of making the script output correctly without too much messing around or is this going to involve strings and arrays and checking ascii?

thanks in advance :idea:

Share this post


Link to post
Share on other sites
jchd

That shouldn't be hard.

But first, can you post an exact sample of the string you retrieve this way. In the case of your example, the string should be "L'été est la meilleure saison" but the html codes don't reflect that.

Try posting more than one.

A simple line of code should do it nicely, but I need to know what to lookup and how to transform back into readable Unicode.


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites
darkleton

Sure :idea:

In translate.google.com, using the string "summer is the best season" I get:

L'été est la meilleure saison

Using the script I get

L\u0026#39;t est la meilleure saison

This is the quick script I put up to test it:

#include <GUIConstantsEx.au3>
dim $combo, $button, $input, $language
Example()

Func Example()
    Local $msg
    GUICreate("Quick Translator",330,180)  ; will create a dialog box that when displayed is centered
    GUICtrlCreateLabel("Translate:",10,20)
    $input = GUICtrlCreateInput("Input your text here",10,40, 300)
    GUICtrlCreateLabel("to",10,65)
    $combo = GUICtrlCreateCombo("French", 10, 85,70) ; create first item
    GUICtrlSetData(-1, "German|Spanish") ; add other item snd set a new default
    
    $button = GUICtrlCreateButton("Translate",10,130)
    
    GUISetState()

    While 1
        $msg = GUIGetMsg()
    Select  
    Case $msg = $GUI_EVENT_CLOSE
        ExitLoop
    Case $msg = $button
        Translate()
    EndSelect
    WEnd
EndFunc 

Func Translate()
$language = GUICtrlRead($combo)
$text = guictrlread($input)
If $language = "French" Then
    $tranlang = "fr"
    TranslatedBox($text, "en", $tranlang)
EndIf
If $language = "German" Then
    $tranlang = "de"
    TranslatedBox($text, "en", $tranlang)
EndIf
If $language = "Spanish" Then
    $tranlang = "es"
    TranslatedBox($text, "en", $tranlang)
EndIf
EndFunc

Func TranslatedBox($sText, $sFrom, $sTo)
    $translatedtext = _GoogleTranslate($sText, $sFrom, $sTo)
    $translatebox = MsgBox(4,"Copy to clipboard?",_GoogleTranslate($sText, $sFrom, $sTo))
    If $translatebox = 6 Then
        ClipPut($translatedtext)
    EndIf
EndFunc

Func _GoogleTranslate($sText, $sFrom = "en", $sTo = "ja")
    Local Const $FileName = "Translation.jsn"
    Local Const $Pattern = '"translatedText":"([^"]+)"'
    Local $GoogleURL = "http://ajax.googleapis.com/ajax/services/language/translate?v=1.0&q=%s&langpair=%s%%7C%s"
    Local $File
    
    $GoogleURL = StringFormat($GoogleURL, $sText, $sFrom, $sTo)
    
    If Not InetGet($GoogleURL, $FileName, 1) Then Return SetError(1, 0, 0)
        
    $File = FileOpen($FileName, 4)
    FileGetSize($FileName)
    $Translation = FileRead($File, FileGetSize($FileName))
    FileClose($File)
    FileDelete($FileName)
    
    $Translation = BinaryToString($Translation, 4)

    If StringRegExp($Translation , $Pattern) Then
        $Translation = StringRegExp($Translation , $Pattern, 1)
        $Translation = $Translation[0]
        
        Return $Translation
    EndIf
EndFunc

Thanks

Share this post


Link to post
Share on other sites
jchd

Well, your case is a little involved, but can be done nicely anyway. The trick was to unserstand what was going on and distroying data, then to realize that it's html escapes encoded in JSON.

Test this (I left _ConsoleWrite to let you see the steps):

#include <GUIConstantsEx.au3>
dim $combo, $button, $input, $language
Example()

Func Example()
    Local $msg
    GUICreate("Quick Translator",330,180) ; will create a dialog box that when displayed is centered
    GUICtrlCreateLabel("Translate:",10,20)
    $input = GUICtrlCreateInput("summer is the best season",10,40, 300)
    GUICtrlCreateLabel("to",10,65)
    $combo = GUICtrlCreateCombo("French", 10, 85,70) ; create first item
    GUICtrlSetData(-1, "German|Spanish") ; add other item snd set a new default

    $button = GUICtrlCreateButton("Translate",10,130)

    GUISetState()

    While 1
    $msg = GUIGetMsg()
    Select
    Case $msg = $GUI_EVENT_CLOSE
    ExitLoop
    Case $msg = $button
    Translate()
    EndSelect
    WEnd
EndFunc

Func Translate()
$language = GUICtrlRead($combo)
$text = guictrlread($input)
If $language = "French" Then
    $tranlang = "fr"
    TranslatedBox($text, "en", $tranlang)
EndIf
If $language = "German" Then
    $tranlang = "de"
    TranslatedBox($text, "en", $tranlang)
EndIf
If $language = "Spanish" Then
    $tranlang = "es"
    TranslatedBox($text, "en", $tranlang)
EndIf
EndFunc

Func TranslatedBox($sText, $sFrom, $sTo)
    $translatedtext = _GoogleTranslate($sText, $sFrom, $sTo)
    $translatebox = MsgBox(4,"Copy to clipboard?",_GoogleTranslate($sText, $sFrom, $sTo))
    If $translatebox = 6 Then
    ClipPut($translatedtext)
    EndIf
EndFunc

Func _GoogleTranslate($sText, $sFrom = "en", $sTo = "ja")
    Local Const $FileName = "Translation.jsn"
    Local Const $Pattern = '"translatedText":"([^"]+)"'
    Local $GoogleURL = "http://ajax.googleapis.com/ajax/services/language/translate?v=1.0&q=%s&langpair=%s%%7C%s"
    Local $File

    $GoogleURL = StringFormat($GoogleURL, $sText, $sFrom, $sTo)

    $Translation = BinaryToString(InetRead($GoogleURL, 1+4), 4)
_ConsoleWrite($Translation & @LF)
    $Translation = StringRegExp($Translation, '{"translatedText":"([^"]*)"},', 1)
_ConsoleWrite($Translation[0] & @LF)
    $Translation = Execute('"' & StringRegExpReplace($Translation[0], '(?:\\u(\d{4}))', '" & ChrW(0x$1) & "') & '"')
_ConsoleWrite($Translation & @LF)
    $Translation = Execute('"' & StringRegExpReplace($Translation, '(?:&#(\d+);)', '" & ChrW($1) & "') & '"')
_ConsoleWrite($Translation & @LF)
    Return $Translation
EndFunc

; this version will hapily display Unicode, if you switch Scite to Unicode display
; to do so, open Global Properties and apply this change:
;# Internationalisation
;# Japanese input code page 932 and ShiftJIS character set 128
;#code.page=932
;#character.set=128
;# Unicode
;code.page=65001    <<--
;#code.page=0   <<--
Func __ConsoleWrite($sText)
    Local $aResult = DllCall("kernel32.dll", "int", "WideCharToMultiByte", "uint", 65001, "dword", 0, "wstr", $sText, "int", -1, _
                                "ptr", 0, "int", 0, "ptr", 0, "ptr", 0)
    Local $tText = DllStructCreate("char[" & $aResult[0] & "]")
    DllCall("Kernel32.dll", "int", "WideCharToMultiByte", "uint", 65001, "dword", 0, "wstr", $sText, "int", -1, _
                            "ptr", DllStructGetPtr($tText), "int", $aResult[0], "ptr", 0, "ptr", 0)
    ConsoleWrite(DllStructGetData($tText, 1))
EndFunc ;==>__ConsoleWrite

This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites
darkleton

that is amazing. thank you so much for the help. :idea:

Share this post


Link to post
Share on other sites
jchd

You're welcome. This isn't as straightforward as usual (typically JSON \uabcd to translate into 0xabcd hex Unicode codepoint with ChrW, or html escape e.g. &amp; for an ampersand) due to their "need" to go thru both encoding successively. But I'm sure you got the idea with this example.

I also left out error checking after InetRead.

Cheers.


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.