david1337

StringReplace special characters in htm file

7 posts in this topic

Hey guys

Can anyone help me explain this? :)

$szFile = "test.htm"

$szText = FileRead($szFile)


$szText = StringReplace($szText, "hello", "ö")

FileDelete($szFile)
FileWrite($szFile,$szText)

If the file "test.htm" has it's text changed into something containing non US characters, in this example "ö", the output is " ö " when shown in a browser.
If i manually change the text in the "test.htm" file to "ö" - the output in the browser is "ö" !
In both cases, if the htm file is opened in notepad, the content is just "ö" - but the one changed from the script, still opens as " ö " in a browser. How weird is this?

I am aware that I can replace the text to " ö" , which is the HTML code for "ö" - then the output is correct in the browser, but this is just dumb when there are a lot of characters to be changed :)


Does anyone know why this happens, and how to solve it in a more simple way?

 

Share this post


Link to post
Share on other sites



You're seeing UTF8 encoding of characters. Change the html header to indicate UTF8.


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites
27 minutes ago, jchd said:

You're seeing UTF8 encoding of characters. Change the html header to indicate UTF8.

Are you sure it's the header. I think it's the filewrite using the wrong mode.

@david1337: so test first:

$szFile = "test.htm"

$szText = FileRead($szFile)


$szText = StringReplace($szText, "hello", "ö")
$hFile=FileOpen($hFile,$FO_OVERWRITE + $FO_UTF8)
FileWrite($hFile,$szText)
FileClose($hFile)

 

Share this post


Link to post
Share on other sites

Either you leave the header as ISO and use FileOpen with the ANSI mode to write the file,
or switch to full Unicode and switch the header to UTF8.

The later solution is universal, not the former.

1 person likes this

This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

#5 ·  Posted (edited)

Hey guys, thanks for your answer.

2 hours ago, AutoBert said:

Are you sure it's the header. I think it's the filewrite using the wrong mode.

@david1337: so test first:

$szFile = "test.htm"

$szText = FileRead($szFile)


$szText = StringReplace($szText, "hello", "ö")
$hFile=FileOpen($hFile,$FO_OVERWRITE + $FO_UTF8)
FileWrite($hFile,$szText)
FileClose($hFile)

 

You code gives an error: (Am I missing an include for or something for the code to understand what "$FO_UTF8" is?)
==> Variable used without being declared.:
$hFile=FileOpen($hFile,$FO_OVERWRITE + $FO_UTF8)
$hFile=FileOpen(^ ERROR

 

2 hours ago, jchd said:

Either you leave the header as ISO and use FileOpen with the ANSI mode to write the file,
or switch to full Unicode and switch the header to UTF8.

The later solution is universal, not the former.

You are correct. Changing the HTML header to UTF-8 fixed the problem :)
But what if I do the same thing with a txt file, and open that in a web browser? Then I have the same problem, and I can't add a header to a txt file.
 

Edited by david1337

Share this post


Link to post
Share on other sites
4 minutes ago, david1337 said:

You code gives an error: (Am I missing an include for or something for the code to understand what "$FO_UTF8" is?)

Yes a include is missing, but there's a 2. error (typo):

#include <FileConstants.au3>

$szFile = "test.htm"

$szText = FileRead($szFile)


$szText = StringReplace($szText, "hello", "ö")
$hFile=FileOpen($szFile,$FO_OVERWRITE + $FO_UTF8)
FileWrite($hFile,$szText)
FileClose($hFile)

and maybe you need other mode:

$hFile=FileOpen($szFile,$FO_OVERWRITE + $FO_ANSI)

 

1 person likes this

Share this post


Link to post
Share on other sites

#7 ·  Posted (edited)

56 minutes ago, AutoBert said:

 

#include <FileConstants.au3>

$szFile = "test.htm"

$szText = FileRead($szFile)


$szText = StringReplace($szText, "hello", "ö")
$hFile=FileOpen($szFile,$FO_OVERWRITE + $FO_UTF8)
FileWrite($hFile,$szText)
FileClose($hFile)

 

AutoBert, this was exactly what I was looking for, and it worked perfectly! Thanks a lot :)

Edited by david1337

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now

  • Similar Content

    • AnAdventurer
      Newbie with a question... or two (IE focused)
      By AnAdventurer
      Hello hello!
      As the title suggests, I am fairly new to AutoIt. In fact, I am new to scripting/coding in general! I've done a few Codecademy courses on CSS and HTML and perhaps Java though this was all a few years back. I've recently come across AutoIt and decided to give it a try since I do quite a few repetitive tasks on a daily basis. In the last couple of weeks I've managed to master (or at least get comfortable with) mouse clicks(left/right), window focus, sending key strokes, controls, and pixel search.
      Now let's get to the topic.
      At this point in time I've tried out a few simple IE scripts but I am having difficulty understanding some things and tying everything together into one tool.
      Specifically, I am struggling with this little bit of code I got from DaleHohm in his IE examples thread. Post #3 (The last example.)
      #include <IE.au3> $sImgDir = "c:\foo\"; Please make certain this folder already exists (silent failure if not) $sWebPage = "http://www.autoitscript.com/forum/index.php?"; webpage with images $oIE = _IECreate() _IENavigate($oIE, $sWebPage) $oIMGs = _IETagNameGetCollection($oIE.document, "img") ; Loop through all IMG tags and save file to local directory using INetGet For $oIMG in $oIMGs $sImgUrl = $oIMG.src $sImgFileName = $oIMG.nameProp INetGet($sImgUrl, $sImgDir & $sImgFileName) Next I have a couple questions about the code above.
      1) ".src" ".nameProp" What are these called? I figured out that I can change the .src to something like .href and it gets anything on the webpage with a .href tag but where can I learn more about these? I still haven't been able to figure out what ".nameProp" is for or what it does. Is there any documentation/list of all the different ".PurpleTextAfterAVariable" (Edit: Not sure why it's red in the above example, just checked SciTE and it's purple there) that I can use?
      2) I understand that the code above gets every "For $oIMG in $oIMGs" on the page but how can I make it only get the first 5? I've tried doing a "count" and a "for" but I am unsure what to replace the "For...in" statement with to keep the script functional. Is there a way to limit the _IETagNameGetCollection function to only get a specific amount of tags?
       
      Finally, the reason I can't just use the code as is.
      The site I am trying to get images from works in this way:
      A href= "Link-To-Picture.jpg" Img src= "Link-To-Picture-thumbnail.jpg" The script above downloads every single thumbnail from the image gallery which is great, it does what it's supposed to but I need the full resolution image.
      After changing the script to get anything with an "A href" tag it does what I need it to do, it gets every single image in full resolution... along with every single one of the 80-100 extra files/links to other sites that are listed under an "A href" tag.
       
      Now I've come up with two solutions but unfortunately, as I mentioned above. I don't know how to put my solution into the code above to make it work.
      Solution 1) Only get the first 5 instances of "A href" on the page.
      As mentioned above. I don't know how to do this.
      Solution 2) Read the entire page, find "-Thumbnail.jpg" replace with ".jpg" and use the script as is.
      I understand how to do a replace. All I am missing is how to do a replace within a field in the code of an IE page. I assume that I have to use the HTMLRead functions but how do I use/alter the data read?
      I really hope all of this make sense and that someone here will be able to help me figure out a solution to my issue or at least answer one of my questions! I do have plenty more questions and I am sure that I'll have even more by the time I figure this out.
      Thank you very much for your time!
    • SorryButImaNewbie
      [SOLVED: basic COM help] Working on string returned from API, m'i doing this okey?
      By SorryButImaNewbie
      Hello, I try to pull some data from a webpage.
      I need the value of local currency compared to the euro. I can go and open the required API page on the required date interval, read in from elsewhere, its format in the memory of the script is like this: 20161005 so YYYYMMDD.
      The return string if I try to view the opened API's source code is simple, but if I use _IEBodyReadHTML, _IEDocReadHTML, _IEBodyReadText i get it back with a lot of html code (i guess, it looks like HTML, and one of them doesn't show any string in the MsgBox when I try to chechk it) about its color etc. I need dates and the corresponding currency exchange rates (these can be found between  <kozep>exchangerate</kozep>, but I need the first only after every month because the second is the avarage exchange rate of the month (i guessed this again).
      Now I have an approach which will work evantually I guess, but I'm pretty sure its not the standard aproach or how the creators of autoit envisiond the useage of their functions
      So I post my code here hoping, someone tells me how to do this simply and inteligently.
      Sorry for such question but I only used regex for much, much simplier tasks. My approach is to identify everything I dont need basicly, after getting rid of a few key problematic chars (like " ) and do this untill I'm only left with what I need. THe problem with this if anything change in the envierment the script has like 99,9999% chance to not run properly, and I would like to handle this better, even if APIs usually don't change that much according to my knowladge.
      Also I write this in a separat function for now, I will plan to call it from my other function which does different things with the corresponding excel files, among them is the calculation of local currency values of the bills with data from MNB (Hungarian National Bank or something)
      Here is my code so far, and what its gives back, I will update this with the pic from the source code I see from internet explorer and the webpage I see. Thank you for your help and insight!
      Func InternetRead() ;Create the URL for napiarfolyam API #cs http://api.napiarfolyam.hu/?bank=mnb&valuta=eur&datum=20160901&datumend=20160926 </penznem> után jön a használt árfolyam Példa Return: <item> <bank>mnb</bank> <datum>2016-09-06 11:25:18</datum> <penznem>EUR</penznem> <kozep>309.8500</kozep> <kozep>310.1700</kozep> </item> #ce ;Global $MinTime ;20160601000000 these are example variables I read in, during the function that will call this one ;Global $MaxTime ;20160610000000 Local $URLbase = "http://api.napiarfolyam.hu/?bank=mnb&valuta=eur" ;view-source: ;Local $MinTimeFormated = StringTrimRight($MinTime, 6) ;Local $MaxTimeFormated = StringTrimRight($MaxTime, 6) Local $URL = $URLbase & ("&datum=" & "20160601" & "&datumend=" & "20160603" & "") ;20160603 $MinTimeFormated, $MaxTimeFormated MsgBox(64, "Értesítés", "URL:" & $URL & "") Local $oIE = _IECreate($URL) Sleep(1000) Local $sHTML = _IEDocReadHTML($oIE) ;_IEBodyReadHTML - Is string but, MsgBox shows nothing ;_IEDocReadHTML - at least retunrs something (extra then what i see from thw source code, ctrl+u) ;_IEBodyReadText - at least retunrs something (extra then what i see from thw source code, ctrl+u) $sHTML = String($sHTML) If IsString($sHTML) Then MsgBox(64, "HTML String?", "The variable is a string") Else MsgBox(64, "HTML String?", "The variable is not a string") EndIf ;Variable is a String! ;StringSplit ;">datum</span>&gt;</a>" & "20160601" Local $Stuff = Chr(34) ;The " char ;Local $Stuff2 = "<a xmlns=http://www.w3.org/1999/xhtml class=collapse style=color: blue; marginleft: 2em; position: relative; href=#>&lt;<span style=color: rgb(153,0,0);>" Local $StringInput = $sHTML Local $sHTML = StringRegExpReplace($StringInput, "[-]", "") Local $StringInput = $sHTML Local $sHTML = StringStripWS($StringInput, $STR_STRIPLEADING + $STR_STRIPTRAILING + $STR_STRIPSPACES) Local $StringInput = $sHTML Local $sHTML = StringReplace($StringInput, $Stuff, "") ;Local $StringInput = $sHTML ;Local $sHTML = StringReplace($StringInput, $Stuff2, "") Local $StringInput = $sHTML Local $ValutaPosition = StringInStr($StringInput, "</valuta>") Local $sHTML = StringTrimLeft($StringInput, $ValutaPosition+8) Local $StringInput = $sHTML ;StringReplace($StringInput, "<a xmlns="http://" ;Local $StringInput = $sHTML ;StringInStr ;Local $sHTML = StringTrimLeft($StringInput, 1850) ;Local $aDays = StringSplit($sHTML, ">datum</span>&gt;</a>") ;_ArrayDisplay($aDays) ;If @error Then Exit MsgBox($MB_SYSTEMMODAL, "StringRegExpReplace Error", "Error listing:" & @CRLF & "@error = " & @error & ", @extended = " & @extended) MsgBox(64, "HTML String?", "$sHTML:" & $sHTML) EndFunc ;==>InternetRead  

      Edit:
      Sorry for the long post and I hope I was able to write dowm my problem in a way that others can understand, pls ask anything if you don't.
    • coffeeturtle
      StringReplace differentiating numeric values
      By coffeeturtle
      Hello. I need to perform a specific string replace, but not sure how to go about it.
      The scenario is this: I have a large block of text. Within the text colons appear ":",  Sometimes the colons are used in a sentence appearing after a word. Other times they appear in between numbers like a ratio or a sport score (e.g. "6:8").
      I want to replace the colons appearing between numeric values like 6:8 with the word "to", but not the ones appearing at the end of a sentence.
      Is there a way that I can have StringReplace (or any other method) differentiate when to replace the colon based on it appearing between numbers?
      I did try searching for a similar scenario.
      Thank you for any help. 
    • cookiemonster
      StringReplace from string with line break
      By cookiemonster
      Im trying to edit a file, I want to find a string which has a line break in it, and replace it with a string that has multiple line breaks in it.
      editfile.txt looks like this:
      dog cat mouse chicken my au3 script looks like this but is not working, i suspect because of how I am trying to do the line breaks?
      Func EditFile($CurrentFile) $szFile = "$CurrentFile" $szText = FileRead($szFile,FileGetSize($szFile)) $szText = StringReplace($szText, "Cat" & @CRLF & "Dog", "Hippo" & @CRLF "Lion" & @CRLF & "Tiger") FileDelete($szFile) FileWrite($szFile,$szText) EndFunc ;--EditFile-- But once ran the file should look like this:
      Hippo Lion Tiger cat mouse chicken I cannot replace by line number as the animals are not in the same line in each copy of the file i want to run against.
       
      Can anyone help?
    • jsmcpn
      Having trouble with $CmdLine when parsing an arg with spaces and special chars
      By jsmcpn
      Hello all!  I'm trying to make a little script that sends a message over TCP.  An example payload would be:
      MY_PAYLOAD#<ATTRS><ATTR><NAME>MAILSERVER</NAME><OPERATION>set_value(ServiceStatus)</OPERATION><VALUE><![CDATA["Not Running"]]></VALUE><TIMESTAMP>1349297031</TIMESTAMP></ATTR></ATTRS> Note how the payload contains special characters, and how the CDATA contains a SPACE (i.e.  "Not Running")
      My compiled AU3 script expects exactly two command line arguments:
      1) destination server/port expressed as 127.0.0.1:80
      2) the payload I want to send to my server application (which may have multiple ATTRs, this example only has one ATTR)
      MyApp.exe 127.0.0.1:80 "MY_PAYLOAD#<ATTRS><ATTR><NAME>MAILSERVER</NAME><OPERATION>set_value(ServiceStatus)</OPERATION><VALUE><![CDATA["Not Running"]]></VALUE><TIMESTAMP>1349297031</TIMESTAMP></ATTR></ATTRS>"  
      My problems (plural) are with parsing of the second argument:
      1)  If I don't wrap second argument in double-quotes, then Windows bails out before even launching EXE with "< was unexpected at this time"
      2)  If I DO wrap second argument in double-quotes, then two problems can arise when my EXE parses the arguments:
                   aa) The double-quotes are stripped from the CDATA value. ["Running"] is stored in the array as [Running]
                   bb) If CDATA value DOES have a space, such as ["Not Running"], then I get an extra unwanted $CmdLine[3] because the SPACE between Not and Running is treated as a separator.  My second argument is thus split into two separate arguments, [2] and [3] instead of being treated as one arg stored in [2]
                              cc) Additionally, my script exits because it has an "IF $CmdLine[0] <>2 THEN Exit (1)" to validate the command line args.
      I've tried launching the EXE with the second argument wrapped and escaped in a multitude of ways.  Double double-quotes, triple double-quotes, single-quotes on the outside with double-quotes inside (i.e.  '"ETC"'), starting the second arg with a caret (i.e. ^"ETC") and one of three things happens:
      1) "< was unexpected at this time"
      2) my CDATA value is stripped of its double-quotes
      3) my payload argument gets split at the space in the CDATA value and the script bails out due to the unexpected 3rd argument.
       
      Any ideas?  How can I preserve the second argument literally, including all special characters, quotes and spaces?
      I tried adding some string manipulation code to grab the 2nd argument from $CmdLineRaw and it works (preserves the "Not Running" with quotes), but my number of arguments is still 3 instead of 2 if there is a space in the CDATA value.
       
      Any assistance would be greatly appreciated!