Xenobiologist

_FTP_... functions use codepage (zOS) (solved)

25 posts in this topic

#1 ·  Posted (edited)

Hi folks,

I'm using the udf #include <FTPEx.au3> to "download" some files from our zOS (MVS) system via autoit FTP with _FTP_FileGet.

The FileGet works, but the codepage is wrong. Special German chars like öäüß are not transfered correctly.

Any idea how to set the codepage? Swichting between binary or ascii doesn't solve the problem.

Using the ftp command in a cmd.exe window it works like this : quote site sbd=(IBM-273,iso8859-1)

Thanks

Mega

Edited by Xenobiologist

Scripts & functions Organize Includes Let Scite organize the include files

Yahtzee The game "Yahtzee" (Kniffel, DiceLion)

LoginWrapper Secure scripts by adding a query (authentication)

_RunOnlyOnThis UDF Make sure that a script can only be executed on ... (Windows / HD / ...)

Internet-Café Server/Client Application Open CD, Start Browser, Lock remote client, etc.

MultipleFuncsWithOneHotkey Start different funcs by hitting one hotkey different times

Share this post


Link to post
Share on other sites



Please make any testing file (the best will be simple txt file) zip !!! them and post this ZIP archive here..
 


Signature beginning:   Wondering who uses AutoIT and what it can be used for ?
* GHAPI UDF - modest begining - comunication with GitHub REST API *
ADO.au3 UDF     POP3.au3 UDF     XML.au3 UDF    How to use IE.au3  UDF with  AutoIt v3.3.14.x  for other useful stuff click the following button

Spoiler

Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind. 

My contribution (my own projects): * Debenu Quick PDF Library - UDF * Debenu PDF Viewer SDK - UDF * Acrobat Reader - ActiveX Viewer * UDF for PDFCreator v1.x.x * XZip - UDF * AppCompatFlags UDF * CrowdinAPI UDF * _WinMergeCompare2Files() * _JavaExceptionAdd() * _IsBeta() * Writing DPI Awareness App - workaround * _AutoIt_RequiredVersion() * Chilkatsoft.au3 UDF * TeamViewer.au3 UDF * JavaManagement UDF * VIES over SOAP * WinSCP UDF * GHAPI UDF - modest begining - comunication with GitHub REST API *

My contribution to others projects or UDF based on  others projects: * _sql.au3 UDF  * POP3.au3 UDF *  RTF Printer - UDF * XML.au3 - BETA * ADO.au3 UDF SMTP Mailer UDF *

Useful links: * Forum Rules * Forum etiquette *  Forum Information and FAQs * How to post code on the forum * AutoIt Online Documentation * AutoIt Online Beta Documentation * SciTE4AutoIt3 getting started * Convert text blocks to AutoIt code * Games made in Autoit * Programming related sites * Polish AutoIt Tutorial * DllCall Code Generator * 

Wiki: Expand your knowledge - AutoIt Wiki * Collection of User Defined Functions * How to use HelpFile * Best coding practices * 

IE Related:  * How to use IE.au3  UDF with  AutoIt v3.3.14.x * Why isn't Autoit able to click a Javascript Dialog? * Clicking javascript button with no ID * IE document >> save as MHT file * IETab Switcher (by LarsJ ) * HTML Entities * _IEquerySelectorAll() (by uncommon) * 

I encourage you to read: * Global Vars * Best Coding Practices * Please explain code used in Help file for several File functions * OOP-like approach in AutoIt * UDF-Spec Questions *  EXAMPLE: How To Catch ConsoleWrite() output to a file or to CMD *

"Homo sum; humani nil a me alienum puto" - Publius Terentius Afer
"Program are meant to be read by humans and only incidentally for computers and execute" - Donald Knuth, "The Art of Computer Programming"
:naughty:  :ranting:, be  :) and       \\//_.

Anticipating Errors :  "Any program that accepts data from a user must include code to validate that data before sending it to the data store. You cannot rely on the data store, ...., or even your programming language to notify you of problems. You must check every byte entered by your users, making sure that data is the correct type for its field and that required fields are not empty."

Signature last update: 2017-06-04

Share this post


Link to post
Share on other sites

#3 ·  Posted (edited)

I don't know how you can test this. This is a FTP download from a zOS system.  Use a file which is in codepage EBCDIC with some öäüß and try to download it via FTPGet.

I'm using somethig like this:

#include <Array.au3>
#include <FTPEx.au3>
#include <MsgBoxConstants.au3>
#include <Process.au3>

Local $sServer = 'etet' ; zoS MVS System (IBM)
Local $sUsername = @UserName
Local $sPass = 'xxx'

Local $hOpen = _FTP_Open('MyFTP Control')
Local $hConn = _FTP_Connect($hOpen, $sServer, $sUsername, $sPass, 0, 21, $INTERNET_SERVICE_FTP, $INTERNET_FLAG_TRANSFER_ASCII)
;~ Local $hConn = _FTP_Connect($hOpen, $sServer, $sUsername, $sPass, 0, 21, $INTERNET_SERVICE_FTP, $INTERNET_FLAG_TRANSFER_BINARY)
If @error Then
    MsgBox($MB_SYSTEMMODAL, '_FTP_Connect', 'ERROR=' & @error)
Else
;~  ConsoleWrite(_FTP_Command($hConn, 'quote site sbd=(IBM-273,iso8859-1)') & @CRLF)
    ConsoleWrite(_FTP_FileGet($hConn, "'SRV.DWS.TEMP.BNKINFOD'", @ScriptDir & '\Daten1.txt', False, 0, $FTP_TRANSFER_TYPE_ASCII) & @CRLF)
EndIf
Local $iFtpc = _FTP_Close($hConn)
Local $iFtpo = _FTP_Close($hOpen)

in the file "downloaded" all looks fine, but the special chars do not: ü is } ß is ~ and so on.

 

 

Edited by Xenobiologist

Scripts & functions Organize Includes Let Scite organize the include files

Yahtzee The game "Yahtzee" (Kniffel, DiceLion)

LoginWrapper Secure scripts by adding a query (authentication)

_RunOnlyOnThis UDF Make sure that a script can only be executed on ... (Windows / HD / ...)

Internet-Café Server/Client Application Open CD, Start Browser, Lock remote client, etc.

MultipleFuncsWithOneHotkey Start different funcs by hitting one hotkey different times

Share this post


Link to post
Share on other sites

Did you try on another FTP Server to confirm that this is not related to "zoS MVS System (IBM)"
You can just use CesarFTP or FileZillaServer instaled on your own PC.

btw.
Please make some TXT test file zip them and post it here.

 


Signature beginning:   Wondering who uses AutoIT and what it can be used for ?
* GHAPI UDF - modest begining - comunication with GitHub REST API *
ADO.au3 UDF     POP3.au3 UDF     XML.au3 UDF    How to use IE.au3  UDF with  AutoIt v3.3.14.x  for other useful stuff click the following button

Spoiler

Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind. 

My contribution (my own projects): * Debenu Quick PDF Library - UDF * Debenu PDF Viewer SDK - UDF * Acrobat Reader - ActiveX Viewer * UDF for PDFCreator v1.x.x * XZip - UDF * AppCompatFlags UDF * CrowdinAPI UDF * _WinMergeCompare2Files() * _JavaExceptionAdd() * _IsBeta() * Writing DPI Awareness App - workaround * _AutoIt_RequiredVersion() * Chilkatsoft.au3 UDF * TeamViewer.au3 UDF * JavaManagement UDF * VIES over SOAP * WinSCP UDF * GHAPI UDF - modest begining - comunication with GitHub REST API *

My contribution to others projects or UDF based on  others projects: * _sql.au3 UDF  * POP3.au3 UDF *  RTF Printer - UDF * XML.au3 - BETA * ADO.au3 UDF SMTP Mailer UDF *

Useful links: * Forum Rules * Forum etiquette *  Forum Information and FAQs * How to post code on the forum * AutoIt Online Documentation * AutoIt Online Beta Documentation * SciTE4AutoIt3 getting started * Convert text blocks to AutoIt code * Games made in Autoit * Programming related sites * Polish AutoIt Tutorial * DllCall Code Generator * 

Wiki: Expand your knowledge - AutoIt Wiki * Collection of User Defined Functions * How to use HelpFile * Best coding practices * 

IE Related:  * How to use IE.au3  UDF with  AutoIt v3.3.14.x * Why isn't Autoit able to click a Javascript Dialog? * Clicking javascript button with no ID * IE document >> save as MHT file * IETab Switcher (by LarsJ ) * HTML Entities * _IEquerySelectorAll() (by uncommon) * 

I encourage you to read: * Global Vars * Best Coding Practices * Please explain code used in Help file for several File functions * OOP-like approach in AutoIt * UDF-Spec Questions *  EXAMPLE: How To Catch ConsoleWrite() output to a file or to CMD *

"Homo sum; humani nil a me alienum puto" - Publius Terentius Afer
"Program are meant to be read by humans and only incidentally for computers and execute" - Donald Knuth, "The Art of Computer Programming"
:naughty:  :ranting:, be  :) and       \\//_.

Anticipating Errors :  "Any program that accepts data from a user must include code to validate that data before sending it to the data store. You cannot rely on the data store, ...., or even your programming language to notify you of problems. You must check every byte entered by your users, making sure that data is the correct type for its field and that required fields are not empty."

Signature last update: 2017-06-04

Share this post


Link to post
Share on other sites

Here is a file.

I do not need to test it with another server. It should work with the one I test on.

Daten1.txt


Scripts & functions Organize Includes Let Scite organize the include files

Yahtzee The game "Yahtzee" (Kniffel, DiceLion)

LoginWrapper Secure scripts by adding a query (authentication)

_RunOnlyOnThis UDF Make sure that a script can only be executed on ... (Windows / HD / ...)

Internet-Café Server/Client Application Open CD, Start Browser, Lock remote client, etc.

MultipleFuncsWithOneHotkey Start different funcs by hitting one hotkey different times

Share this post


Link to post
Share on other sites

#6 ·  Posted (edited)

7 minutes ago, Xenobiologist said:

It should work with the one I test on.

I understand but, this is normal testing procedure. There is a need to eliminate all possible issues.

So you are talking that this: Daten1.ext was also tested by you and the problem occurs, when you get them via _FTP_FileGet(), I will do some test.

 

Edited by mLipok

Signature beginning:   Wondering who uses AutoIT and what it can be used for ?
* GHAPI UDF - modest begining - comunication with GitHub REST API *
ADO.au3 UDF     POP3.au3 UDF     XML.au3 UDF    How to use IE.au3  UDF with  AutoIt v3.3.14.x  for other useful stuff click the following button

Spoiler

Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind. 

My contribution (my own projects): * Debenu Quick PDF Library - UDF * Debenu PDF Viewer SDK - UDF * Acrobat Reader - ActiveX Viewer * UDF for PDFCreator v1.x.x * XZip - UDF * AppCompatFlags UDF * CrowdinAPI UDF * _WinMergeCompare2Files() * _JavaExceptionAdd() * _IsBeta() * Writing DPI Awareness App - workaround * _AutoIt_RequiredVersion() * Chilkatsoft.au3 UDF * TeamViewer.au3 UDF * JavaManagement UDF * VIES over SOAP * WinSCP UDF * GHAPI UDF - modest begining - comunication with GitHub REST API *

My contribution to others projects or UDF based on  others projects: * _sql.au3 UDF  * POP3.au3 UDF *  RTF Printer - UDF * XML.au3 - BETA * ADO.au3 UDF SMTP Mailer UDF *

Useful links: * Forum Rules * Forum etiquette *  Forum Information and FAQs * How to post code on the forum * AutoIt Online Documentation * AutoIt Online Beta Documentation * SciTE4AutoIt3 getting started * Convert text blocks to AutoIt code * Games made in Autoit * Programming related sites * Polish AutoIt Tutorial * DllCall Code Generator * 

Wiki: Expand your knowledge - AutoIt Wiki * Collection of User Defined Functions * How to use HelpFile * Best coding practices * 

IE Related:  * How to use IE.au3  UDF with  AutoIt v3.3.14.x * Why isn't Autoit able to click a Javascript Dialog? * Clicking javascript button with no ID * IE document >> save as MHT file * IETab Switcher (by LarsJ ) * HTML Entities * _IEquerySelectorAll() (by uncommon) * 

I encourage you to read: * Global Vars * Best Coding Practices * Please explain code used in Help file for several File functions * OOP-like approach in AutoIt * UDF-Spec Questions *  EXAMPLE: How To Catch ConsoleWrite() output to a file or to CMD *

"Homo sum; humani nil a me alienum puto" - Publius Terentius Afer
"Program are meant to be read by humans and only incidentally for computers and execute" - Donald Knuth, "The Art of Computer Programming"
:naughty:  :ranting:, be  :) and       \\//_.

Anticipating Errors :  "Any program that accepts data from a user must include code to validate that data before sending it to the data store. You cannot rely on the data store, ...., or even your programming language to notify you of problems. You must check every byte entered by your users, making sure that data is the correct type for its field and that required fields are not empty."

Signature last update: 2017-06-04

Share this post


Link to post
Share on other sites

#7 ·  Posted (edited)

Yes exactly, I got the file via _FTP_FileGet from the mainframe. On my windows PC the special chars are not displayed corretly in notepad++ or ultraedit or whatever.

I thought, that I have to set the codepage before using the FileGet like I did with the ftp command in a dosprompt.

Maybe I'm wrong and there is no other chance then using a StringReplace for every special char afterwards.

Edited by Xenobiologist

Scripts & functions Organize Includes Let Scite organize the include files

Yahtzee The game "Yahtzee" (Kniffel, DiceLion)

LoginWrapper Secure scripts by adding a query (authentication)

_RunOnlyOnThis UDF Make sure that a script can only be executed on ... (Windows / HD / ...)

Internet-Café Server/Client Application Open CD, Start Browser, Lock remote client, etc.

MultipleFuncsWithOneHotkey Start different funcs by hitting one hotkey different times

Share this post


Link to post
Share on other sites

In the $iFlags try setting it to $FTP_TRANSFER_TYPE_BINARY instead of the default.


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites

#9 ·  Posted (edited)

11 minutes ago, BrewManNH said:

In the $iFlags try setting it to $FTP_TRANSFER_TYPE_BINARY instead of the default.

I did. That doesn't help. The file on the mainfraime is a normal text file. Transfering it with binary_mode creates a file like this:

 

ÇÁÄÕÙ^ÒäÙéÕÁÔÅ^ÂÁÕÒÕÁÔÅ^×Óé^ÖÙã^ãÅÉÓÖÙã^âãÙÁââÅ^ãÅÓÅÆÖÕ^ãÅÓÅÆÁç^ÒÖÕãÁÒã^ÂÕÒÂÅÙ^ÂÕÒÂÅÙäÉ^ÂÓé^ÒÙÅÉâ^ãÖäÙÉÔâ^Èãã×^ÕÙÂâÖÙÄé^ãÄÂâ^Áâðôð^Áâðóð^Áâðòð^Áâðñð^×éÂÉÓñð^×éÂÉÓñö^×éÇäåðñ^×éÂÉÓòó^ãÄÁÕÇð÷^ÁéÁÕÇðö^ÁéÁÕÇñò^ÁéÁÕÇñô^×éÇäåðó^×éÇäåðø^×éÇäåñð^×éÇäåñõ^×éÇäåñö^×éÇäåñ÷^ÕÙåÅÙÂ^æÇéÕÙ^åÖÙâãñ^åÖÙâãò^åÖÙâãó^åÖÙâãô^åÖÙâãõ^åÖÙâãö^åÖÙâã÷^åÖÙâãø^åÖÙâãù^×Óé×Æ^×ÖâãÆÁÃÈ^ÇÅÕÖÉÄ^éÅÕãÙÁÓÂÁÕÒ^åÅÙÂÁÕÄ^ÇæÔÓÉé^âÓÓÉé^ÁÙåÖÙâ^ÂÂó^@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

Edited by Xenobiologist

Scripts & functions Organize Includes Let Scite organize the include files

Yahtzee The game "Yahtzee" (Kniffel, DiceLion)

LoginWrapper Secure scripts by adding a query (authentication)

_RunOnlyOnThis UDF Make sure that a script can only be executed on ... (Windows / HD / ...)

Internet-Café Server/Client Application Open CD, Start Browser, Lock remote client, etc.

MultipleFuncsWithOneHotkey Start different funcs by hitting one hotkey different times

Share this post


Link to post
Share on other sites

I just noticed you're using the ASCII flag in the FTP_Open as well, should probably change it there as well
.


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites

#11 ·  Posted (edited)

The solution is to get the text as binary and use Windows primitives to translate between EBCDIC (which one BTW, there are so many of versions!) and native AutoIt (Unicode).

Func _CodepageToStr($sCP, $iCodepage = Default)
    If $iCodepage = Default Then $iCodepage = Int(RegRead("HKLM\SYSTEM\CurrentControlSet\Control\Nls\Codepage", "OEMCP"))
    Local $tText = DllStructCreate("byte[" & BinaryLen($sCP) & "]")
    DllStructSetData($tText, 1, $sCP)
    Local $aResult = DllCall("kernel32.dll", "int", "MultiByteToWideChar", "uint", $iCodepage, "dword", 0, "struct*", $tText, "int", BinaryLen($sCP), _
            "ptr", 0, "int", 0)
    Local $tWstr = DllStructCreate("wchar[" & $aResult[0] & "]")
    $aResult = DllCall("kernel32.dll", "int", "MultiByteToWideChar", "uint", $iCodepage, "dword", 0, "struct*", $tText, "int", BinaryLen($sCP), _
            "struct*", $tWstr, "int", $aResult[0])
    Return DllStructGetData($tWstr, 1)
EndFunc   ;==>_CodepageToStr

It's now up to you to select the correct EBCDIC codepage identifier from here.

Edited by jchd

This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

#12 ·  Posted (edited)

I tried all four combinations.

_FTP_Connect with $INTERNET_FLAG_TRANSFER_ASCII, $INTERNET_FLAG_TRANSFER_BINARY

_FTP_FileGet with $FTP_TRANSFER_TYPE_ASCII, $FTP_TRANSFER_TYPE_BINARY

Edited by Xenobiologist

Scripts & functions Organize Includes Let Scite organize the include files

Yahtzee The game "Yahtzee" (Kniffel, DiceLion)

LoginWrapper Secure scripts by adding a query (authentication)

_RunOnlyOnThis UDF Make sure that a script can only be executed on ... (Windows / HD / ...)

Internet-Café Server/Client Application Open CD, Start Browser, Lock remote client, etc.

MultipleFuncsWithOneHotkey Start different funcs by hitting one hotkey different times

Share this post


Link to post
Share on other sites

Use Binary mode instead.


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

Please note that the example file Daten1.txt is not EBCDIC at all, as a quick look on it's hex content immediately shows.

Nonetheless you'll have to determine which 8-bit charset the zOS machine uses for this file and pass the corresponding identifier to the function posted.


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

Thanks, I tried this:

 

#include <Array.au3>
#include <FTPEx.au3>
#include <MsgBoxConstants.au3>

Local $sServer = 'etet' ; zoS MVS System (IBM)
Local $sUsername = @UserName
Local $sPass = 'xxx'

Local $hOpen = _FTP_Open('MyFTP Control')
;~ Local $hConn = _FTP_Connect($hOpen, $sServer, $sUsername, $sPass, 0, 21, $INTERNET_SERVICE_FTP, $INTERNET_FLAG_TRANSFER_ASCII)
Local $hConn = _FTP_Connect($hOpen, $sServer, $sUsername, $sPass, 0, 21, $INTERNET_SERVICE_FTP, $INTERNET_FLAG_TRANSFER_BINARY)
If @error Then
    MsgBox($MB_SYSTEMMODAL, '_FTP_Connect', 'ERROR=' & @error)
Else
;~  ConsoleWrite(_FTP_Command($hConn, 'quote site sbd=(IBM-273,iso8859-1)') & @CRLF)
;~  ConsoleWrite(_FTP_FileGet($hConn, "'SRV.DWS.TEMP.BNKINFOD'", @ScriptDir & '\Daten1.txt', False, 0, $FTP_TRANSFER_TYPE_ASCII) & @CRLF)
    ConsoleWrite(_FTP_FileGet($hConn, "'SRV.DWS.TEMP.BNKINFOD'", @ScriptDir & '\Daten1.txt', False, 0, $FTP_TRANSFER_TYPE_BINARY) & @CRLF)
EndIf
Local $iFtpc = _FTP_Close($hConn)
Local $iFtpo = _FTP_Close($hOpen)

$c = FileRead(@ScriptDir & '\Daten1.txt')

FileWrite(@ScriptDir & '\Daten2.txt', _CodepageToStr($c, 1141) & @CRLF)

Func _CodepageToStr($sCP, $iCodepage = Default)
    If $iCodepage = Default Then $iCodepage = Int(RegRead("HKLM\SYSTEM\CurrentControlSet\Control\Nls\Codepage", "OEMCP"))
    Local $tText = DllStructCreate("byte[" & BinaryLen($sCP) & "]")
    DllStructSetData($tText, 1, $sCP)
    Local $aResult = DllCall("kernel32.dll", "int", "MultiByteToWideChar", "uint", $iCodepage, "dword", 0, "struct*", $tText, "int", BinaryLen($sCP), _
            "ptr", 0, "int", 0)
    Local $tWstr = DllStructCreate("wchar[" & $aResult[0] & "]")
    $aResult = DllCall("kernel32.dll", "int", "MultiByteToWideChar", "uint", $iCodepage, "dword", 0, "struct*", $tText, "int", BinaryLen($sCP), _
            "struct*", $tWstr, "int", $aResult[0])
    Return DllStructGetData($tWstr, 1)
EndFunc   ;==>_CodepageToStr

This way I get a second file Daten2.txt which now shows öäüß corretly. The only problem I've to figure out is how to insert a CRLF back into the file. The file contains currently only one long line.


Scripts & functions Organize Includes Let Scite organize the include files

Yahtzee The game "Yahtzee" (Kniffel, DiceLion)

LoginWrapper Secure scripts by adding a query (authentication)

_RunOnlyOnThis UDF Make sure that a script can only be executed on ... (Windows / HD / ...)

Internet-Café Server/Client Application Open CD, Start Browser, Lock remote client, etc.

MultipleFuncsWithOneHotkey Start different funcs by hitting one hotkey different times

Share this post


Link to post
Share on other sites

#16 ·  Posted (edited)

There is already a Windows-style line termination at the end of the Daten1.txt file (0x0D 0x0A).

Can you post the binary data as received (written as binary file)?

Edited by jchd

This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

Good morning,

#include <Array.au3>
#include <FTPEx.au3>
#include <MsgBoxConstants.au3>

Local $sServer = 'etet' ; zoS MVS System (IBM)
Local $sUsername = @UserName
Local $sPass = 'xxx'

Local $hOpen = _FTP_Open('MyFTP Control')
;~ Local $hConn = _FTP_Connect($hOpen, $sServer, $sUsername, $sPass, 0, 21, $INTERNET_SERVICE_FTP, $INTERNET_FLAG_TRANSFER_ASCII)
Local $hConn = _FTP_Connect($hOpen, $sServer, $sUsername, $sPass, 0, 21, $INTERNET_SERVICE_FTP, $INTERNET_FLAG_TRANSFER_BINARY)
If @error Then
    MsgBox($MB_SYSTEMMODAL, '_FTP_Connect', 'ERROR=' & @error)
Else
;~  ConsoleWrite(_FTP_Command($hConn, 'quote site sbd=(IBM-273,iso8859-1)') & @CRLF)
;~  ConsoleWrite(_FTP_FileGet($hConn, "'SRV.DWS.TEMP.BNKINFOD'", @ScriptDir & '\Daten1.txt', False, 0, $FTP_TRANSFER_TYPE_ASCII) & @CRLF)
    ConsoleWrite(_FTP_FileGet($hConn, "'SRV.DWS.TEMP.BNKINFOD'", @ScriptDir & '\Daten1.txt', False, 0, $FTP_TRANSFER_TYPE_BINARY) & @CRLF)
EndIf
Local $iFtpc = _FTP_Close($hConn)
Local $iFtpo = _FTP_Close($hOpen)

$c = FileRead(@ScriptDir & '\Daten1.txt')
FileDelete(@ScriptDir & '\Daten3.txt')
FileWrite(@ScriptDir & '\Daten3.txt' , StringStripWS(StringReplace(_CodepageToStr($c, 1141), ';      ', ';' & @CRLF), 7) & @CRLF)

Func _CodepageToStr($sCP, $iCodepage = Default)
    If $iCodepage = Default Then $iCodepage = Int(RegRead("HKLM\SYSTEM\CurrentControlSet\Control\Nls\Codepage", "OEMCP"))
    Local $tText = DllStructCreate("byte[" & BinaryLen($sCP) & "]")
    DllStructSetData($tText, 1, $sCP)
    Local $aResult = DllCall("kernel32.dll", "int", "MultiByteToWideChar", "uint", $iCodepage, "dword", 0, "struct*", $tText, "int", BinaryLen($sCP), _
            "ptr", 0, "int", 0)
    Local $tWstr = DllStructCreate("wchar[" & $aResult[0] & "]")
    $aResult = DllCall("kernel32.dll", "int", "MultiByteToWideChar", "uint", $iCodepage, "dword", 0, "struct*", $tText, "int", BinaryLen($sCP), _
            "struct*", $tWstr, "int", $aResult[0])
    Return DllStructGetData($tWstr, 1)
EndFunc   ;==>_CodepageToStr

This way the file looks the way it should. Of course it is not the best way to add the CR but hey, it works :-)

Thanks to all!

Mega


Scripts & functions Organize Includes Let Scite organize the include files

Yahtzee The game "Yahtzee" (Kniffel, DiceLion)

LoginWrapper Secure scripts by adding a query (authentication)

_RunOnlyOnThis UDF Make sure that a script can only be executed on ... (Windows / HD / ...)

Internet-Café Server/Client Application Open CD, Start Browser, Lock remote client, etc.

MultipleFuncsWithOneHotkey Start different funcs by hitting one hotkey different times

Share this post


Link to post
Share on other sites

#18 ·  Posted (edited)

Still interested in how to set a codepage before using FileGet or FilePut.

Without setting it I always need to manipulate the files before transfering them.

Edited by Xenobiologist

Scripts & functions Organize Includes Let Scite organize the include files

Yahtzee The game "Yahtzee" (Kniffel, DiceLion)

LoginWrapper Secure scripts by adding a query (authentication)

_RunOnlyOnThis UDF Make sure that a script can only be executed on ... (Windows / HD / ...)

Internet-Café Server/Client Application Open CD, Start Browser, Lock remote client, etc.

MultipleFuncsWithOneHotkey Start different funcs by hitting one hotkey different times

Share this post


Link to post
Share on other sites

#19 ·  Posted (edited)

AFAIK FTP protocol has only two modes: text or nibary (Oops, binary!). I.e. it has no knowledge or say about the content of what's being got/put. Such "decoration" is higher in the ISO model, so belongs to code processing things before put or after get.

Edited by jchd

This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

Then what is this?

When I use the ftp.exe from C:\Windows\System32 in a DosPrompt.

ftp etet

Type in user and password

Then it looks like this

ftp>

Now I can type in this: ftp> quote site sbd=(IBM-273,iso8859-1)
and get this --> 200 SITE command was accepted

After that the transfer is managed correctly. The umlauts etc. are shown correctly.

It seems the command tells the server to act different.


Scripts & functions Organize Includes Let Scite organize the include files

Yahtzee The game "Yahtzee" (Kniffel, DiceLion)

LoginWrapper Secure scripts by adding a query (authentication)

_RunOnlyOnThis UDF Make sure that a script can only be executed on ... (Windows / HD / ...)

Internet-Café Server/Client Application Open CD, Start Browser, Lock remote client, etc.

MultipleFuncsWithOneHotkey Start different funcs by hitting one hotkey different times

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now

  • Similar Content

    • Xandy
      By Xandy
      Screenshot program that can upload/FTP to website/storage with hotkey.
      Features GUI to display programmable keys.

      Set the hotkeys with this function to use, and display hotkeys.
      ; hotkey_set() Parameters: ; ----------------------------------------------------------- ; $aHotkey -                The array hotkeys are stored in. This function sets the values of this array[hotkey_id][$hotkey_data] ; $dHotkey_id -             The enum for this hotkey index ; $sHotkey_description -     The label displayed to the user to represent function of hotkey ; $sHotkey_name -            The hotkey it'self.  Whatever name string you want to give the key ; $dHotkey_key -             The _IsPressed keycode ;    Keycodes AT: https://www.autoitscript.com/autoit3/docs/libfunctions/_IsPressed.htm ; $dHotkey_shift -             Shift flag 0 or 1 Default off ; $dHotkey_ctrl -             Ctrl flag 0 or 1 Default off ; $dHotkey_alt -             Alt flag 0 or 1 Default off Func hotkey_set(ByRef $aHotkey, $dHotkey_id, $sHotkey_description, $sHotkey_name, $dHotkey_key, $dHotkey_shift = 0, $dHotkey_ctrl = 0, $dHotkey_alt = 0)     $aHotkey[$dHotkey_id][$eHotkey_data_key_description] = $sHotkey_description     $aHotkey[$dHotkey_id][$eHotkey_data_key_name] = $sHotkey_name     $aHotkey[$dHotkey_id][$eHotkey_data_key] = $dHotkey_key     $aHotkey[$dHotkey_id][$eHotkey_data_shift] = $dHotkey_shift     $aHotkey[$dHotkey_id][$eHotkey_data_ctrl] = $dHotkey_ctrl     $aHotkey[$dHotkey_id][$eHotkey_data_alt] = $dHotkey_alt EndFunc   ;==>hotkey_set ; You can set your hotkeys here ; Please visit the hotkey_set() function for parameter information hotkey_set($aHotkey, $eHotkey_screenshot_ftp, "Selected Window to FTP", "F12", "7B", 0, 1, 0); F12 hotkey_set($aHotkey, $eHotkey_screenshot_disk, "Selected Window to Disk", "S", "53", 1, 1, 1); S hotkey_set($aHotkey, $eHotkey_clipboard_send, "Send Clipboard keystrokes", "F10", "79", 1, 1, 1); F10  
      Configure settings dialog:

      Screenshot Filename and Screenshot Counter, are used to create simple unique filenames that can cycle. Copy URL to clipboard option. - For linking your screenshots. The screenshot file type is for local copy only. App always uses .JPG for FTP right now, but I could add FTP screenshot file type specification.  
      Any suggestions?  Did I break anything, what did I miss?
      Package uses TTS.au3 by Beege: 
       
      FTP_Screen.zip
      File includes:
         - FTP_Screen.au3
         - FTPScreen.ico
         - TTS.au3 - by Beege
    • drorshem
      By drorshem
      Hi,
      I'm looking to create a script which will download all *.txt files from a remote FTP server.
      Once downloaded, delete all *.txt files.
       
      Can this be achieved using AutoIt ?
    • Jefrey
      By Jefrey
      Non-important short long story: I was facing several troubles when working with a webservice made in PHP and an AutoIt client. The AutoIt client had to send some data to the PHP webservice, then it would get back part of the data (with additional data) back to AutoIt, then send through Json to another webservice and end up in a TV system (huff). As I was using mostly Inet* functions, working with charsets became complicated, as it had to encode, decode, encode... and the mess is done. On the other end I was receiving a completely buggy string with several problems in accentuated characters (note that I live in Brazil, we speak portuguese that contains a lot of accented chars - Ááãç...). I tried and tried mixing utf_encodes and decodes everywhere, in PHP and Javascript. Wouldn't it be easier if I could just force the string to be UTF-8 and screw everything else? So I found toUTF8() PHP function.
      I've ported toUTF8() function (truly, the whole Encoding class) by Sebastián Grignoli to AutoIt. It offers useful functions to force a string to be in a specified charset in a really easy way.
      From the readme file:
      Usage
      $utf8_string = toUTF8($utf8_or_latin1_or_mixed_string) $latin1_string = toLatin1($utf8_or_latin1_or_mixed_string) Also:
      $utf8_string = fixUTF8($garbled_utf8_string) fixUTF8() converts the string to UTF-8 repeatedly until make sure it has only UTF-8 valid chars (it's really UTF-8).
      Example:
      #include 'forceutf8.au3' MsgBox(0, '', fixUTF8( 'ãé' ) ) Will output:
      ãé Note that it's just a port. If you look at both the source codes together (PHP and AutoIt), you'll see that they're exactly the same thing, but in different approaches (PHP arrays converted to Scripting.Dictionary objects, function renames, syntax porting, a few functions completely rewritten due to differences between PHP and AutoIt). Therefore, all credits goes to Sebastián Grignol.
      It seems that it works only with latin/roman alphabet (used by English).
      Downloads
      Download ZIP from Github
      Wanna help?
      Fork me on Github
    • DCCD
      By DCCD
      Hi, how to get full error reporting if FTP failed?
       
      tried  _WinAPI_GetLastErrorMessage() func but That didn't help at all.  the information was returned by _WinAPI_GetLastErrorMessage() can't help me!
       
       
      _FTP_Open, return Success: a handle, Failure: 0 and sets the @error flag to non-zero
      _FTP_Connect,  return Success: an handle to connected session. Failure: 0 and sets the @error flag to non-zero.
      _FTP_DirCreate, return Success: 1. Failure: 0.
      _FTP_FilePut, return Success: 1. Failure: 0 and sets the @error flag to non-zero.
       
      Any help would be much appreciated
       
       
       
       
    • Elia
      By Elia
      Hi All,
      I need some help, I am writing a script that FTPs some files and then outputs the results, but I am unable to figure out how to output the file name on line 58. Here is the code (modified to exclude personal information):
       
      #include <FTPEx.au3> #include <MsgBoxConstants.au3> #include <WinAPIFiles.au3> #include <Inet.au3> #include <Debug.au3> #include <File.au3> #include <Array.au3> Local $Server = 'ftp.com' Local $Username = 'user' Local $Password = 'password' Local $LocalFolder = 'C:\Folder' Local $RemoteFolder = '/users/user1' Local $SMTPServer = "Exchange.domain.com" Local $FromName = "FTP" Local $FromAddress = "email@address.com" Local $ToAddress = " email@address.com " Local $Subject = "FTP Transfer Failed, please check server" Local $Subject1 = "FTP Connection Failed, please check server" Local $Body[2] $Body[0] = "Line 1" $Body[1] = "Line 2" Local $Body1[2] $Body1[0] = "Line 1" $Body1[1] = "Line 2" $LogFolder = "\\COMP\LOGS"   _DebugSetup ("Debug", True)   FileMove ("\\COMP\Test\*.txt", "\\COMP\FTP")   Local $FolderContents = _FileListToArray("\\COMP\FTP") Local $LogFile = "\FilesToSend_" & @YEAR & @MON & @MDAY & @HOUR & @MIN & @SEC & ".log" _FileCreate($LogFolder & $LogFile) _FileWriteFromArray($LogFolder & $LogFile, $FolderContents)   If FileExists ( "\\COMP\FTP" ) Then     Local $Open = _FTP_Open('FTP')     Local $CallBack = _FTP_SetStatusCallback($Open, 'FTPStatusCallbackHandler')     Local $Connect = _FTP_Connect($Open, $Server, $Username, $Password, 0, $INTERNET_DEFAULT_FTP_PORT, $INTERNET_SERVICE_FTP, 0, $CallBack)         If $Connect <> 0 Then             Local $Transfer = _FTP_DirPutContents($Connect, $LocalFolder, $RemoteFolder, 0)             _FTP_Close($Connect)             _FTP_Close($Open)                 Else             _INetSmtpMail($SMTPServer, $FromName, $FromAddress, $ToAddress, $Subject1, $Body, "EHLO" & @ComputerName, -1)         EndIf             If $Transfer = 1 Then                 FileMove ( "\\COMP\FTP\*.txt", "\\COMP\Test\SENT")                 Else                 _INetSmtpMail($SMTPServer, $FromName, $FromAddress, $ToAddress, $Subject, $Body1, "EHLO" & @ComputerName, -1)             EndIf EndIf Func FTPStatusCallbackHandler($hInternet, $iContext, $iInternetStatus, $pStatusInformation, $iStatusInformationLength)                     #forceref $hInternet, $iContext                     If $iInternetStatus = $INTERNET_STATUS_REQUEST_SENT or $iInternetStatus = $INTERNET_STATUS_RESPONSE_RECEIVED Then                         Local $iBytesRead                         Local $tStatus = DllStructCreate('dword')                         _WinAPI_ReadProcessMemory(_WinAPI_GetCurrentProcess(), $pStatusInformation, $tStatus, $iStatusInformationLength, $iBytesRead)                         _DebugOut(_FTP_DecodeInternetStatus($iInternetStatus) & ' | Size = ' & DllStructGetData($tStatus, 1) & ' Bytes    Bytes read = ' & $iBytesRead)                     Else                         _DebugOut(_FTP_DecodeInternetStatus($iInternetStatus))                     EndIf                     EndFunc I would like the output on line 58 to include a file name, at the moment it only displays the size of the file and Bytes read. Here is what is currently outputted:
      Handle created
      Resolving name ...
      Name resolved
      Connecting to server ...
      Connected to server
      Receiving response ...
      Response received | Size = 58 Bytes    Bytes read = 4
      Sending request ...
      Request sent | Size = 13 Bytes    Bytes read = 4
      Receiving response ...
      Response received | Size = 20 Bytes    Bytes read = 4
      Sending request ...
      Request sent | Size = 19 Bytes    Bytes read = 4
      Receiving response ...
      Response received | Size = 20 Bytes    Bytes read = 4
      Handle created
      Handle created
      Handle created
      Handle created
      Handle created
      Closing connection ...
      Connection closed
      Handle closing ...
      Any help would be greatly appreciated!
      Thanks.