Xenobiologist

_FTP_... functions use codepage (zOS) (solved)

25 posts in this topic

#1 ·  Posted (edited)

Hi folks,

I'm using the udf #include <FTPEx.au3> to "download" some files from our zOS (MVS) system via autoit FTP with _FTP_FileGet.

The FileGet works, but the codepage is wrong. Special German chars like öäüß are not transfered correctly.

Any idea how to set the codepage? Swichting between binary or ascii doesn't solve the problem.

Using the ftp command in a cmd.exe window it works like this : quote site sbd=(IBM-273,iso8859-1)

Thanks

Mega

Edited by Xenobiologist

Scripts & functions Organize Includes Let Scite organize the include files

Yahtzee The game "Yahtzee" (Kniffel, DiceLion)

LoginWrapper Secure scripts by adding a query (authentication)

_RunOnlyOnThis UDF Make sure that a script can only be executed on ... (Windows / HD / ...)

Internet-Café Server/Client Application Open CD, Start Browser, Lock remote client, etc.

MultipleFuncsWithOneHotkey Start different funcs by hitting one hotkey different times

Share this post


Link to post
Share on other sites



Please make any testing file (the best will be simple txt file) zip !!! them and post this ZIP archive here..
 


Signature beginning:   Wondering who uses AutoIT and what it can be used for ?
* GHAPI UDF - modest begining - comunication with GitHub REST API *
ADO.au3 UDF     POP3.au3 UDF     XML.au3 UDF    How to use IE.au3  UDF with  AutoIt v3.3.14.x  for other useful stuff click the following button

Spoiler

Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind. 

My contribution (my own projects): * Debenu Quick PDF Library - UDF * Debenu PDF Viewer SDK - UDF * Acrobat Reader - ActiveX Viewer * UDF for PDFCreator v1.x.x * XZip - UDF * AppCompatFlags UDF * CrowdinAPI UDF * _WinMergeCompare2Files() * _JavaExceptionAdd() * _IsBeta() * Writing DPI Awareness App - workaround * _AutoIt_RequiredVersion() * Chilkatsoft.au3 UDF * TeamViewer.au3 UDF * JavaManagement UDF * VIES over SOAP * WinSCP UDF * GHAPI UDF - modest begining - comunication with GitHub REST API *

My contribution to others projects or UDF based on  others projects: * _sql.au3 UDF  * POP3.au3 UDF *  RTF Printer - UDF * XML.au3 - BETA * ADO.au3 UDF SMTP Mailer UDF *

Useful links: * Forum Rules * Forum etiquette *  Forum Information and FAQs * How to post code on the forum * AutoIt Online Documentation * AutoIt Online Beta Documentation * SciTE4AutoIt3 getting started * Convert text blocks to AutoIt code * Games made in Autoit * Programming related sites * Polish AutoIt Tutorial * DllCall Code Generator * 

Wiki: Expand your knowledge - AutoIt Wiki * Collection of User Defined Functions * How to use HelpFile * Best coding practices * 

IE Related:  * How to use IE.au3  UDF with  AutoIt v3.3.14.x * Why isn't Autoit able to click a Javascript Dialog? * Clicking javascript button with no ID * IE document >> save as MHT file * IETab Switcher (by LarsJ ) * HTML Entities * _IEquerySelectorAll() (by uncommon) * 

I encourage you to read: * Global Vars * Best Coding Practices * Please explain code used in Help file for several File functions * OOP-like approach in AutoIt * UDF-Spec Questions *  EXAMPLE: How To Catch ConsoleWrite() output to a file or to CMD *

"Homo sum; humani nil a me alienum puto" - Publius Terentius Afer
"Program are meant to be read by humans and only incidentally for computers and execute" - Donald Knuth, "The Art of Computer Programming"
:naughty:  :ranting:, be  :) and       \\//_.

Anticipating Errors :  "Any program that accepts data from a user must include code to validate that data before sending it to the data store. You cannot rely on the data store, ...., or even your programming language to notify you of problems. You must check every byte entered by your users, making sure that data is the correct type for its field and that required fields are not empty."

Signature last update: 2017-06-04

Share this post


Link to post
Share on other sites

#3 ·  Posted (edited)

I don't know how you can test this. This is a FTP download from a zOS system.  Use a file which is in codepage EBCDIC with some öäüß and try to download it via FTPGet.

I'm using somethig like this:

#include <Array.au3>
#include <FTPEx.au3>
#include <MsgBoxConstants.au3>
#include <Process.au3>

Local $sServer = 'etet' ; zoS MVS System (IBM)
Local $sUsername = @UserName
Local $sPass = 'xxx'

Local $hOpen = _FTP_Open('MyFTP Control')
Local $hConn = _FTP_Connect($hOpen, $sServer, $sUsername, $sPass, 0, 21, $INTERNET_SERVICE_FTP, $INTERNET_FLAG_TRANSFER_ASCII)
;~ Local $hConn = _FTP_Connect($hOpen, $sServer, $sUsername, $sPass, 0, 21, $INTERNET_SERVICE_FTP, $INTERNET_FLAG_TRANSFER_BINARY)
If @error Then
    MsgBox($MB_SYSTEMMODAL, '_FTP_Connect', 'ERROR=' & @error)
Else
;~  ConsoleWrite(_FTP_Command($hConn, 'quote site sbd=(IBM-273,iso8859-1)') & @CRLF)
    ConsoleWrite(_FTP_FileGet($hConn, "'SRV.DWS.TEMP.BNKINFOD'", @ScriptDir & '\Daten1.txt', False, 0, $FTP_TRANSFER_TYPE_ASCII) & @CRLF)
EndIf
Local $iFtpc = _FTP_Close($hConn)
Local $iFtpo = _FTP_Close($hOpen)

in the file "downloaded" all looks fine, but the special chars do not: ü is } ß is ~ and so on.

 

 

Edited by Xenobiologist

Scripts & functions Organize Includes Let Scite organize the include files

Yahtzee The game "Yahtzee" (Kniffel, DiceLion)

LoginWrapper Secure scripts by adding a query (authentication)

_RunOnlyOnThis UDF Make sure that a script can only be executed on ... (Windows / HD / ...)

Internet-Café Server/Client Application Open CD, Start Browser, Lock remote client, etc.

MultipleFuncsWithOneHotkey Start different funcs by hitting one hotkey different times

Share this post


Link to post
Share on other sites

Did you try on another FTP Server to confirm that this is not related to "zoS MVS System (IBM)"
You can just use CesarFTP or FileZillaServer instaled on your own PC.

btw.
Please make some TXT test file zip them and post it here.

 


Signature beginning:   Wondering who uses AutoIT and what it can be used for ?
* GHAPI UDF - modest begining - comunication with GitHub REST API *
ADO.au3 UDF     POP3.au3 UDF     XML.au3 UDF    How to use IE.au3  UDF with  AutoIt v3.3.14.x  for other useful stuff click the following button

Spoiler

Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind. 

My contribution (my own projects): * Debenu Quick PDF Library - UDF * Debenu PDF Viewer SDK - UDF * Acrobat Reader - ActiveX Viewer * UDF for PDFCreator v1.x.x * XZip - UDF * AppCompatFlags UDF * CrowdinAPI UDF * _WinMergeCompare2Files() * _JavaExceptionAdd() * _IsBeta() * Writing DPI Awareness App - workaround * _AutoIt_RequiredVersion() * Chilkatsoft.au3 UDF * TeamViewer.au3 UDF * JavaManagement UDF * VIES over SOAP * WinSCP UDF * GHAPI UDF - modest begining - comunication with GitHub REST API *

My contribution to others projects or UDF based on  others projects: * _sql.au3 UDF  * POP3.au3 UDF *  RTF Printer - UDF * XML.au3 - BETA * ADO.au3 UDF SMTP Mailer UDF *

Useful links: * Forum Rules * Forum etiquette *  Forum Information and FAQs * How to post code on the forum * AutoIt Online Documentation * AutoIt Online Beta Documentation * SciTE4AutoIt3 getting started * Convert text blocks to AutoIt code * Games made in Autoit * Programming related sites * Polish AutoIt Tutorial * DllCall Code Generator * 

Wiki: Expand your knowledge - AutoIt Wiki * Collection of User Defined Functions * How to use HelpFile * Best coding practices * 

IE Related:  * How to use IE.au3  UDF with  AutoIt v3.3.14.x * Why isn't Autoit able to click a Javascript Dialog? * Clicking javascript button with no ID * IE document >> save as MHT file * IETab Switcher (by LarsJ ) * HTML Entities * _IEquerySelectorAll() (by uncommon) * 

I encourage you to read: * Global Vars * Best Coding Practices * Please explain code used in Help file for several File functions * OOP-like approach in AutoIt * UDF-Spec Questions *  EXAMPLE: How To Catch ConsoleWrite() output to a file or to CMD *

"Homo sum; humani nil a me alienum puto" - Publius Terentius Afer
"Program are meant to be read by humans and only incidentally for computers and execute" - Donald Knuth, "The Art of Computer Programming"
:naughty:  :ranting:, be  :) and       \\//_.

Anticipating Errors :  "Any program that accepts data from a user must include code to validate that data before sending it to the data store. You cannot rely on the data store, ...., or even your programming language to notify you of problems. You must check every byte entered by your users, making sure that data is the correct type for its field and that required fields are not empty."

Signature last update: 2017-06-04

Share this post


Link to post
Share on other sites

Here is a file.

I do not need to test it with another server. It should work with the one I test on.

Daten1.txt


Scripts & functions Organize Includes Let Scite organize the include files

Yahtzee The game "Yahtzee" (Kniffel, DiceLion)

LoginWrapper Secure scripts by adding a query (authentication)

_RunOnlyOnThis UDF Make sure that a script can only be executed on ... (Windows / HD / ...)

Internet-Café Server/Client Application Open CD, Start Browser, Lock remote client, etc.

MultipleFuncsWithOneHotkey Start different funcs by hitting one hotkey different times

Share this post


Link to post
Share on other sites

#6 ·  Posted (edited)

7 minutes ago, Xenobiologist said:

It should work with the one I test on.

I understand but, this is normal testing procedure. There is a need to eliminate all possible issues.

So you are talking that this: Daten1.ext was also tested by you and the problem occurs, when you get them via _FTP_FileGet(), I will do some test.

 

Edited by mLipok

Signature beginning:   Wondering who uses AutoIT and what it can be used for ?
* GHAPI UDF - modest begining - comunication with GitHub REST API *
ADO.au3 UDF     POP3.au3 UDF     XML.au3 UDF    How to use IE.au3  UDF with  AutoIt v3.3.14.x  for other useful stuff click the following button

Spoiler

Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind. 

My contribution (my own projects): * Debenu Quick PDF Library - UDF * Debenu PDF Viewer SDK - UDF * Acrobat Reader - ActiveX Viewer * UDF for PDFCreator v1.x.x * XZip - UDF * AppCompatFlags UDF * CrowdinAPI UDF * _WinMergeCompare2Files() * _JavaExceptionAdd() * _IsBeta() * Writing DPI Awareness App - workaround * _AutoIt_RequiredVersion() * Chilkatsoft.au3 UDF * TeamViewer.au3 UDF * JavaManagement UDF * VIES over SOAP * WinSCP UDF * GHAPI UDF - modest begining - comunication with GitHub REST API *

My contribution to others projects or UDF based on  others projects: * _sql.au3 UDF  * POP3.au3 UDF *  RTF Printer - UDF * XML.au3 - BETA * ADO.au3 UDF SMTP Mailer UDF *

Useful links: * Forum Rules * Forum etiquette *  Forum Information and FAQs * How to post code on the forum * AutoIt Online Documentation * AutoIt Online Beta Documentation * SciTE4AutoIt3 getting started * Convert text blocks to AutoIt code * Games made in Autoit * Programming related sites * Polish AutoIt Tutorial * DllCall Code Generator * 

Wiki: Expand your knowledge - AutoIt Wiki * Collection of User Defined Functions * How to use HelpFile * Best coding practices * 

IE Related:  * How to use IE.au3  UDF with  AutoIt v3.3.14.x * Why isn't Autoit able to click a Javascript Dialog? * Clicking javascript button with no ID * IE document >> save as MHT file * IETab Switcher (by LarsJ ) * HTML Entities * _IEquerySelectorAll() (by uncommon) * 

I encourage you to read: * Global Vars * Best Coding Practices * Please explain code used in Help file for several File functions * OOP-like approach in AutoIt * UDF-Spec Questions *  EXAMPLE: How To Catch ConsoleWrite() output to a file or to CMD *

"Homo sum; humani nil a me alienum puto" - Publius Terentius Afer
"Program are meant to be read by humans and only incidentally for computers and execute" - Donald Knuth, "The Art of Computer Programming"
:naughty:  :ranting:, be  :) and       \\//_.

Anticipating Errors :  "Any program that accepts data from a user must include code to validate that data before sending it to the data store. You cannot rely on the data store, ...., or even your programming language to notify you of problems. You must check every byte entered by your users, making sure that data is the correct type for its field and that required fields are not empty."

Signature last update: 2017-06-04

Share this post


Link to post
Share on other sites

#7 ·  Posted (edited)

Yes exactly, I got the file via _FTP_FileGet from the mainframe. On my windows PC the special chars are not displayed corretly in notepad++ or ultraedit or whatever.

I thought, that I have to set the codepage before using the FileGet like I did with the ftp command in a dosprompt.

Maybe I'm wrong and there is no other chance then using a StringReplace for every special char afterwards.

Edited by Xenobiologist

Scripts & functions Organize Includes Let Scite organize the include files

Yahtzee The game "Yahtzee" (Kniffel, DiceLion)

LoginWrapper Secure scripts by adding a query (authentication)

_RunOnlyOnThis UDF Make sure that a script can only be executed on ... (Windows / HD / ...)

Internet-Café Server/Client Application Open CD, Start Browser, Lock remote client, etc.

MultipleFuncsWithOneHotkey Start different funcs by hitting one hotkey different times

Share this post


Link to post
Share on other sites

In the $iFlags try setting it to $FTP_TRANSFER_TYPE_BINARY instead of the default.


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites

#9 ·  Posted (edited)

11 minutes ago, BrewManNH said:

In the $iFlags try setting it to $FTP_TRANSFER_TYPE_BINARY instead of the default.

I did. That doesn't help. The file on the mainfraime is a normal text file. Transfering it with binary_mode creates a file like this:

 

ÇÁÄÕÙ^ÒäÙéÕÁÔÅ^ÂÁÕÒÕÁÔÅ^×Óé^ÖÙã^ãÅÉÓÖÙã^âãÙÁââÅ^ãÅÓÅÆÖÕ^ãÅÓÅÆÁç^ÒÖÕãÁÒã^ÂÕÒÂÅÙ^ÂÕÒÂÅÙäÉ^ÂÓé^ÒÙÅÉâ^ãÖäÙÉÔâ^Èãã×^ÕÙÂâÖÙÄé^ãÄÂâ^Áâðôð^Áâðóð^Áâðòð^Áâðñð^×éÂÉÓñð^×éÂÉÓñö^×éÇäåðñ^×éÂÉÓòó^ãÄÁÕÇð÷^ÁéÁÕÇðö^ÁéÁÕÇñò^ÁéÁÕÇñô^×éÇäåðó^×éÇäåðø^×éÇäåñð^×éÇäåñõ^×éÇäåñö^×éÇäåñ÷^ÕÙåÅÙÂ^æÇéÕÙ^åÖÙâãñ^åÖÙâãò^åÖÙâãó^åÖÙâãô^åÖÙâãõ^åÖÙâãö^åÖÙâã÷^åÖÙâãø^åÖÙâãù^×Óé×Æ^×ÖâãÆÁÃÈ^ÇÅÕÖÉÄ^éÅÕãÙÁÓÂÁÕÒ^åÅÙÂÁÕÄ^ÇæÔÓÉé^âÓÓÉé^ÁÙåÖÙâ^ÂÂó^@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

Edited by Xenobiologist

Scripts & functions Organize Includes Let Scite organize the include files

Yahtzee The game "Yahtzee" (Kniffel, DiceLion)

LoginWrapper Secure scripts by adding a query (authentication)

_RunOnlyOnThis UDF Make sure that a script can only be executed on ... (Windows / HD / ...)

Internet-Café Server/Client Application Open CD, Start Browser, Lock remote client, etc.

MultipleFuncsWithOneHotkey Start different funcs by hitting one hotkey different times

Share this post


Link to post
Share on other sites

I just noticed you're using the ASCII flag in the FTP_Open as well, should probably change it there as well
.


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites

#11 ·  Posted (edited)

The solution is to get the text as binary and use Windows primitives to translate between EBCDIC (which one BTW, there are so many of versions!) and native AutoIt (Unicode).

Func _CodepageToStr($sCP, $iCodepage = Default)
    If $iCodepage = Default Then $iCodepage = Int(RegRead("HKLM\SYSTEM\CurrentControlSet\Control\Nls\Codepage", "OEMCP"))
    Local $tText = DllStructCreate("byte[" & BinaryLen($sCP) & "]")
    DllStructSetData($tText, 1, $sCP)
    Local $aResult = DllCall("kernel32.dll", "int", "MultiByteToWideChar", "uint", $iCodepage, "dword", 0, "struct*", $tText, "int", BinaryLen($sCP), _
            "ptr", 0, "int", 0)
    Local $tWstr = DllStructCreate("wchar[" & $aResult[0] & "]")
    $aResult = DllCall("kernel32.dll", "int", "MultiByteToWideChar", "uint", $iCodepage, "dword", 0, "struct*", $tText, "int", BinaryLen($sCP), _
            "struct*", $tWstr, "int", $aResult[0])
    Return DllStructGetData($tWstr, 1)
EndFunc   ;==>_CodepageToStr

It's now up to you to select the correct EBCDIC codepage identifier from here.

Edited by jchd

This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

#12 ·  Posted (edited)

I tried all four combinations.

_FTP_Connect with $INTERNET_FLAG_TRANSFER_ASCII, $INTERNET_FLAG_TRANSFER_BINARY

_FTP_FileGet with $FTP_TRANSFER_TYPE_ASCII, $FTP_TRANSFER_TYPE_BINARY

Edited by Xenobiologist

Scripts & functions Organize Includes Let Scite organize the include files

Yahtzee The game "Yahtzee" (Kniffel, DiceLion)

LoginWrapper Secure scripts by adding a query (authentication)

_RunOnlyOnThis UDF Make sure that a script can only be executed on ... (Windows / HD / ...)

Internet-Café Server/Client Application Open CD, Start Browser, Lock remote client, etc.

MultipleFuncsWithOneHotkey Start different funcs by hitting one hotkey different times

Share this post


Link to post
Share on other sites

Use Binary mode instead.


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

Please note that the example file Daten1.txt is not EBCDIC at all, as a quick look on it's hex content immediately shows.

Nonetheless you'll have to determine which 8-bit charset the zOS machine uses for this file and pass the corresponding identifier to the function posted.


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

Thanks, I tried this:

 

#include <Array.au3>
#include <FTPEx.au3>
#include <MsgBoxConstants.au3>

Local $sServer = 'etet' ; zoS MVS System (IBM)
Local $sUsername = @UserName
Local $sPass = 'xxx'

Local $hOpen = _FTP_Open('MyFTP Control')
;~ Local $hConn = _FTP_Connect($hOpen, $sServer, $sUsername, $sPass, 0, 21, $INTERNET_SERVICE_FTP, $INTERNET_FLAG_TRANSFER_ASCII)
Local $hConn = _FTP_Connect($hOpen, $sServer, $sUsername, $sPass, 0, 21, $INTERNET_SERVICE_FTP, $INTERNET_FLAG_TRANSFER_BINARY)
If @error Then
    MsgBox($MB_SYSTEMMODAL, '_FTP_Connect', 'ERROR=' & @error)
Else
;~  ConsoleWrite(_FTP_Command($hConn, 'quote site sbd=(IBM-273,iso8859-1)') & @CRLF)
;~  ConsoleWrite(_FTP_FileGet($hConn, "'SRV.DWS.TEMP.BNKINFOD'", @ScriptDir & '\Daten1.txt', False, 0, $FTP_TRANSFER_TYPE_ASCII) & @CRLF)
    ConsoleWrite(_FTP_FileGet($hConn, "'SRV.DWS.TEMP.BNKINFOD'", @ScriptDir & '\Daten1.txt', False, 0, $FTP_TRANSFER_TYPE_BINARY) & @CRLF)
EndIf
Local $iFtpc = _FTP_Close($hConn)
Local $iFtpo = _FTP_Close($hOpen)

$c = FileRead(@ScriptDir & '\Daten1.txt')

FileWrite(@ScriptDir & '\Daten2.txt', _CodepageToStr($c, 1141) & @CRLF)

Func _CodepageToStr($sCP, $iCodepage = Default)
    If $iCodepage = Default Then $iCodepage = Int(RegRead("HKLM\SYSTEM\CurrentControlSet\Control\Nls\Codepage", "OEMCP"))
    Local $tText = DllStructCreate("byte[" & BinaryLen($sCP) & "]")
    DllStructSetData($tText, 1, $sCP)
    Local $aResult = DllCall("kernel32.dll", "int", "MultiByteToWideChar", "uint", $iCodepage, "dword", 0, "struct*", $tText, "int", BinaryLen($sCP), _
            "ptr", 0, "int", 0)
    Local $tWstr = DllStructCreate("wchar[" & $aResult[0] & "]")
    $aResult = DllCall("kernel32.dll", "int", "MultiByteToWideChar", "uint", $iCodepage, "dword", 0, "struct*", $tText, "int", BinaryLen($sCP), _
            "struct*", $tWstr, "int", $aResult[0])
    Return DllStructGetData($tWstr, 1)
EndFunc   ;==>_CodepageToStr

This way I get a second file Daten2.txt which now shows öäüß corretly. The only problem I've to figure out is how to insert a CRLF back into the file. The file contains currently only one long line.


Scripts & functions Organize Includes Let Scite organize the include files

Yahtzee The game "Yahtzee" (Kniffel, DiceLion)

LoginWrapper Secure scripts by adding a query (authentication)

_RunOnlyOnThis UDF Make sure that a script can only be executed on ... (Windows / HD / ...)

Internet-Café Server/Client Application Open CD, Start Browser, Lock remote client, etc.

MultipleFuncsWithOneHotkey Start different funcs by hitting one hotkey different times

Share this post


Link to post
Share on other sites

#16 ·  Posted (edited)

There is already a Windows-style line termination at the end of the Daten1.txt file (0x0D 0x0A).

Can you post the binary data as received (written as binary file)?

Edited by jchd

This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

Good morning,

#include <Array.au3>
#include <FTPEx.au3>
#include <MsgBoxConstants.au3>

Local $sServer = 'etet' ; zoS MVS System (IBM)
Local $sUsername = @UserName
Local $sPass = 'xxx'

Local $hOpen = _FTP_Open('MyFTP Control')
;~ Local $hConn = _FTP_Connect($hOpen, $sServer, $sUsername, $sPass, 0, 21, $INTERNET_SERVICE_FTP, $INTERNET_FLAG_TRANSFER_ASCII)
Local $hConn = _FTP_Connect($hOpen, $sServer, $sUsername, $sPass, 0, 21, $INTERNET_SERVICE_FTP, $INTERNET_FLAG_TRANSFER_BINARY)
If @error Then
    MsgBox($MB_SYSTEMMODAL, '_FTP_Connect', 'ERROR=' & @error)
Else
;~  ConsoleWrite(_FTP_Command($hConn, 'quote site sbd=(IBM-273,iso8859-1)') & @CRLF)
;~  ConsoleWrite(_FTP_FileGet($hConn, "'SRV.DWS.TEMP.BNKINFOD'", @ScriptDir & '\Daten1.txt', False, 0, $FTP_TRANSFER_TYPE_ASCII) & @CRLF)
    ConsoleWrite(_FTP_FileGet($hConn, "'SRV.DWS.TEMP.BNKINFOD'", @ScriptDir & '\Daten1.txt', False, 0, $FTP_TRANSFER_TYPE_BINARY) & @CRLF)
EndIf
Local $iFtpc = _FTP_Close($hConn)
Local $iFtpo = _FTP_Close($hOpen)

$c = FileRead(@ScriptDir & '\Daten1.txt')
FileDelete(@ScriptDir & '\Daten3.txt')
FileWrite(@ScriptDir & '\Daten3.txt' , StringStripWS(StringReplace(_CodepageToStr($c, 1141), ';      ', ';' & @CRLF), 7) & @CRLF)

Func _CodepageToStr($sCP, $iCodepage = Default)
    If $iCodepage = Default Then $iCodepage = Int(RegRead("HKLM\SYSTEM\CurrentControlSet\Control\Nls\Codepage", "OEMCP"))
    Local $tText = DllStructCreate("byte[" & BinaryLen($sCP) & "]")
    DllStructSetData($tText, 1, $sCP)
    Local $aResult = DllCall("kernel32.dll", "int", "MultiByteToWideChar", "uint", $iCodepage, "dword", 0, "struct*", $tText, "int", BinaryLen($sCP), _
            "ptr", 0, "int", 0)
    Local $tWstr = DllStructCreate("wchar[" & $aResult[0] & "]")
    $aResult = DllCall("kernel32.dll", "int", "MultiByteToWideChar", "uint", $iCodepage, "dword", 0, "struct*", $tText, "int", BinaryLen($sCP), _
            "struct*", $tWstr, "int", $aResult[0])
    Return DllStructGetData($tWstr, 1)
EndFunc   ;==>_CodepageToStr

This way the file looks the way it should. Of course it is not the best way to add the CR but hey, it works :-)

Thanks to all!

Mega


Scripts & functions Organize Includes Let Scite organize the include files

Yahtzee The game "Yahtzee" (Kniffel, DiceLion)

LoginWrapper Secure scripts by adding a query (authentication)

_RunOnlyOnThis UDF Make sure that a script can only be executed on ... (Windows / HD / ...)

Internet-Café Server/Client Application Open CD, Start Browser, Lock remote client, etc.

MultipleFuncsWithOneHotkey Start different funcs by hitting one hotkey different times

Share this post


Link to post
Share on other sites

#18 ·  Posted (edited)

Still interested in how to set a codepage before using FileGet or FilePut.

Without setting it I always need to manipulate the files before transfering them.

Edited by Xenobiologist

Scripts & functions Organize Includes Let Scite organize the include files

Yahtzee The game "Yahtzee" (Kniffel, DiceLion)

LoginWrapper Secure scripts by adding a query (authentication)

_RunOnlyOnThis UDF Make sure that a script can only be executed on ... (Windows / HD / ...)

Internet-Café Server/Client Application Open CD, Start Browser, Lock remote client, etc.

MultipleFuncsWithOneHotkey Start different funcs by hitting one hotkey different times

Share this post


Link to post
Share on other sites

#19 ·  Posted (edited)

AFAIK FTP protocol has only two modes: text or nibary (Oops, binary!). I.e. it has no knowledge or say about the content of what's being got/put. Such "decoration" is higher in the ISO model, so belongs to code processing things before put or after get.

Edited by jchd

This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

Then what is this?

When I use the ftp.exe from C:\Windows\System32 in a DosPrompt.

ftp etet

Type in user and password

Then it looks like this

ftp>

Now I can type in this: ftp> quote site sbd=(IBM-273,iso8859-1)
and get this --> 200 SITE command was accepted

After that the transfer is managed correctly. The umlauts etc. are shown correctly.

It seems the command tells the server to act different.


Scripts & functions Organize Includes Let Scite organize the include files

Yahtzee The game "Yahtzee" (Kniffel, DiceLion)

LoginWrapper Secure scripts by adding a query (authentication)

_RunOnlyOnThis UDF Make sure that a script can only be executed on ... (Windows / HD / ...)

Internet-Café Server/Client Application Open CD, Start Browser, Lock remote client, etc.

MultipleFuncsWithOneHotkey Start different funcs by hitting one hotkey different times

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now

  • Similar Content

    • Sigmac
      By Sigmac
      I have used this code for a long time, but for a week or 2 it stopped working?
      Connection is ok, the put create the file on server if I delete it on the FTP server, but the file is empty and after 30 secondees the function return 0
      I have tested on 2 machines, Filezilla FTP client have no issue to the server.
      Firewall is disabled under test.
        If $writeFTP = True Then
            $Open=_FTPOpen('MyFTP Control')
            $Conn=_FTPConnect($Open, $destinationServer, $destinationUsername, $destinationPass)
       MsgBox(0,"start","start")
            $Ftpp=_FtpPutFile($Conn, $filePath&$name&".json", "httpdocs/p_calender/" & $name & ".json")
      MsgBox(0,"return","return"&$Ftpp)
      Exit
            $Ftpc=_FTPClose($Open)
           EndIf
       
      From the Lib:
      Func _FTPPutFile($l_FTPSession, $s_LocalFile, $s_RemoteFile, $l_Flags = 0, $l_Context = 0)
          Local $ai_FTPPutFile = DllCall('wininet.dll', 'int', 'FtpPutFile', 'long', $l_FTPSession, 'str', $s_LocalFile, 'str', $s_RemoteFile, 'long', $l_Flags, 'long', $l_Context)
          If @error OR $ai_FTPPutFile[0] = 0 Then
              SetError(-1)
              Return 0
          EndIf
          Return $ai_FTPPutFile[0]
      EndFunc ;==> _FTPPutFile()
       
    • mLipok
      By mLipok
      As for now I was using such kind of snippet:
      Local $aFTP_FileList = _FTP_ListToArray($hFTPSession, 2) For $iFTPFileSize_idx = 1 To $aFTP_FileList[0] ConsoleWrite('Pobieram informacje o pliku:' & $aFTP_FileList[$iFTPFileSize_idx] ) $iWielkoscPliku = _FTP_FileGetSize($hFTPSession, $aFTP_FileList[$iFTPFileSize_idx]) ConsoleWrite($iFTPFileSize_idx & ' : ' & $aFTP_FileList[$iFTPFileSize_idx] & ' rozmiar=' & $iWielkoscPliku) Sleep(200) Next  
      But from some time my clients start buying cheap WD MyCloud NAS, and my problems starts.
      On all others NAS there is no problem (Seagate BlackArmour, QNAP, SYNOLOGY .... ).

      When on list there is for example 20 files then first two of them are very fast checked and reported they FileSize.
      After 2 files there is about 20 sec stop....... and next two files are checked with no errors but FileSize == 0 
      again there is about 20 sec stop....... and next two files are checked with no errors but FileSize == 0 
      again there is about 20 sec stop....... and next two files are checked with no errors but FileSize == 0 
      again there is about 20 sec stop....... and next two files are checked with no errors but FileSize == 0 
      again there is about 20 sec stop....... and next two files are checked with no errors but FileSize == 0 
      .......
       
      In time when I process investigation I checked how :  _FTP_ListToArray2D and _FTP_ListToArrayEx works and I was shocked that they works super fast.
       
      Here are the questions:
      Q1: Why does the problem occur only on WD MyCloud NAS?
      Q2: What is the significant difference that makes using _FTP_ListToArray2D and _FTP_ListToArrayEx to retrieve file sizes is still fast?
      Q3: Why _FTP_FileGetSize() not reporting @errors but returned FileSize = 0 
       
      Regards,
      mLipok
       
    • mLipok
      By mLipok
      In documentation for:
       
      #include <FTPEx.au3> _FTP_Connect ( $hInternetSession, $sServerName, $sUsername, $sPassword [, $iPassive = 0 [, $iServerPort = 0 [, $iService = $INTERNET_SERVICE_FTP [, $iFlags = 0 [, $fuContext = 0]]]]] ) There are two parameters for Passive:
       
      My question is: 

      What is a difference beetwen  using :
      $iPassive = 1
      and
      $iFlags =     $INTERNET_FLAG_PASSIVE
      Regards,
      mLipok
       
    • Xandy
      By Xandy
      Screenshot program that can upload/FTP to website/storage with hotkey.
      Features GUI to display programmable keys.

      Set the hotkeys with this function to use, and display hotkeys.
      ; hotkey_set() Parameters: ; ----------------------------------------------------------- ; $aHotkey -                The array hotkeys are stored in. This function sets the values of this array[hotkey_id][$hotkey_data] ; $dHotkey_id -             The enum for this hotkey index ; $sHotkey_description -     The label displayed to the user to represent function of hotkey ; $sHotkey_name -            The hotkey it'self.  Whatever name string you want to give the key ; $dHotkey_key -             The _IsPressed keycode ;    Keycodes AT: https://www.autoitscript.com/autoit3/docs/libfunctions/_IsPressed.htm ; $dHotkey_shift -             Shift flag 0 or 1 Default off ; $dHotkey_ctrl -             Ctrl flag 0 or 1 Default off ; $dHotkey_alt -             Alt flag 0 or 1 Default off Func hotkey_set(ByRef $aHotkey, $dHotkey_id, $sHotkey_description, $sHotkey_name, $dHotkey_key, $dHotkey_shift = 0, $dHotkey_ctrl = 0, $dHotkey_alt = 0)     $aHotkey[$dHotkey_id][$eHotkey_data_key_description] = $sHotkey_description     $aHotkey[$dHotkey_id][$eHotkey_data_key_name] = $sHotkey_name     $aHotkey[$dHotkey_id][$eHotkey_data_key] = $dHotkey_key     $aHotkey[$dHotkey_id][$eHotkey_data_shift] = $dHotkey_shift     $aHotkey[$dHotkey_id][$eHotkey_data_ctrl] = $dHotkey_ctrl     $aHotkey[$dHotkey_id][$eHotkey_data_alt] = $dHotkey_alt EndFunc   ;==>hotkey_set ; You can set your hotkeys here ; Please visit the hotkey_set() function for parameter information hotkey_set($aHotkey, $eHotkey_screenshot_ftp, "Selected Window to FTP", "F12", "7B", 0, 1, 0); F12 hotkey_set($aHotkey, $eHotkey_screenshot_disk, "Selected Window to Disk", "S", "53", 1, 1, 1); S hotkey_set($aHotkey, $eHotkey_clipboard_send, "Send Clipboard keystrokes", "F10", "79", 1, 1, 1); F10  
      Configure settings dialog:

      Screenshot Filename and Screenshot Counter, are used to create simple unique filenames that can cycle. Copy URL to clipboard option. - For linking your screenshots. The screenshot file type is for local copy only. App always uses .JPG for FTP right now, but I could add FTP screenshot file type specification.  
      Any suggestions?  Did I break anything, what did I miss?
      Package uses TTS.au3 by Beege: 
       
      FTP_Screen.zip
      File includes:
         - FTP_Screen.au3
         - FTPScreen.ico
         - TTS.au3 - by Beege
    • drorshem
      By drorshem
      Hi,
      I'm looking to create a script which will download all *.txt files from a remote FTP server.
      Once downloaded, delete all *.txt files.
       
      Can this be achieved using AutoIt ?