Jump to content
Sign in to follow this  
GregThompson

This isn't a bug, but rather, a limitation (I think), and a request.

Recommended Posts

GregThompson

I'm pulling back large amounts of data using an ADODB connection with AutoIT and if I put the data into a 2D array it's truncating the results per element(cell). If I change it up and pull the data back with the SSMS and paste into an Excel file, then read cell by cell into strings I get all the data I'm looking for so I know it's some kind of size limit, or character limit with 2D array elements. Any thoughts on this, I'd LOVE to be able to just increase the element sizes up to like 100,000 characters?

Edited by GregThompson

Share this post


Link to post
Share on other sites
JLogan3o13

Here's a thought, how about providing an example ;)


√-1 2^3 ∑ π, and it was delicious!

Share this post


Link to post
Share on other sites
jdelaney

What's the end result? If the data set is very large, it takes a lot of time to migrate it to an array...which is a waste of time, when you can instead loop through, and print to the end location, such as an excel document, txt file, etc...cut out the middle man


IEbyXPATH-Grab IE DOM objects by XPATH IEscriptRecord-Makings of an IE script recorder ExcelFromXML-Create Excel docs without excel installed GetAllWindowControls-Output all control data on a given window.

Share this post


Link to post
Share on other sites
kylomas

Greg,

so I know it's some kind of size limit, or character limit with 2D array elements. Any thoughts on this

Yes, proof of concept. The console outputs the length of each cell in a 2D array.

;
;
;

local $str, $array[10][5]

for $1 = 0 to ubound($array,1) - 1
    for $2 = 0 to ubound($array,2) - 1
        for $3 = 1 to 50000
            $str &= string($3)
        Next
        $array[$1][$2] = $str
        $str = ''
    next
next

for $1 = 0 to ubound($array,1) - 1
    ConsoleWrite('Length : ')
    for $2 = 0 to ubound($array,2) - 1
        consolewrite('Element # [' & $1 & ',' & $2 & '] ' & stringlen($array[$1][$2]) & ' ')
    next
    ConsoleWrite(@LF)
next

Post your code as suggested by JLogan3o13...

kylomas


Forum Rules         Procedure for posting code

"I like pigs.  Dogs look up to us.  Cats look down on us.  Pigs treat us as equals."

- Sir Winston Churchill

Share this post


Link to post
Share on other sites
Bowmore

There is no limit on the number of characters that can be strored in an AutoIt array element, other than the basic Max string length, array size, max integer etc.

I believe what the OP is probably coming up against the 259 character limit for what can be displayed in a standard windows ListView item and sub-items. The full string is still correctly stored in the ListView just not viewable. To view it you need to display the Listview sub-item, with focus, text in an edit control. There is also a limit on the total number of characters that can be viewed in a all the subitems of a single list view item. I can't remember the exact value, around 4000 I think This can result in some fields appearing to be empty when using a ListView to display large database records that exceed this value.


"Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the universe trying to build bigger and better idiots. So far, the universe is winning."- Rick Cook

Share this post


Link to post
Share on other sites
Edano

There is no limit on the number of characters that can be strored in an AutoIt array element, other than the basic Max string length, array size, max integer etc.

I believe what the OP is probably coming up against the 259 character limit for what can be displayed in a standard windows ListView item and sub-items. The full string is still correctly stored in the ListView just not viewable. To view it you need to display the Listview sub-item, with focus, text in an edit control. There is also a limit on the total number of characters that can be viewed in a all the subitems of a single list view item. I can't remember the exact value, around 4000 I think This can result in some fields appearing to be empty when using a ListView to display large database records that exceed this value.

.

just want to agree, if others have the same problem. took me some time to find it out.

there is a limit on the displayed string length of listview (sub)items, but if you read it out with _GUICtrlListView_GetItemText, it is stored correctly. so only a display limit (still sad tho)

E.


[color=rgb(255,0,0);][font="'comic sans ms', cursive;"]FukuLeaks[/color][/font]

Share this post


Link to post
Share on other sites
jchd

OTOH is it reasonable to expect that a field of length 100K be listed in full in a simple control like ListView?

From any UI point of view this is a no-no.


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites
guinness

You could use _WinAPI_PathCompactPathEx in WinAPIEx?


UDF List:

 
_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_ArrayFilter/_ArrayReduce_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 22/04/2018

Share this post


Link to post
Share on other sites
BrewManNH

You could use _WinAPI_PathCompactPathEx in WinAPIEx?

Thinking outside the box, I like it. I would never have realized it would work with a simple string as well as a path name to get the results you need, especially considering its name.


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites
Edano

hmm, is there a way to make it visible ?


[color=rgb(255,0,0);][font="'comic sans ms', cursive;"]FukuLeaks[/color][/font]

Share this post


Link to post
Share on other sites
BrewManNH

Make what visible?


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites
czardas

hmm, is there a way to make it visible ?

 

Write the array to the SciTE console or a txt file.

Edited by czardas

Share this post


Link to post
Share on other sites
guinness

Thinking outside the box, I like it. I would never have realized it would work with a simple string as well as a path name to get the results you need, especially considering its name.

There still might be a limitation, so play around with this...

ConsoleWrite(_PathCompactPath(@ScriptFullPath, 10) & @CRLF)

Func _PathCompactPath($sFilePath, $iFlag = 10) ; By AZJIO.
    If StringLen($sFilePath) > $iFlag Then
        Return StringRegExpReplace($sFilePath, '(^.{3,11}\\|.{11})(.*)(\\.{6,27}|.{27})$', '\1...\3')
    EndIf
    Return $sFilePath
EndFunc   ;==>_PathCompactPath

UDF List:

 
_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_ArrayFilter/_ArrayReduce_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 22/04/2018

Share this post


Link to post
Share on other sites
Edano

Make what visible?

.

the entire subitem text without truncation IN the listview

Edited by Edano

[color=rgb(255,0,0);][font="'comic sans ms', cursive;"]FukuLeaks[/color][/font]

Share this post


Link to post
Share on other sites
BrewManNH

What part of this ENTIRE thread eluded you? No, it's NOT possible because Windows won't allow it.


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites
Edano

What part of this ENTIRE thread eluded you? No, it's NOT possible because Windows won't allow it.

.

OK thx :)


[color=rgb(255,0,0);][font="'comic sans ms', cursive;"]FukuLeaks[/color][/font]

Share this post


Link to post
Share on other sites
czardas

You could try spliting the strings using something like StringEqualSplit and then display them as a number of separate items. It might not work.

Edited by czardas

Share this post


Link to post
Share on other sites
Edano

You could try spliting the strings using something like StringEqualSplit and then display them as a number of separate items. It might not work.

.

i tried $LVS_EX_LABELTIP, but the tip is too long to fit the screen ..... same problem.

now i decided to make my own custom tooltip with $LVN_HOTTRACK and splitted item text with @CRLF. that's satisfying.


[color=rgb(255,0,0);][font="'comic sans ms', cursive;"]FukuLeaks[/color][/font]

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×