FTP_ListToArray not getting the entire file list???

[RickB75]

Guys,

     I'm a little stumped on this one. I've been running my script for about 2 yrs now and not had too many bumps in the road. Today I've ran into a small issue and I can't quite figure it out. For some reason FTP_ListToArray isn't getting the entire list of files on my server in the specified directory. I have 1366 files on the server in the specified directory and using FTP_ListToArray, I get a total of 1328 files. I use the array list to cross reference files I've already downloaded to find files that I need to download.  Any ideas on how I can get my entire file list?? This is a new issue and just popped up. I'm not getting any errors in the control panel.

[RickB75]

Any pointers anyone???

[RickB75]

I keep hearing crickets chirping guys...

[kaotkbliss]

I would guess it's your filter on your filelist.

*edit*

without seeing any code, the issue is anyone's guess (it doesn't have to be the real code, just sample code or edited code that reproduces the problem)

Question, have you had that many files on the ftp before, or is that new as well?

Have you checked to see if the server is timing out?

Edited November 15, 2013 by kaotkbliss

[RickB75]

I've checked the server and it's not timing out. I thought it maybe was returning to quickly because of the amount of files so I put a sleep cmd in there for about 4 seconds and it returned the same number of files. Here's the code I'm using. As I stated earlier, I've been using it for quite some time now and it never failed me before. I've had as many as 2,000 files it the directory before and it worked ok then. 

$host = "myftpserver"           ; Hostname of the webserver (Can be a domain name or an IP)
$user = "username"          ; FTP Username
$pass = "password"          ; FTP Password



$open = _FTP_Open("Myftp")
$conn = _FTP_Connect($open, $host, $user, $pass,1)
_FTP_DirSetCurrent($conn,"/mydirectory/")
$wlvins = _FTP_ListToArray($conn,2)
_FTP_Close($open)

_ArrayTrim($wlvins,4,1)

_ArrayDisplay($wlvins)

[kaotkbliss]

Have you put in an _arraydisplay before closing the connection and trimming just to see if maybe something wasn't getting trimmed off or that you were getting the same files in the array as the last _arraydisplay? 

*edit*

and since you're returning files only, are some of the files in a sub dir? maybe try with default return (both files and dir)

Edited November 16, 2013 by kaotkbliss

[RickB75]

Kaotkbliss

     For what ever reason, today the script is returning the correct number of files. I don't understand it. I did try the array display before trimming the array and closing the ftp session. it was still returning the same number. Is there another way to check things like this besides an array display? Is there another way I can get the total number of files to check against that you may know of? I Just thought of it. I can get the directory file size and use a different function to get the file list along with each file size and add the file sizes  together for a total. I can use that as my check.

Thank you Kaotkbliss for your help. It is greatly appreciated. I guess it was a bug in one of the systems somewhere.

[kaotkbliss]

So now the question I guess, is if this happens again how are you going to handle it?

You mentioned this was for looking for new files you need to download, so maybe keep a text file of the files you've already downloaded. You can read that to an array, then get your list from the ftp and start looping through the arrays searching to see if a file is in your "already downloaded" list. If it is, you can skip it, if not download it. Then, if there's a period where you don't get the full file list from the ftp, you still have a way to automatically check against old files.

[RickB75]

Ok, I need to step back. It is still having the same issue but todays number of files is 1366 and it should be 1400. How / what does Autoit use for the ftp client and does it have a cache?? I was researching and reading different post on the web, IE can hold onto a cache file for ftp. Firefox is set to always get the newest version. Does Autoit used IE as part of the FTP UDF??

[RickB75]

Thats it!!!! I just cleared out my cache file in IE and changed my IE settings to check for new content each time I visit a webpage. I reran the script and it pulled 1400!!!

[RickB75]

That seems to have fixed this issue. After doing some research, I need add a flag to the end of the function like this

_FTP_ListToArray($conn,2,$INTERNET_FLAG_NO_CACHE_WRITE)

I never would have guessed that there was a cache file being written each time it was connecting and getting the file list built but there is. Here's info about it. If your having problems getting the correct file list, make sure to add the correct flags.

[kaotkbliss]

Glad it got sorted

[RickB75]

Kaotkbliss,

Thank you for all your help, tips, input and advice. It is truly and greatly appreciated.