Last modification of file on web server?

[GaryC]

My program uses a .CSV file that is downloaded from a web page. I would like to determine when the file is updated. I know that when I load a HTML page there is a last modified attribute in the document object. When I do a HEAD request for the file via NC I don't get the modification date:

nc -v -v 92.242.144.2 80 <head.txt
[head.txt contains:
HEAD /NWCC_Roster.csv HTTP/1.0

]

92.242.144.2: inverse host lookup failed: h_errno 11004: NO_DATA        
(UNKNOWN) [92.242.144.2] 80 (http) open
HTTP/1.1 200 OK
Date: Wed, 19 Jan 2011 17:28:04 GMT
Server: Apache
Connection: close
Content-Type: text/html; charset=UTF-8

sent 34, rcvd 131: NOTSOCK

Is there a way to get the last modification date of the file?

Thanks.

Gary

[Richard Robertson]

You may need to provide a different User Agent header.

Edited January 19, 2011 by Richard Robertson

[GaryC]

I did some more experimenting and was successful. First, the IP address I used, which I got from the IE8 properties, was wrong. I thought that nc wouldn't do a host look-up, but it does. I also changed to a http1.1 request and I got a Last-Modified header.

I used nc (netcat) to do a quick test because I had it laying around:

nc -v -v www.nwcousins.net 80 <head1.1.txt
[head1.1.txt contains:
HEAD /NWCC_Roster.csv HTTP/1.1
Host: www.nwcousins.net
Connection: close

]

Response:

DNS fwd/rev mismatch: www.nwcousins.net != choklat.getnetserver.com
www.nwcousins.net [64.22.106.177] 80 (http) open
HTTP/1.1 200 OK
Date: Fri, 21 Jan 2011 00:45:49 GMT
Server: Apache
Last-Modified: Mon, 03 Jan 2011 19:36:26 GMT
ETag: "42b0033-d1fe-498f644884280"
Accept-Ranges: bytes
Content-Length: 53758
Connection: close
Content-Type: text/plain; charset=UTF-8

sent 78, rcvd 259: NOTSOCK

Sending with a HTTP1.0 request got an error, which I don't understand since servers that accept 1.1 requests are also supposed to accept 1.0 requests. I found this page helpful: HTTP Made Really Easy. My next step will be to modify the code in to do what I need.

Hope someone finds this useful.

Gary

Edited January 22, 2011 by GaryC