Sign in to follow this  
Followers 0
Synapse

Content-type from URL

3 posts in this topic

Hi i made a script the downloads file from a given url. The problem is that i want it to download all kind of files except web pages (html, php, asp..etc).

How can i get the content-type of the url and decide if it`s a webpage or a file (exe, zip, etc) ?

Share this post


Link to post
Share on other sites



#2 ·  Posted (edited)

You need to get header of the page and then parse it, something like this:

Global $HTTPUserAgent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0"

$Host = "www.autoitscript.com"
$Page = "/forum/index.php"

$Ret = _HTTPGetContentType($Host, $Page)
MsgBox(64, $Host, "Content-Type for <" & $Host & $Page & "> is: " & @LF & _
    $Ret & @LF & @LF & "Response time: " & @extended & " milliseconds")

Func _HTTPGetContentType($Host, $Page)
    Local $Response_Time = TimerInit()
    Local $Socket = _HTTPConnect($Host)
    If @error Then Return SetError(1, 0, "")
    
    _HTTPHead($Host, $Page, $Socket)
    
    Local $Recv = "", $CurrentRecv
    While 1
        $CurrentRecv = TCPRecv($Socket, 16)
        If @error <> 0 Then ExitLoop
        If $CurrentRecv <> "" Then $Recv &= $CurrentRecv
    WEnd
    
    $Response_Time = Round(TimerDiff($Response_Time), 1)
    
    _HTTPClose($Socket)
    
    Local $ContentType = StringRegExp($Recv, '(?s)(?i)Content-Type: (.*?)\n', 3)
    If Not IsArray($ContentType) Or $ContentType[0] = "" Then Return "Unknown"
    
    Return SetExtended($Response_Time, $ContentType[0])
EndFunc

Func _HTTPConnect($Host)
    TCPStartup()
    Local $Name_To_IP = TCPNameToIP($Host)
    Local $Socket = TCPConnect($Name_To_IP, 80)
    
    If $Socket = -1 Then
        TCPCloseSocket($Socket)
        Return SetError(1, 0, "")
    EndIf
    
    Return $Socket
EndFunc

Func _HTTPHead($Host, $Page, $Socket)
    Local $Command = "HEAD " & $Page & " HTTP/1.1" & @CRLF
    $Command &= "Host: " & $Host & @CRLF
    $Command &= "User-Agent: " & $HTTPUserAgent & @CRLF
    $Command &= "Connection: close" & @CRLF & @CRLF
    
    Local $BytesSent = TCPSend($Socket, $Command)
    If $BytesSent = 0 Then Return SetError(2, @error, 0)
    Return $BytesSent
EndFunc

Func _HTTPClose($Socket)
    TCPCloseSocket($Socket)
    TCPShutdown()
EndFunc
Edited by MsCreatoR

 

Spoiler

Using OS: Win 7 Professional, Using AutoIt Ver(s): 3.3.6.1 / 3.3.8.1

AutoIt_Rus_Community.png AutoIt Russian Community

My Work...

Spoiler

AutoIt_Icon_small.pngProjects: ATT - Application Translate Tool {new}| BlockIt - Block files & folders {new}| SIP - Selected Image Preview {new}| SISCABMAN - SciTE Abbreviations Manager {new}| AutoIt Path Switcher | AutoIt Menu for Opera! | YouTube Download Center! | Desktop Icons Restorator | Math Tasks | KeyBoard & Mouse Cleaner | CaptureIt - Capture Images Utility | CheckFileSize Program

AutoIt_Icon_small.pngUDFs: OnAutoItErrorRegister - Handle AutoIt critical errors {new}| AutoIt Syntax Highlight {new}| Opera Library! | Winamp Library | GetFolderToMenu | Custom_InputBox()! | _FileRun UDF | _CheckInput() UDF | _GUIInputSetOnlyNumbers() UDF | _FileGetValidName() UDF | _GUICtrlCreateRadioCBox UDF | _GuiCreateGrid() | _PathSplitByRegExp() | _GUICtrlListView_MoveItems - UDF | GUICtrlSetOnHover_UDF! | _ControlTab UDF! | _MouseSetOnEvent() UDF! | _ProcessListEx - UDF | GUICtrl_SetResizing - UDF! | Mod. for _IniString UDFs | _StringStripChars UDF | _ColorIsDarkShade UDF | _ColorConvertValue UDF | _GUICtrlTab_CoverBackground | CUI_App_UDF | _IncludeScripts UDF | _AutoIt3ExecuteCode | _DragList UDF | Mod. for _ListView_Progress | _ListView_SysLink | _GenerateRandomNumbers | _BlockInputEx | _IsPressedEx | OnAutoItExit Handler | _GUICtrlCreateTFLabel UDF | WinControlSetEvent UDF | Mod. for _DirGetSizeEx UDF
 
AutoIt_Icon_small.pngExamples: 
ScreenSaver Demo - Matrix included | Gui Drag Without pause the script | _WinAttach()! | Turn Off/On Monitor | ComboBox Handler Example | Mod. for "Thinking Box" | Cool "About" Box | TasksBar Imitation Demo

Like the Projects/UDFs/Examples? Please rate the topic (up-right corner of the post header: Rating AutoIt_Rating.gif)

* === My topics === *

==================================================
My_Userbar.gif
==================================================

 

 

 

AutoIt is simple, subtle, elegant. © AutoIt Team

Share this post


Link to post
Share on other sites

You need to get header of the page and then parse it, something like this:

Global $HTTPUserAgent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0"

$Host = "www.autoitscript.com"
$Page = "/forum/index.php"

$Ret = _HTTPGetContentType($Host, $Page)
MsgBox(64, $Host, "Content-Type for <" & $Host & $Page & "> is: " & @LF & _
    $Ret & @LF & @LF & "Response time: " & @extended & " milliseconds")

Func _HTTPGetContentType($Host, $Page)
    Local $Response_Time = TimerInit()
    Local $Socket = _HTTPConnect($Host)
    If @error Then Return SetError(1, 0, "")
    
    _HTTPHead($Host, $Page, $Socket)
    
    Local $Recv = "", $CurrentRecv
    While 1
        $CurrentRecv = TCPRecv($Socket, 16)
        If @error <> 0 Then ExitLoop
        If $CurrentRecv <> "" Then $Recv &= $CurrentRecv
    WEnd
    
    $Response_Time = Round(TimerDiff($Response_Time), 1)
    
    _HTTPClose($Socket)
    
    Local $ContentType = StringRegExp($Recv, '(?s)(?i)Content-Type: (.*?)\n', 3)
    If Not IsArray($ContentType) Or $ContentType[0] = "" Then Return "Unknown"
    
    Return SetExtended($Response_Time, $ContentType[0])
EndFunc

Func _HTTPConnect($Host)
    TCPStartup()
    Local $Name_To_IP = TCPNameToIP($Host)
    Local $Socket = TCPConnect($Name_To_IP, 80)
    
    If $Socket = -1 Then
        TCPCloseSocket($Socket)
        Return SetError(1, 0, "")
    EndIf
    
    Return $Socket
EndFunc

Func _HTTPHead($Host, $Page, $Socket)
    Local $Command = "HEAD " & $Page & " HTTP/1.1" & @CRLF
    $Command &= "Host: " & $Host & @CRLF
    $Command &= "User-Agent: " & $HTTPUserAgent & @CRLF
    $Command &= "Connection: close" & @CRLF & @CRLF
    
    Local $BytesSent = TCPSend($Socket, $Command)
    If $BytesSent = 0 Then Return SetError(2, @error, 0)
    Return $BytesSent
EndFunc

Func _HTTPClose($Socket)
    TCPCloseSocket($Socket)
    TCPShutdown()
EndFunc

Thx man it`s just what i was looking 4 <_< 10x

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0