Jump to content
Sign in to follow this  
andrewz

Implenting script into another one.

Recommended Posts

andrewz

Hey,

first off, thanks to everyone on this forum who has helped me out yet. I wouldnt have made it this far in such a short time, as I am a beginner.

Now this is my last problem: (Well it's not rlly a problem as I could run both seperately, but I would like to create one script.

Here is the first script:

#NoTrayIcon
#include <Inet.au3>
#include <Array.au3>
#include <String.au3>
#include <AutoItConstants.au3>
#include <MsgBoxConstants.au3>
Global $url = InputBox("ScoutID","Please enter the Scout-ID (URL)")

If StringInStr($url,"http://www.immobilienscout24.de/") = false Then
MsgBox(48,"Error","URL not specified. Please make sure you enter the URL !")
Exit
Else
EndIf

Global $content = _INetGetSource($url)
Global $string_A = _StringBetween($content, '<li data-item="result" data-obid=', 'id="result')

$number =UBound ( $string_A ,$UBOUND_ROWS)

$timesran = -1
$i=0

$numberarray = $number
$numberarray -= 1

Do
$i += 1
$timesran += 1
If $timesran = $numberarray = True Then
Exit
Else
EndIf
$id= StringReplace($string_A[$timesran],'"',"")
$urlscout = "http://www.immobilienscout24.de/expose/"&$id
export ()
Until $i = $numberarray

func export ()
$url = $urlscout

If StringInStr($url,"http://www.immobilienscout24.de/expose") = false Then
MsgBox(48,"Error","URL not specified. Please make sure you enter the URL !")
Exit
Else
EndIf

If FileExists("Immobilien.csv") =false Then
FileWrite("Immobilien.csv","Name;Adresse;Telefon;Objekttyp;Ort;Baujahr;Zimmer;Bezugfrei ab;Wfl./ qm;Kaltmiete;Warmmiete;Scout- ID"& @CRLF)
EndIf

$content = _INetGetSource($url)
$name_A = _StringBetween($content, '<span data-qa="contactName" class="font-bold">', '</span>')
$preis_A = _StringBetween($content, '<span class="is24-operator">', '<div class="sourcecode section">')
$strase_A = _StringBetween($content, '<strong class="font-standard">' , '</strong><br/>')
$telefon_A = _StringBetween($content, '<div class="is24-phone-number hide">' ,'</div>')
$objekttyp_A = _StringBetween($content, '<dd class="is24qa-wohnungstyp">' ,'</dd>')
$ort_A = _StringBetween($content, '<span id="quickCheckHeader" class="is24-f">' , '</span>')
$baujahr_A = _StringBetween($content, '<dd class="is24qa-baujahr">','</dd>')
$zimmer_A = _StringBetween($content, '<dd class="is24qa-zimmer">','</dd>')
$bezugsfrei_A = _StringBetween($content, '<dd class="is24qa-bezugsfrei-ab">' ,'</dd>')
$wohnflache_A = _StringBetween($content, '<dd class="is24qa-wohnflaeche-ca">' ,'</dd>')
$preiswarm_A =_StringBetween($content, '<strong class="is24qa-gesamtmiete">','</strong>')

If  IsArray($strase_A) Then
$strase_B = $strase_A[0]
Else
$strase_B = "/"
EndIf

If  IsArray($name_A) Then
$name_B = $name_A[0]
Else
$name_B = "/"
EndIf

If  IsArray($telefon_A) Then
$telefon_B = $telefon_A[0]
Else
$telefon_B = "/"
EndIf

If  IsArray($objekttyp_A) Then
$objekttyp_B = $objekttyp_A[0]
Else
$objekttyp_B = "/"
EndIf

If  IsArray($baujahr_A) Then
$baujahr_B = $baujahr_A[0]
Else
$baujahr_B = "/"
EndIf

If  IsArray($preiswarm_A) Then
$preiswarm_B = $preiswarm_A[0]
$preiswarm_C = StringReplace($preiswarm_B , "disabled","")
$preiswarm_D = StringReplace($preiswarm_C, "disabled","",1) ;caused some bugs
$preiswarm_E = StringReplace($preiswarm_D, "disabled","")
Else
$preiswarm_B = "/"
EndIf

If  IsArray($zimmer_A) Then
$zimmer_B = $zimmer_A[0]
$zimmer_C = StringReplace($zimmer_B , ".",",")
Else
$zimmer_B = "/"
EndIf

If  IsArray($ort_A) Then
$ort_B = $ort_A[0]
Else
$ort_B = "/"
EndIf

If  IsArray($bezugsfrei_A) Then
$bezugsfrei_B = $bezugsfrei_A[0]
Else
$bezugsfrei_B = "/"
EndIf

If  IsArray($wohnflache_A) Then
$wohnflache_B = $wohnflache_A[0]
Else
$wohnflache_B = "/"
EndIf

If  IsArray($preis_A) Then
$preis_B = $preis_A[0]
Else
$preis_B = "/"
EndIf


$aio= $name_B&";"&$strase_B&";"&$telefon_B&";"&$objekttyp_B&";"&$ort_B&";"&$baujahr_B&";"&$zimmer_C&";"&$bezugsfrei_B&";"&$wohnflache_B&";"&$preis_B&";"&$preiswarm_E&";"&$url

$sString1 = StringReplace($aio, "  ", "")
$sString2 = StringReplace($sString1, "<p>", "")
$sString3 = StringReplace($sString2, "<span>", "")
$sString4 = StringReplace($sString3, "</p>", "")
$sString5 = StringReplace($sString4, "Â", "")
$sString6 = StringReplace($sString5, '<span class="is24-operator">=</span>', "")     ;Filtering the given Data to get a proper CSV Format
$sString7 = StringReplace($sString6, "EUR", "")
$sString8 = StringReplace($sString7, "</span>","")
$sString9 = StringReplace($sString8, "Dieses Objekt im Vergleich zu anderen in ","")
$sString10 = StringReplace($sString9, "disabled",",00") ;caused bug

$sString11 = StringReplace($sString10, "ü","ü")
$sString12 = StringReplace($sString11, "ß","ß")     ;Including special characters lower case
$sString13 = StringReplace($sString12, "ä","ä")
$sString14 = StringReplace($sString13, "ö","ö")

$sString15 = StringReplace($sString14, "Ãœ","Ü")    ;Including special characters upper case
$sString16 = StringReplace($sString15, "Ä","Ä")
$sString17 = StringReplace($sString16, "Ö","Ö")

$sStringfinal = StringReplace($sString17, @CRLF, "")


FileWrite ( "Immobilien.csv", $sStringfinal & @CRLF )

ToolTip("Importing...", 0, 0)
Sleep(100)
endfunc

This script works perfectly fine to extract ALL the data from one page which looks like this :

http://www.immobilienscout24.de/Suche/S-T/Wohnung-Miete/Bayern/Muenchen/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/true?enteredFrom=result_list

(The export function downloads all data from each object on the side into a cvs file.)

BUT this is just one of 19 pages, so I made this script which counts the pages and then in a loop

sets $currenturl always +1 until $currenturl and the total pages (19) match and then exits.

#NoTrayIcon
#include <Inet.au3>
#include <Array.au3>
#include <String.au3>
#include <AutoItConstants.au3>
#include <MsgBoxConstants.au3>
Global $url = InputBox("ScoutID","Please enter the Scout-ID (URL)")

If StringInStr($url,"http://www.immobilienscout24.de/") = false Then
MsgBox(48,"Error","URL not specified. Please make sure you enter the URL !")
Exit
Else
EndIf

Global $content1 = _INetGetSource($url)
Global $pagecount = _StringBetween($content1, '<li><span>&hellip;</span></li>', '<span class="smallPager">')
$filtered = StringRegExp($pagecount[0],"\d{2}",$STR_REGEXPARRAYMATCH)
$link0 = StringReplace($url,"http://www.immobilienscout24.de/Suche/S-T/","")
$link1 = StringReplace($url,$link0,"")
$link2 =  "/" & $link0
$pages = $filtered[0] ; 19 in the example
$currentpage = 0


Do
$currentpage +=1

$currenturl = $link1 & "p-" & $currentpage & $link2
MsgBox(0,"",$currenturl)

Until $pages = $currentpage

Now my question is, if there is anyway to implent the second into the first script to work the following way:

1.) Basically run the second script to generate the first $currenturl

2.) Run the first script to grab all data from $currenturl

3.) The second script "prints" the second $currenturl

4.) The first script again grabs all data from $currenturl

....

5.) Until $currenturl = $filtered ($filtered = total amount of pages)

 

I tried my best to explain it as good as possible, even tho its complicated to get what I mean...

THANKS IN ADVANCE! & best regards,

Andrewz

Edited by andrewz

Share this post


Link to post
Share on other sites
computergroove

If you know what to do with the number of pages in your script then I will just give you this:

Add this to the top for your first script in the above example in the global variables section(at the top):

Global $content = _INetGetSource($url)
Global $string_A = _StringBetween($content, '<li data-item="result" data-obid=', 'id="result')
Global $pagecount = _StringBetween($content, '<li><span>&hellip;</span></li>', '<span class="smallPager">')
Global $pages;<-number of websites to scan

NumberOfPages();<-Find the number of pages to scan

Add this to the bottom of your script

Func NumberOfPages()
$filtered = StringRegExp($pagecount[0],"\d{2}",$STR_REGEXPARRAYMATCH)
$link0 = StringReplace($url,"http://www.immobilienscout24.de/Suche/S-T/","")
$link1 = StringReplace($url,$link0,"")
$link2 =  "/" & $link0
$pages = $filtered[0]
EndFunc

I played with this code for awhile and I am having a problem shoehorning this into a loop. I added a for loop:

For $j=1 To $pages

but you have a main url (ie http://www.immobilienscout24.de/Suche/S-T...) and an internal url (ie http://www.immobilienscout24.de/expose/...) that I am having a hard time integrating into the loop. $pages will give you the number of pages you need to scan this program with.

Edited by computergroove

Get Scite to add a popup when you use a 3rd party UDF -> http://www.autoitscript.com/autoit3/scite/docs/SciTE4AutoIt3/user-calltip-manager.html

Share this post


Link to post
Share on other sites
mikell

1.) Basically run the second script to generate the first $currenturl

2.) Run the first script to grab all data from $currenturl

3.) The second script "prints" the second $currenturl

4.) The first script again grabs all data from $currenturl

....

5.) Until $currenturl = $filtered ($filtered = total amount of pages)

 

Couldn't this be done using funcs like in this pseudocode ?

For $var = 1 to $filtered      ; total amount of pages
   $currenturl = _secondScript($var)    ; generate $currenturl
   $data = _firstScript($currenturl)      ; grab data from $currenturl
Next

Share this post


Link to post
Share on other sites
computergroove

I made a script that goes through all the pages and gets the unique number that needs follow  "http://www.immobilienscout24.de/expose/

#include <Inet.au3>
#include <array.au3>
#include <String.au3>
#include <AutoItConstants.au3>
#include <MsgBoxConstants.au3>
#include <File.au3>

Global $Array[0];Array with all the link number info (ie http://www.immobilienscout24.de/expose/'77947839' <- this number
Global $FinalArray[0]
Global $url = "http://www.immobilienscout24.de/Suche/S-T/Wohnung-Miete/Bayern/Muenchen/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/true?enteredFrom=result_list"
Global $content = _INetGetSource($url)
Global $pagecount = _StringBetween($content, '<li><span>&hellip;</span></li>', '<span class="smallPager">')
Global $pages;<-number of websites to scan

NumberOfPages();Get number of pages to scan
For $k = 2 To 20
    GetNumberedLinks()
    _ArrayConcatenate($FInalArray,$Array)
    $url = "http://www.immobilienscout24.de/Suche/S-T/P-" & $k & "/Wohnung-Miete/Bayern/Muenchen/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/true?enteredFrom=result_list"
    $HTML = _INetGetSource($url)
Next
    _ArrayDisplay($Array)

Func GetNumberedLinks();Get the number that follows http://www.immobilienscout24.de/expose/ in the current html
        ToolTip("Processing Stage: " & $k - 1 & " of " & $pages,0,0)
        $HTML = _INetGetSource($url)
        $j = 1
        Do
            $1 = StringInStr($HTML, "/expose/", 0, $j)
            $2 = StringTrimLeft($HTML, $1)
            $3 = StringReplace($2, '"', ";;")
            $4 = StringSplit($3, ";;", 1)
            $5 = $4[1]
            $6 = StringMid($5, 8, 8)
            If $6 >= 1 Then;Needed to add this if statement because I was getting extra characters at the end of the finalarray
                _ArrayAdd($Array,$6,1)
            EndIf
            $j += 1
        Until $1 = 0;Stop loop when there are no more instances of "/expose/"
        $Array = _ArrayUnique($Array,0,0,0,0)
EndFunc

Func NumberOfPages()
    $filtered = StringRegExp($pagecount[0], "\d{2}", $STR_REGEXPARRAYMATCH)
    $link0 = StringReplace($url, "http://www.immobilienscout24.de/Suche/S-T/", "")
    $link1 = StringReplace($url, $link0, "")
    $link2 = "/" & $link0
    $pages = $filtered[0]
EndFunc   ;==>NumberOfPages
Edited by computergroove

Get Scite to add a popup when you use a 3rd party UDF -> http://www.autoitscript.com/autoit3/scite/docs/SciTE4AutoIt3/user-calltip-manager.html

Share this post


Link to post
Share on other sites
computergroove

Looks like I got it to work. Test it and let me know if the data is correct.

;~ #NoTrayIcon
#include <Inet.au3>
#include <Array.au3>
#include <String.au3>
#include <AutoItConstants.au3>
#include <MsgBoxConstants.au3>

Global $Array[0];Array with all the link number info (ie http://www.immobilienscout24.de/expose/'77947839' <- this number
Global $FinalArray[0]
Global $url = "http://www.immobilienscout24.de/Suche/S-T/Wohnung-Miete/Bayern/Muenchen/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/true?enteredFrom=result_list"
Global $content = _INetGetSource($url)
Global $string_A = _StringBetween($content, '<li data-item="result" data-obid=', 'id="result')
Global $pagecount = _StringBetween($content, '<li><span>&hellip;</span></li>', '<span class="smallPager">')
Global $pages;<-number of websites to scan
FileDelete(@ScriptDir & "\Immobilien.csv")

NumberOfPages()

For $k = 2 To 20
    GetNumberedLinks()
    _ArrayConcatenate($FinalArray,$Array)
    $url = "http://www.immobilienscout24.de/Suche/S-T/P-" & $k & "/Wohnung-Miete/Bayern/Muenchen/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/true?enteredFrom=result_list"
    $HTML = _INetGetSource($url)
Next

;~ $number =UBound ( $string_A ,$UBOUND_ROWS)
;~ $timesran = -1
;~ $i=0
;~ $numberarray = $number
;~ $numberarray -= 1

For $m = 0 To Ubound($Array)
$id= $FinalArray[$m]
$urlscout = "http://www.immobilienscout24.de/expose/"&$id
export ()
Next

func export ()
$url = $urlscout

If StringInStr($url,"http://www.immobilienscout24.de/expose") = false Then
MsgBox(48,"Error","URL not specified. Please make sure you enter the URL !")
Exit
Else
EndIf

If FileExists("Immobilien.csv") =false Then
FileWrite("Immobilien.csv","Name;Adresse;Telefon;Objekttyp;Ort;Baujahr;Zimmer;Bezugfrei ab;Wfl./ qm;Kaltmiete;Warmmiete;Scout- ID"& @CRLF)
EndIf

$content = _INetGetSource($url)
$name_A = _StringBetween($content, '<span data-qa="contactName" class="font-bold">', '</span>')
$preis_A = _StringBetween($content, '<span class="is24-operator">', '<div class="sourcecode section">')
$strase_A = _StringBetween($content, '<strong class="font-standard">' , '</strong><br/>')
$telefon_A = _StringBetween($content, '<div class="is24-phone-number hide">' ,'</div>')
$objekttyp_A = _StringBetween($content, '<dd class="is24qa-wohnungstyp">' ,'</dd>')
$ort_A = _StringBetween($content, '<span id="quickCheckHeader" class="is24-f">' , '</span>')
$baujahr_A = _StringBetween($content, '<dd class="is24qa-baujahr">','</dd>')
$zimmer_A = _StringBetween($content, '<dd class="is24qa-zimmer">','</dd>')
$bezugsfrei_A = _StringBetween($content, '<dd class="is24qa-bezugsfrei-ab">' ,'</dd>')
$wohnflache_A = _StringBetween($content, '<dd class="is24qa-wohnflaeche-ca">' ,'</dd>')
$preiswarm_A =_StringBetween($content, '<strong class="is24qa-gesamtmiete">','</strong>')

If  IsArray($strase_A) Then
$strase_B = $strase_A[0]
Else
$strase_B = "/"
EndIf

If  IsArray($name_A) Then
$name_B = $name_A[0]
Else
$name_B = "/"
EndIf

If  IsArray($telefon_A) Then
$telefon_B = $telefon_A[0]
Else
$telefon_B = "/"
EndIf

If  IsArray($objekttyp_A) Then
$objekttyp_B = $objekttyp_A[0]
Else
$objekttyp_B = "/"
EndIf

If  IsArray($baujahr_A) Then
$baujahr_B = $baujahr_A[0]
Else
$baujahr_B = "/"
EndIf

If  IsArray($preiswarm_A) Then
$preiswarm_B = $preiswarm_A[0]
$preiswarm_C = StringReplace($preiswarm_B , "disabled","")
$preiswarm_D = StringReplace($preiswarm_C, "disabled","",1) ;caused some bugs
$preiswarm_E = StringReplace($preiswarm_D, "disabled","")
Else
$preiswarm_B = "/"
EndIf

If  IsArray($zimmer_A) Then
$zimmer_B = $zimmer_A[0]
$zimmer_C = StringReplace($zimmer_B , ".",",")
Else
$zimmer_B = "/"
EndIf

If  IsArray($ort_A) Then
$ort_B = $ort_A[0]
Else
$ort_B = "/"
EndIf

If  IsArray($bezugsfrei_A) Then
$bezugsfrei_B = $bezugsfrei_A[0]
Else
$bezugsfrei_B = "/"
EndIf

If  IsArray($wohnflache_A) Then
$wohnflache_B = $wohnflache_A[0]
Else
$wohnflache_B = "/"
EndIf

If  IsArray($preis_A) Then
$preis_B = $preis_A[0]
Else
$preis_B = "/"
EndIf

$aio= $name_B&";"&$strase_B&";"&$telefon_B&";"&$objekttyp_B&";"&$ort_B&";"&$baujahr_B&";"&$zimmer_C&";"&$bezugsfrei_B&";"&$wohnflache_B&";"&$preis_B&";"&$preiswarm_E&";"&$url

$sString1 = StringReplace($aio, "  ", "")
$sString2 = StringReplace($sString1, "<p>", "")
$sString3 = StringReplace($sString2, "<span>", "")
$sString4 = StringReplace($sString3, "</p>", "")
$sString5 = StringReplace($sString4, "Â", "")
$sString6 = StringReplace($sString5, '<span class="is24-operator">=</span>', "")     ;Filtering the given Data to get a proper CSV Format
$sString7 = StringReplace($sString6, "EUR", "")
$sString8 = StringReplace($sString7, "</span>","")
$sString9 = StringReplace($sString8, "Dieses Objekt im Vergleich zu anderen in ","")
$sString10 = StringReplace($sString9, "disabled",",00") ;caused bug
$sString11 = StringReplace($sString10, "ü","ü")
$sString12 = StringReplace($sString11, "ß","ß")     ;Including special characters lower case
$sString13 = StringReplace($sString12, "ä","ä")
$sString14 = StringReplace($sString13, "ö","ö")
$sString15 = StringReplace($sString14, "Ãœ","Ü")    ;Including special characters upper case
$sString16 = StringReplace($sString15, "Ä","Ä")
$sString17 = StringReplace($sString16, "Ö","Ö")
$sStringfinal = StringReplace($sString17, @CRLF, "")

FileWrite ( "Immobilien.csv", $sStringfinal & @CRLF )

ToolTip("Stage 2 of 2, Processing: " & $m + 1 & " of " & Ubound($Array),0,0)

Sleep(100)
endfunc

Func GetNumberedLinks();Get the number that follows http://www.immobilienscout24.de/expose/ in the current html
        ToolTip("Stage 1 of 2, Processing: " & $k - 1 & " of " & $pages,0,0)
        $HTML = _INetGetSource($url)
        $j = 1
        Do
            $1 = StringInStr($HTML, "/expose/", 0, $j)
            $2 = StringTrimLeft($HTML, $1)
            $3 = StringReplace($2, '"', ";;")
            $4 = StringSplit($3, ";;", 1)
            $5 = $4[1]
            $6 = StringMid($5, 8, 8)
            If $6 >= 1 Then;Needed to add this if statement because I was getting extra characters at the end of the finalarray
                _ArrayAdd($Array,$6,1)
            EndIf
            $j += 1
        Until $1 = 0;Stop loop when there are no more instances of "/expose/"
        $Array = _ArrayUnique($Array,0,0,0,0)
EndFunc

Func NumberOfPages()
    $filtered = StringRegExp($pagecount[0], "\d{2}", $STR_REGEXPARRAYMATCH)
    $link0 = StringReplace($url, "http://www.immobilienscout24.de/Suche/S-T/", "")
    $link1 = StringReplace($url, $link0, "")
    $link2 = "/" & $link0
    $pages = $filtered[0]
EndFunc   ;==>NumberOfPages

Get Scite to add a popup when you use a 3rd party UDF -> http://www.autoitscript.com/autoit3/scite/docs/SciTE4AutoIt3/user-calltip-manager.html

Share this post


Link to post
Share on other sites
andrewz

 

Looks like I got it to work. Test it and let me know if the data is correct.

;~ #NoTrayIcon
#include <Inet.au3>
#include <Array.au3>
#include <String.au3>
#include <AutoItConstants.au3>
#include <MsgBoxConstants.au3>

Global $Array[0];Array with all the link number info (ie http://www.immobilienscout24.de/expose/'77947839' <- this number
Global $FinalArray[0]
Global $url = "http://www.immobilienscout24.de/Suche/S-T/Wohnung-Miete/Bayern/Muenchen/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/true?enteredFrom=result_list"
Global $content = _INetGetSource($url)
Global $string_A = _StringBetween($content, '<li data-item="result" data-obid=', 'id="result')
Global $pagecount = _StringBetween($content, '<li><span>&hellip;</span></li>', '<span class="smallPager">')
Global $pages;<-number of websites to scan
FileDelete(@ScriptDir & "\Immobilien.csv")

NumberOfPages()

For $k = 2 To 20
    GetNumberedLinks()
    _ArrayConcatenate($FinalArray,$Array)
    $url = "http://www.immobilienscout24.de/Suche/S-T/P-" & $k & "/Wohnung-Miete/Bayern/Muenchen/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/true?enteredFrom=result_list"
    $HTML = _INetGetSource($url)
Next

;~ $number =UBound ( $string_A ,$UBOUND_ROWS)
;~ $timesran = -1
;~ $i=0
;~ $numberarray = $number
;~ $numberarray -= 1

For $m = 0 To Ubound($Array)
$id= $FinalArray[$m]
$urlscout = "http://www.immobilienscout24.de/expose/"&$id
export ()
Next

func export ()
$url = $urlscout

If StringInStr($url,"http://www.immobilienscout24.de/expose") = false Then
MsgBox(48,"Error","URL not specified. Please make sure you enter the URL !")
Exit
Else
EndIf

If FileExists("Immobilien.csv") =false Then
FileWrite("Immobilien.csv","Name;Adresse;Telefon;Objekttyp;Ort;Baujahr;Zimmer;Bezugfrei ab;Wfl./ qm;Kaltmiete;Warmmiete;Scout- ID"& @CRLF)
EndIf

$content = _INetGetSource($url)
$name_A = _StringBetween($content, '<span data-qa="contactName" class="font-bold">', '</span>')
$preis_A = _StringBetween($content, '<span class="is24-operator">', '<div class="sourcecode section">')
$strase_A = _StringBetween($content, '<strong class="font-standard">' , '</strong><br/>')
$telefon_A = _StringBetween($content, '<div class="is24-phone-number hide">' ,'</div>')
$objekttyp_A = _StringBetween($content, '<dd class="is24qa-wohnungstyp">' ,'</dd>')
$ort_A = _StringBetween($content, '<span id="quickCheckHeader" class="is24-f">' , '</span>')
$baujahr_A = _StringBetween($content, '<dd class="is24qa-baujahr">','</dd>')
$zimmer_A = _StringBetween($content, '<dd class="is24qa-zimmer">','</dd>')
$bezugsfrei_A = _StringBetween($content, '<dd class="is24qa-bezugsfrei-ab">' ,'</dd>')
$wohnflache_A = _StringBetween($content, '<dd class="is24qa-wohnflaeche-ca">' ,'</dd>')
$preiswarm_A =_StringBetween($content, '<strong class="is24qa-gesamtmiete">','</strong>')

If  IsArray($strase_A) Then
$strase_B = $strase_A[0]
Else
$strase_B = "/"
EndIf

If  IsArray($name_A) Then
$name_B = $name_A[0]
Else
$name_B = "/"
EndIf

If  IsArray($telefon_A) Then
$telefon_B = $telefon_A[0]
Else
$telefon_B = "/"
EndIf

If  IsArray($objekttyp_A) Then
$objekttyp_B = $objekttyp_A[0]
Else
$objekttyp_B = "/"
EndIf

If  IsArray($baujahr_A) Then
$baujahr_B = $baujahr_A[0]
Else
$baujahr_B = "/"
EndIf

If  IsArray($preiswarm_A) Then
$preiswarm_B = $preiswarm_A[0]
$preiswarm_C = StringReplace($preiswarm_B , "disabled","")
$preiswarm_D = StringReplace($preiswarm_C, "disabled","",1) ;caused some bugs
$preiswarm_E = StringReplace($preiswarm_D, "disabled","")
Else
$preiswarm_B = "/"
EndIf

If  IsArray($zimmer_A) Then
$zimmer_B = $zimmer_A[0]
$zimmer_C = StringReplace($zimmer_B , ".",",")
Else
$zimmer_B = "/"
EndIf

If  IsArray($ort_A) Then
$ort_B = $ort_A[0]
Else
$ort_B = "/"
EndIf

If  IsArray($bezugsfrei_A) Then
$bezugsfrei_B = $bezugsfrei_A[0]
Else
$bezugsfrei_B = "/"
EndIf

If  IsArray($wohnflache_A) Then
$wohnflache_B = $wohnflache_A[0]
Else
$wohnflache_B = "/"
EndIf

If  IsArray($preis_A) Then
$preis_B = $preis_A[0]
Else
$preis_B = "/"
EndIf

$aio= $name_B&";"&$strase_B&";"&$telefon_B&";"&$objekttyp_B&";"&$ort_B&";"&$baujahr_B&";"&$zimmer_C&";"&$bezugsfrei_B&";"&$wohnflache_B&";"&$preis_B&";"&$preiswarm_E&";"&$url

$sString1 = StringReplace($aio, "  ", "")
$sString2 = StringReplace($sString1, "<p>", "")
$sString3 = StringReplace($sString2, "<span>", "")
$sString4 = StringReplace($sString3, "</p>", "")
$sString5 = StringReplace($sString4, "Â", "")
$sString6 = StringReplace($sString5, '<span class="is24-operator">=</span>', "")     ;Filtering the given Data to get a proper CSV Format
$sString7 = StringReplace($sString6, "EUR", "")
$sString8 = StringReplace($sString7, "</span>","")
$sString9 = StringReplace($sString8, "Dieses Objekt im Vergleich zu anderen in ","")
$sString10 = StringReplace($sString9, "disabled",",00") ;caused bug
$sString11 = StringReplace($sString10, "ü","ü")
$sString12 = StringReplace($sString11, "ß","ß")     ;Including special characters lower case
$sString13 = StringReplace($sString12, "ä","ä")
$sString14 = StringReplace($sString13, "ö","ö")
$sString15 = StringReplace($sString14, "Ãœ","Ü")    ;Including special characters upper case
$sString16 = StringReplace($sString15, "Ä","Ä")
$sString17 = StringReplace($sString16, "Ö","Ö")
$sStringfinal = StringReplace($sString17, @CRLF, "")

FileWrite ( "Immobilien.csv", $sStringfinal & @CRLF )

ToolTip("Stage 2 of 2, Processing: " & $m + 1 & " of " & Ubound($Array),0,0)

Sleep(100)
endfunc

Func GetNumberedLinks();Get the number that follows http://www.immobilienscout24.de/expose/ in the current html
        ToolTip("Stage 1 of 2, Processing: " & $k - 1 & " of " & $pages,0,0)
        $HTML = _INetGetSource($url)
        $j = 1
        Do
            $1 = StringInStr($HTML, "/expose/", 0, $j)
            $2 = StringTrimLeft($HTML, $1)
            $3 = StringReplace($2, '"', ";;")
            $4 = StringSplit($3, ";;", 1)
            $5 = $4[1]
            $6 = StringMid($5, 8, 8)
            If $6 >= 1 Then;Needed to add this if statement because I was getting extra characters at the end of the finalarray
                _ArrayAdd($Array,$6,1)
            EndIf
            $j += 1
        Until $1 = 0;Stop loop when there are no more instances of "/expose/"
        $Array = _ArrayUnique($Array,0,0,0,0)
EndFunc

Func NumberOfPages()
    $filtered = StringRegExp($pagecount[0], "\d{2}", $STR_REGEXPARRAYMATCH)
    $link0 = StringReplace($url, "http://www.immobilienscout24.de/Suche/S-T/", "")
    $link1 = StringReplace($url, $link0, "")
    $link2 = "/" & $link0
    $pages = $filtered[0]
EndFunc   ;==>NumberOfPages

 

Wow thanks :P Testing it right now and will report back in a few minutes!

Holy ... it works perfectly! So basically it first saves all links and then starts the import ... genius :D, I didnt think of doing it this way ^^

Really nicely solved + looks clean now .

Is there any way I can thank you? Like a small donation (Well I dont have access to a creditcard yet, so it's gonna be a Paysafecard or iTunes/Google Play -Card)?

Edited by andrewz

Share this post


Link to post
Share on other sites
andrewz

 

Couldn't this be done using funcs like in this pseudocode ?

For $var = 1 to $filtered      ; total amount of pages
   $currenturl = _secondScript($var)    ; generate $currenturl
   $data = _firstScript($currenturl)      ; grab data from $currenturl
Next

 

Nope somehow not, that was my first attempt to fix this problem, but it messed up some arrays or variables, dunno why.

Share this post


Link to post
Share on other sites
computergroove

Wow thanks :P Testing it right now and will report back in a few minutes!

Holy ... it works perfectly! So basically it first saves all links and then starts the import ... genius :D, I didnt think of doing it this way ^^

Really nicely solved + looks clean now .

Is there any way I can thank you? Like a small donation (Well I dont have access to a creditcard yet, so it's gonna be a Paysafecard or iTunes/Google Play -Card)?

Yes please lol. I spent all day working on it. Lucky for you I have some free time and I like to code. PM me if you want.


Get Scite to add a popup when you use a 3rd party UDF -> http://www.autoitscript.com/autoit3/scite/docs/SciTE4AutoIt3/user-calltip-manager.html

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Similar Content

    • nooneclose
      By nooneclose
      I want to check some Excel data against data on a website in Chrome. I use Chrome because the site I use does not function properly in Internet Explorer or Firefox. I know how to do the Excel stuff I just can not figure out how to send to Chrome, let alone check to see if the data matches or not. I am also having trouble finding any help online while searching for Chrome functions for Autoit. I have a Chrome UDF installed but I still can not figure out how to get my code to properly function. (I am not posting code because I am  sure my code isn't right, to begin with)
      As usual, any and all help would be greatly appreciated. 
    • ohaya
      By ohaya
      Hi,
      I have a situation where I need to make one request to get a kind of pre-logon page, then click a link on that pre-logon page, which then returns a login form page, and then fill in information in the form page and submit it.
      How can I handle that using AutoIT?
      I can get the AutoIT script to tab to the link, then send an ENTER, which then causes the FORM page to appear in IE.  However, I cannot seem to get the form object from the FORM page (_IEFormGetObjByName fails when it searches for the form).  Why is that, and how can I get the AutoIT to process the FORM page?
      Thanks,
      Jim
    • rm4453
      By rm4453
      Hello,
       
      I am currently writing a program that parses a massive table from a website, and need a way to add a progress bar while parsing.
      I am currently using the function _IETableWriteToArray($oObj, True) to parse the array. I need the progress bar to update as the table is parsed, not just at the end of the parsing.
      Any help at all would be very much appreciated!
       
      *EDIT --> The array I am left with after parsing is $array[0-50000][16]
    • JuanFelipe
      By JuanFelipe
      Hello friends, I have a simple question, can I have a variable that is not reset, even if my program is closed?
      That is, I want to make a program that can only be opened once, have a variable $ ini = 0, when it is opened for the first and only time it becomes $ ini = 1, so that it can not use the program a second time, as if it was a test software.
      I do not want the variable to depend on a reading of an external file that is easily manipulated. Thank you.
    • nooneclose
      By nooneclose
      My program has to first search for names in Column D that do not match up with column C. I got that search to work using arrays. It was slow and I could not figure out how to delete them so I just manually put coded the names that do not belong. I found their cell location but I do not know how to store that location and delete it.
      This is what I have so far.
      Local $NameToDelete1[6]  = _Excel_RangeFind($OpenWorkbook, "Smith, Bill") _ArrayDisplay($NameToDelete1, "Excel UDF: _Excel_RangeFind Example 1", "", 0, "|", "Sheet|Name|Cell|Value|Formula|Comment") _Excel_RangeDelete($OpenWorkbook.ActiveSheet, $NameToDelete1[2], $xlShiftUp)  
      Please help, I wanted to have this program done yesterday but I did not see this problem until yesterday. 
×