Jump to content
Sign in to follow this  

Craigslist Section to Array

Recommended Posts

#include <IE.au3>
#include <INet.au3>
#include <Array.au3>

Const $CRAIGSLISTURL = "craigslist.org"

$vgm = _CLSectionToArray("albany", "vgm", 1)
If Not @error Then
    _ArrayDisplay($vgm, "Craigslist Section")

Func _CLSectionToArray($sCity, $sSection, $iPages = 1)
    Local $id = 0, $sIndex, $sURL_Source
    Local $asURL_Parse, $asURL_Listings[1][2]
    Local $sURL = "http://" & $sCity & Chr(46) & $CRAIGSLISTURL & Chr(47) & $sSection & Chr(47)

    For $i = 0 To $iPages - 1
        If $i >= 1 Then
            $id += 100
            $sIndex = "index" & $id & ".html"
            $sIndex = "index.html"
        $sURL_Source &= _INetGetSource($sURL & $sIndex)

    $asURL_Parse = StringRegExp($sURL_Source, '<a href="' & $sURL & '(.*?)</a>', 3)
    If @error Then
        Return 0

    For $i = 0 To UBound($asURL_Parse) - 1
        ReDim $asURL_Listings[UBound($asURL_Listings) + 1][2]
        $asSplit = StringSplit($asURL_Parse[$i], '">', 3)
        $asURL_Listings[$i + 1][0] = $asSplit[0]
        $asURL_Listings[$i + 1][1] = $asSplit[1]
    $asURL_Listings[0][0] = (UBound($asURL_Listings) - 1)
    Return $asURL_Listings
EndFunc   ;==>_CLSectionToArray

Function with example. Simple enough.

URL for Craigslist - go to the section you want to read and look at the URL.

_CLSectionToArray("City", "Section", "Pages - default is 1")

So http://phoenix.craigslist.org/wvl/vgm/ would be

_CLSectionToArray("phoenix", "wvl/vgm")

If you wanted the first few pages

_CLSectionToArray("phoenix", "wvl/vgm", 3)

And it'll return a 2 Dimensional array with the page ID in the first field and the description in the second.

Edit: I only designed and tested it in sections I needed it for, so no promises in other classified sections heh

Edit 2: Doesn't work on city's with sub sections, i'll mess around with that. You just need to search each sub section.

Edited by ZacUSNYR

Share this post

Link to post
Share on other sites

I see that using the example I gave (never tested it just chose a city at random lol).

Using Albany, VGM it works fine. Under Phoenix/boston/other large cities it actually sub categorizes.

I'll mess around with that heh

Share this post

Link to post
Share on other sites

Been designing these functions to work as declared. The subsections you just need to declare in the section.


Would be

_CLSectionToArray("phoenix", "wvl/vgm", 1)

Not as dynamic but it works :)

Share this post

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Create New...