Jump to content
Sign in to follow this  
ericnail

Detecting New Posts

Recommended Posts

ericnail

I am creating a script to check craigslist for new posts based on my search parameters.

I want it to check for new posts and then if any are new then run a function.

Currently I have this:

#include <INet.au3>
Global $postTitle, $postDate, $postPrice, $postLink, $postNumber, $updateTimer, $count = 0
Global $intervalTime = 900000 ;15 Minutes
Global $stackQueue[1] = [0] ;Queue for found ads
Global $stackOverflow[1] = [0] ;Queue for matched ads
Dim $iStack[1][5]
$iStack[0][0] = '<font size=5>Title</font>'
$iStack[0][1] = '<font size=5>Price</font>'
$iStack[0][2] = '<font size=5>Date</font>'
$iStack[0][3] = '<font size=5>Phone Number</font>'
$iStack[0][4] = '<font size=5>URL</font>'
$check_numbers = 0
 
 
GetSearchResults("iPhone 4", "150", "250", 1)
Func GetSearchResults($a, $b, $c, $d = 1)
fillStack($a, $b, $c)
$html = _ArrayToHTML($iStack)
$html = '<html>' & @CRLF & $html & '</html>'
$html = StringReplace($html, "Title", "<b>Title</b>")
$html = StringReplace($html, "Date", "<b>Date</b>")
$html = StringReplace($html, "Price", "<b>Price</b>")
$html = StringReplace($html, "URL", "<b>URL</b>")
$html = StringReplace($html, 'http://atlanta.craigslist.org/', '<a href="http://atlanta.craigslist.org/')
$html = StringReplace($html, '.html', '.html">Link</a>')
If $d = 1 Then FileSaveAndOpen("index.html", $html)
EndFunc
Func fillStack($iQuery, $iMin, $iMax)
 
Local $iData[4][1]
$iQuery = StringReplace($iQuery, " ", "+")
$iURL = "http://atlanta.craigslist.org/search/sss?maxAsk="&$iMax&"&amp;minAsk="&$iMin&"&amp;query="&$iQuery&"&amp;srchType=A"
$source = _INetGetSource($iURL)
$postTitle = StringRegExp($source, '<a href="http://atlanta.craigslist.org/[^0-9][^0-9][^0-9]/[^0-9][^0-9][^0-9]/...........html">(.*?)-</a>', 3)
$postDate = StringRegExp($source, ";</span>\n   [^0-9][^0-9][^0-9] [0-9][0-9] - <a", 3)
$postPrice = StringRegExp($source, '\n[^a-z][^a-z][^a-z](.*?)href="/[^0-9][^0-9][^0-9]/"', 3)
$postLink = StringRegExp($source, '<a href="http://atlanta.craigslist.org/[^0-9][^0-9][^0-9]/[^0-9][^0-9][^0-9]/...........html">', 3)
ReDim $iData[5][UBound($postTitle)]
 
For $i = 0 to UBound($postTitle) -1 Step 1
  $iData[0][$i] = StringTrimRight(StringTrimLeft($postTitle[$i], 0), 0)
Next
For $i = 0 to UBound($postDate) -1 Step 1
  $iData[1][$i] = StringTrimRight(StringTrimLeft($postDate[$i], 17), 5)
Next
For $i = 0 to UBound($postPrice) -1 Step 1
  $iData[2][$i] = "$"&StringTrimLeft(StringLeft($postPrice[$i], 5), 2)
Next
 
For $i = 0 to UBound($postLink) -1 Step 1
  $iData[4][$i] = StringTrimRight(StringTrimLeft($postLink[$i], 9), 2)
Next
 
Local $iTemp_a = UBound($postPrice)-1
Local $iTemp_b = UBound($postTitle)-1
If $check_numbers Then ProgressOn("Eric's Craigslist App", "Collecting Phone Numbers...")
For $i = 0 to $iTemp_b Step 1
  If $check_numbers Then ProgressSet(($i / $iTemp_b) * 100, Round(($i / $iTemp_b) * 100) & "%")
  If $i > $iTemp_a Then ExitLoop
  If StringInStr($postTitle, "want") Then ContinueLoop
  If StringInStr($postTitle, "trade") Then ContinueLoop
  If $postTitle = "" Then ContinueLoop
  AddToStack($iStack, $iData[0][$i], $iData[2][$i], $iData[1][$i], $iData[4][$i], $iQuery)
Next
If $check_numbers Then ProgressOff()
 
Global $Count = 0
 
EndFunc
Func AddToStack(ByRef $iStack, $iTitle, $iPrice, $iDate, $iLink, $iQuery)
 
Local $iNumber = "-"
 
$iQuery = StringReplace($iQuery, "+", " ")
 
If Not StringInStr($iTitle, $iQuery) Then Return
 
;~  If $History Then
;~   If ManageArchive("Check", $iLink) Then Return
;~   ManageArchive("Write", $iLink)
;~  EndIf
 
$count+= 1
 
ReDim $iStack[$count+1][5]
 
If $check_numbers Then
  $src = _INetGetSource($iLink)
  $iTemp = StringRegExp($src, "[0-9][0-9][0-9][^0-9][0-9][0-9][0-9][^0-9][0-9][0-9][0-9][0-9]", 3)
  If IsArray($iTemp) Then
   $iTemp = StringRegExpReplace($iTemp[0], '(\d{3}).(\d{3}).(\d{4})', ' $1-$2-$3 ')
   $iNumber = $iTemp
  Else
   $iTemp = StringRegExpReplace($iTemp, '(\d{3}).(\d{3}).(\d{4})', ' $1-$2-$3 ')
   $iNumber = $iTemp
  EndIf
 
  If $iNumber = 1 Then $iNumber = "-"
EndIf
 
$iStack[$count][0] = $iTitle
$iStack[$count][1] = $iPrice
$iStack[$count][2] = $iDate
$iStack[$count][3] = $iNumber
$iStack[$count][4] = $iLink
 
EndFunc
Func _ArrayToHTML(Const ByRef $avArray, $attrib = 'border="2"', $iStart = 0, $iEnd = 0)
If Not IsArray($avArray) Then Return SetError(1, 0, "")
If UBound($avArray, 0) <> 2 Then Return SetError(2, 0, "")
Local $sResult, $iUBound = UBound($avArray) - 1
Local $row, $sDelimCol = "</td>" & @CRLF, $sDelimRow = '</tr>' & @CRLF
; Bounds checking
If $iEnd < 1 Or $iEnd > $iUBound Then $iEnd = $iUBound
If $iStart < 0 Then $iStart = 0
If $iStart > $iEnd Then Return SetError(3, 0, "")
$sResult = '<table ' & $attrib & '>' & @CRLF
 
; Combine
For $i = $iStart To $iEnd ; rows
    $row = '<tr>' & @CRLF
    For $j = 0 To UBound($avArray,2) - 1 ; columns
        If $j <> 0 Then
   $row &= '<td><font size=2>' & '<center>' & $avArray[$i][$j] & $sDelimCol & '</center>' & '</font>'
  Else
   $row &= '<td><font size=2>' & $avArray[$i][$j] & $sDelimCol & '</font>'
  EndIf
Next
    $sResult &= $row & $sDelimRow
Next
 
Return $sResult & '</table>' & @CRLF
EndFunc
 
Func FileSaveAndOpen($file, $content)
$file = @TempDir & '\' & $file
FileDelete($file)
FileWrite($file, $content)
ShellExecute($file)
EndFunc
 
Func ManageArchive($a, $b)
 
If $a = "Write" Then
  IniWrite("archive.ini", "archived", "viewed", IniRead("archive.ini", "archived", "viewed", "") & "|" & $b)
EndIf
 
If $a = "Remove" Then
  IniWrite("archive.ini", "archived", "viewed", StringReplace(IniRead("archive.ini", "archived", "viewed", ""), $b & ",", ""))
EndIf
 
If $a = "Check" Then
  $iTemp = IniRead("archive.ini", "archived", "viewed", "")
  $iTemp = StringSplit($iTemp, "|")
  For $i = 1 to $iTemp[0] Step 1
   If $b = $iTemp[$i] Then Return True
  Next
 
  Return False
EndIf
EndFunc

Which will search for the terms and find specified info and then make and open an HTML File. I want to be able to make it search for only new posts.

I cannot come up with a way to do this in my head. Maybe there is a simple solution but I have yet to figure it out.

Any help here would be great!

Thanks,

Eric

Share this post


Link to post
Share on other sites
Beege

When you say "new" do you mean timeline (it was just posted 10 min ago) new or your script has not ever seen this post before "new"?

Edited by Beege

Share this post


Link to post
Share on other sites
ericnail

When you say "new" do you mean timeline (it was just posted 10 min ago) new or your script has not ever seen this post before "new"?

Hey,

I mean Timeline.

Share this post


Link to post
Share on other sites
Beege

I looked of the html source for the link in the example and dont see any dates on the search page. I dont see any other way of doing it other than downloading the source of the links you find, then pulling the dates of them. Btw nice script. I like the html table thing you got going. :graduated:

Edit: Looking at the source's of the links deeper, it should really be cake compared to the work you have already done. Of all the links I looked at theres only one date on the page and that date is the same format for all of them. your clearly good with regexpressions so I dont think it will be at all. Unless im missing something else your needing.

Edited by Beege

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×