Scrape Multiple Page Table from Website

[TryMakeLifeEasier]

Having a problem to set up what I'm trying to do:

I want to scrape the data from a soccer stats webpage http://us.soccerway.com/teams/venezuela/margarita/29769/

I want to scrape specifically the data within the small mutli-page table of matches in the middle of the page (see image attached)

 

I think that _IETableGetCollection and _IETableWriteToArray should be where i should go....however I'm not clear on how to harness the table since I can't find it in the page source code. 

I'm super new at all of this....so any help is most appreciated!!

Here is what I have so far:

#include <Array.au3>
#include <IE.au3>

$oIE =
_IECreate("http://us.soccerway.com/teams/venezuela/margarita/29769/")

;Not sure how to actually isolate the table I want in the pagesource
$Table =
_IETableGetCollection("http://us.soccerway.com/teams/venezuela/margarita/29769/")

$aTableData = _IETableWriteToArray($Table)

 

 

 

[Danp2]

Try this --

#include <Array.au3>
#include <IE.au3>

Local $oIE = _IECreate("http://us.soccerway.com/teams/venezuela/margarita/29769/")

$oDiv = _IEGetObjById($oIE, 'page_team_1_block_team_matches_summary_7')
$oTable = _IETableGetCollection($oDiv, 0)

$aMatches = _IETableWriteToArray($oTable, True)
_ArrayDisplay($aMatches, 'Matches')

 

[TryMakeLifeEasier]

That worked!  Could you explain to me how you were able to identify the object needed for the table?

The table that the data was just scraped from is the last page of the table.

I would like to be able to scrape all the preceding pages as well.  

Not sure how to do that as of now!

 

[TryMakeLifeEasier]

Ok looking through the source code I am able to see where in the code you found the id for that table.

But still I don't understand how I'd be able to get all the pages of that table.

 

Looking forward to anyone's reply!

 

 

[TryMakeLifeEasier]

I have successfully scraped some of the data from a table on a webpage

However, my problem is the table has multiple pages.

I don't know how I would be able to grab all of the data through all of the pages

Here is the code

#include <Array.au3>
#include <IE.au3>

Local $oIE = _IECreate("http://us.soccerway.com/teams/venezuela/margarita/29769/")

$oDiv = _IEGetObjById($oIE, 'page_team_1_block_team_matches_summary_7')
$oTable = _IETableGetCollection($oDiv, 0)

$aMatches = _IETableWriteToArray($oTable, True)
_ArrayDisplay($aMatches, 'Matches')

[Danp2]

Please stick to one thread.

[TryMakeLifeEasier]

Sorry...wasn't sure if my last thread went stale.  

[Danp2]

There are links (Next & Previous) for traversing the pages. Have you tried to click them to switch pages?

[TryMakeLifeEasier]

yes I could manually do that

However....I'm trying to automate that.

Not sure how to automate switching between pages

[Danp2]

1 hour ago, TryMakeLifeEasier said:

Not sure how to automate switching between pages

You would use the appropriate _IE function to click the desired link. :-P

Take a look at _IELinkClickByText.

[TryMakeLifeEasier]

Ok.....the "link" that I need to click isn't the normal kind of links I usually see  

it is some sort of link without a href...

 

I took a stab at trying to Click by the corresponding text associated with it (like it were a string)

_IELinkClickByText($oIE,"« Previous")

but that didn't work

Something specific need to be done for this specific type of "link"?

[Danp2]

You should be able to do something like this --

$oPrev = _IEGetObjById($oIE, "page_team_1_block_team_matches_summary_7_previous")
_IEAction($oPrev, 'click')
_IELoadWait($oIE)

However, this site doesn't appear to be working correctly in IE, at least for me.

[junkew]

I assume this is much easier on 1 page

http://us.soccerway.com/teams/venezuela/margarita/29769/matches/

 

[Danp2]

Unfortunately, this still uses multiple pages. I wonder if there's a way to trick it into displaying all matches at one time?

[junkew]

yes, the trick is to pay them for an xml api feed 

 

[VaishnaviBUtpat]

you can use form elements....

for example,

 

Local $oIE = _IECreate("www.google.com",1,1,1,1)
Sleep(2000)_IELoadWait($oIE)
Sleep(2000)
$o_form = _IEFormGetObjByName($oIE, "f")
$element = _IEFormElementGetObjByName($o_form,"q")
_IEFormElementSetValue($element,"INDIA")

You can get the reference objects using F12