TryMakeLifeEasier

Scrape Multiple Page Table from Website

16 posts in this topic

Having a problem to set up what I'm trying to do:

I want to scrape the data from a soccer stats webpage http://us.soccerway.com/teams/venezuela/margarita/29769/

I want to scrape specifically the data within the small mutli-page table of matches in the middle of the page (see image attached)

 

I think that _IETableGetCollection and _IETableWriteToArray should be where i should go....however I'm not clear on how to harness the table since I can't find it in the page source code. 

I'm super new at all of this....so any help is most appreciated!!

Here is what I have so far:

#include <Array.au3>
#include <IE.au3>

$oIE =
_IECreate("http://us.soccerway.com/teams/venezuela/margarita/29769/")

;Not sure how to actually isolate the table I want in the pagesource
$Table =
_IETableGetCollection("http://us.soccerway.com/teams/venezuela/margarita/29769/")

$aTableData = _IETableWriteToArray($Table)

 

 

 

TabletoGet.png

Share this post


Link to post
Share on other sites



That worked!  Could you explain to me how you were able to identify the object needed for the table?

The table that the data was just scraped from is the last page of the table.

I would like to be able to scrape all the preceding pages as well.  

Not sure how to do that as of now!

 

Share this post


Link to post
Share on other sites

Ok looking through the source code I am able to see where in the code you found the id for that table.

But still I don't understand how I'd be able to get all the pages of that table.

 

Looking forward to anyone's reply!

 

 

Share this post


Link to post
Share on other sites

I have successfully scraped some of the data from a table on a webpage

However, my problem is the table has multiple pages.

I don't know how I would be able to grab all of the data through all of the pages

Here is the code

#include <Array.au3>
#include <IE.au3>

Local $oIE = _IECreate("http://us.soccerway.com/teams/venezuela/margarita/29769/")

$oDiv = _IEGetObjById($oIE, 'page_team_1_block_team_matches_summary_7')
$oTable = _IETableGetCollection($oDiv, 0)

$aMatches = _IETableWriteToArray($oTable, True)
_ArrayDisplay($aMatches, 'Matches')

Share this post


Link to post
Share on other sites

Please stick to one thread.

Share this post


Link to post
Share on other sites

Sorry...wasn't sure if my last thread went stale.  

Share this post


Link to post
Share on other sites

There are links (Next & Previous) for traversing the pages. Have you tried to click them to switch pages?

Share this post


Link to post
Share on other sites

yes I could manually do that

However....I'm trying to automate that.

Not sure how to automate switching between pages

Share this post


Link to post
Share on other sites

#10 ·  Posted

1 hour ago, TryMakeLifeEasier said:

Not sure how to automate switching between pages

You would use the appropriate _IE function to click the desired link. :-P

Take a look at _IELinkClickByText.

Share this post


Link to post
Share on other sites

#11 ·  Posted

Ok.....the "link" that I need to click isn't the normal kind of links I usually see  

it is some sort of link without a href...

 

I took a stab at trying to Click by the corresponding text associated with it (like it were a string)

_IELinkClickByText($oIE,"« Previous")

but that didn't work

Something specific need to be done for this specific type of "link"?

Share this post


Link to post
Share on other sites

#12 ·  Posted

You should be able to do something like this --

$oPrev = _IEGetObjById($oIE, "page_team_1_block_team_matches_summary_7_previous")
_IEAction($oPrev, 'click')
_IELoadWait($oIE)

However, this site doesn't appear to be working correctly in IE, at least for me.

Share this post


Link to post
Share on other sites

#14 ·  Posted

Unfortunately, this still uses multiple pages. I wonder if there's a way to trick it into displaying all matches at one time?

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now