TryMakeLifeEasier Posted September 6, 2017 Share Posted September 6, 2017 Having a problem to set up what I'm trying to do: I want to scrape the data from a soccer stats webpage http://us.soccerway.com/teams/venezuela/margarita/29769/ I want to scrape specifically the data within the small mutli-page table of matches in the middle of the page (see image attached) I think that _IETableGetCollection and _IETableWriteToArray should be where i should go....however I'm not clear on how to harness the table since I can't find it in the page source code. I'm super new at all of this....so any help is most appreciated!! Here is what I have so far: #include <Array.au3> #include <IE.au3> $oIE = _IECreate("http://us.soccerway.com/teams/venezuela/margarita/29769/") ;Not sure how to actually isolate the table I want in the pagesource $Table = _IETableGetCollection("http://us.soccerway.com/teams/venezuela/margarita/29769/") $aTableData = _IETableWriteToArray($Table) Link to comment Share on other sites More sharing options...
Danp2 Posted September 6, 2017 Share Posted September 6, 2017 Try this -- #include <Array.au3> #include <IE.au3> Local $oIE = _IECreate("http://us.soccerway.com/teams/venezuela/margarita/29769/") $oDiv = _IEGetObjById($oIE, 'page_team_1_block_team_matches_summary_7') $oTable = _IETableGetCollection($oDiv, 0) $aMatches = _IETableWriteToArray($oTable, True) _ArrayDisplay($aMatches, 'Matches') Latest Webdriver UDF Release Webdriver Wiki FAQs Link to comment Share on other sites More sharing options...
TryMakeLifeEasier Posted September 7, 2017 Author Share Posted September 7, 2017 That worked! Could you explain to me how you were able to identify the object needed for the table? The table that the data was just scraped from is the last page of the table. I would like to be able to scrape all the preceding pages as well. Not sure how to do that as of now! Link to comment Share on other sites More sharing options...
TryMakeLifeEasier Posted September 7, 2017 Author Share Posted September 7, 2017 Ok looking through the source code I am able to see where in the code you found the id for that table. But still I don't understand how I'd be able to get all the pages of that table. Looking forward to anyone's reply! Link to comment Share on other sites More sharing options...
TryMakeLifeEasier Posted September 7, 2017 Author Share Posted September 7, 2017 I have successfully scraped some of the data from a table on a webpage However, my problem is the table has multiple pages. I don't know how I would be able to grab all of the data through all of the pages Here is the code #include <Array.au3> #include <IE.au3> Local $oIE = _IECreate("http://us.soccerway.com/teams/venezuela/margarita/29769/") $oDiv = _IEGetObjById($oIE, 'page_team_1_block_team_matches_summary_7') $oTable = _IETableGetCollection($oDiv, 0) $aMatches = _IETableWriteToArray($oTable, True) _ArrayDisplay($aMatches, 'Matches') Link to comment Share on other sites More sharing options...
Danp2 Posted September 7, 2017 Share Posted September 7, 2017 Please stick to one thread. Latest Webdriver UDF Release Webdriver Wiki FAQs Link to comment Share on other sites More sharing options...
TryMakeLifeEasier Posted September 7, 2017 Author Share Posted September 7, 2017 Sorry...wasn't sure if my last thread went stale. Link to comment Share on other sites More sharing options...
Danp2 Posted September 7, 2017 Share Posted September 7, 2017 There are links (Next & Previous) for traversing the pages. Have you tried to click them to switch pages? Latest Webdriver UDF Release Webdriver Wiki FAQs Link to comment Share on other sites More sharing options...
TryMakeLifeEasier Posted September 7, 2017 Author Share Posted September 7, 2017 yes I could manually do that However....I'm trying to automate that. Not sure how to automate switching between pages Link to comment Share on other sites More sharing options...
Danp2 Posted September 7, 2017 Share Posted September 7, 2017 1 hour ago, TryMakeLifeEasier said: Not sure how to automate switching between pages You would use the appropriate _IE function to click the desired link. :-P Take a look at _IELinkClickByText. Latest Webdriver UDF Release Webdriver Wiki FAQs Link to comment Share on other sites More sharing options...
TryMakeLifeEasier Posted September 7, 2017 Author Share Posted September 7, 2017 Ok.....the "link" that I need to click isn't the normal kind of links I usually see it is some sort of link without a href... I took a stab at trying to Click by the corresponding text associated with it (like it were a string) _IELinkClickByText($oIE,"« Previous") but that didn't work Something specific need to be done for this specific type of "link"? Link to comment Share on other sites More sharing options...
Danp2 Posted September 7, 2017 Share Posted September 7, 2017 You should be able to do something like this -- $oPrev = _IEGetObjById($oIE, "page_team_1_block_team_matches_summary_7_previous") _IEAction($oPrev, 'click') _IELoadWait($oIE) However, this site doesn't appear to be working correctly in IE, at least for me. Latest Webdriver UDF Release Webdriver Wiki FAQs Link to comment Share on other sites More sharing options...
junkew Posted September 7, 2017 Share Posted September 7, 2017 I assume this is much easier on 1 page http://us.soccerway.com/teams/venezuela/margarita/29769/matches/ FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets Link to comment Share on other sites More sharing options...
Danp2 Posted September 7, 2017 Share Posted September 7, 2017 Unfortunately, this still uses multiple pages. I wonder if there's a way to trick it into displaying all matches at one time? Latest Webdriver UDF Release Webdriver Wiki FAQs Link to comment Share on other sites More sharing options...
junkew Posted September 7, 2017 Share Posted September 7, 2017 yes, the trick is to pay them for an xml api feed FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets Link to comment Share on other sites More sharing options...
VaishnaviBUtpat Posted September 8, 2017 Share Posted September 8, 2017 you can use form elements.... for example, Local $oIE = _IECreate("www.google.com",1,1,1,1) Sleep(2000)_IELoadWait($oIE) Sleep(2000) $o_form = _IEFormGetObjByName($oIE, "f") $element = _IEFormElementGetObjByName($o_form,"q") _IEFormElementSetValue($element,"INDIA") You can get the reference objects using F12 Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now