Dizzy Posted December 27, 2014 Share Posted December 27, 2014 Hi all, i've tried to get the prices of the different petrol sorts - but i didn't get them. As aou can see, there are "Diesel", "Super E10", "Super E5", "SuperPlus" and "Autogas". --> http://www.clever-tanken.de/tankstelle_details/20205 Can someone help me to get them from this site to a excel sheet? Thanks Dizzy Link to comment Share on other sites More sharing options...
JohnOne Posted December 27, 2014 Share Posted December 27, 2014 Show what you hae tried. AutoIt Absolute Beginners Require a serial Pause Script Video Tutorials by Morthawt ipify Monkey's are, like, natures humans. Link to comment Share on other sites More sharing options...
Dizzy Posted December 28, 2014 Author Share Posted December 28, 2014 Hi JohnOne, i've tried to use the _IE options to get the value. The best result was to read the whole html to text - but this can't be the right way. I use the Firefox inspector to see which name, ID or ... this belongs to. I think, that all needed information shoud be found around the <div ID "fuel_price_list" - see "Firefox_Inspector.JPG". But i didn't know how i should look for this "ID" and how to get to 5 different prices into variables (maybe in one array?). So i stumple around to get this values - using debug and "try&error" I want to read these values (Petrol.jpg) every 30 minutes and store them in a excel sheet. 48 entries / day and over ~ 30 days. Thanks for helping me. Dizzy And : do you know how to change the title? There is a little misspelling there "Excel" ... #include <IE.au3> #include <MsgBoxConstants.au3> #include <Debug.au3> _DebugSetup() Local $oIE = _IECreate("http://www.clever-tanken.de/tankstelle_details/20205", 0, 0) _IELoadWait($oIE) ;Local $sText = _IEBodyReadText($oIE) Local $iNumFrames $oFrames = _IEFrameGetCollection ($oIE) ;_DebugReport("$oFrames = " & $oFrames) $iNumFrames = @extended ;_DebugReport("$iNumFrames = " & $iNumFrames) For $i = 0 to ($iNumFrames - 1) $oFrame = _IEFrameGetCollection ($oIE, $i ) msgBox(1, " var type " & VarGetType( $oFrame) , @error ) $oForms = _IEFormGetCollection ($oFrame) Next Link to comment Share on other sites More sharing options...
water Posted December 28, 2014 Share Posted December 28, 2014 When using the Firefox browser you need to use the Firefox UDF as well. The IE UDF only works for the Internet Explorer. My UDFs and Tutorials: Spoiler UDFs:Active Directory (NEW 2022-02-19 - Version 1.6.1.0) - Download - General Help & Support - Example Scripts - WikiExcelChart (2017-07-21 - Version 0.4.0.1) - Download - General Help & Support - Example ScriptsOutlookEX (2021-11-16 - Version 1.7.0.0) - Download - General Help & Support - Example Scripts - WikiOutlookEX_GUI (2021-04-13 - Version 1.4.0.0) - DownloadOutlook Tools (2019-07-22 - Version 0.6.0.0) - Download - General Help & Support - WikiPowerPoint (2021-08-31 - Version 1.5.0.0) - Download - General Help & Support - Example Scripts - WikiTask Scheduler (NEW 2022-07-28 - Version 1.6.0.1) - Download - General Help & Support - Wiki Standard UDFs:Excel - Example Scripts - WikiWord - Wiki Tutorials:ADO - WikiWebDriver - Wiki Link to comment Share on other sites More sharing options...
Dizzy Posted December 28, 2014 Author Share Posted December 28, 2014 Hi water, i just use Firefox to "inspect" the website-code. It is regardless wether using Firefox or IE. Do you have a first step for me? Thanks Dizzy Link to comment Share on other sites More sharing options...
Moderators SmOke_N Posted December 28, 2014 Moderators Share Posted December 28, 2014 It displays in FF for me, but not in IE Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer. Link to comment Share on other sites More sharing options...
Moderators SmOke_N Posted December 28, 2014 Moderators Share Posted December 28, 2014 It still won't show in my IE, probably a version issue, but try this expandcollapse popup_myScrape($oIE) Func _myScrape(ByRef $oObj) If Not IsObj($oObj) Then Return SetError(1, 0, 0) EndIf ; step 1. <div class="fuel_price_entry ng-scope" Local $oDivs = _IETagNameGetCollection($oObj, "DIV") If Not IsObj($oDivs) Then Return SetError(2, 0, 0) EndIf ; find matching class objeects Local $aItems[0][3], $iDim = 0 For $oItm In $oDivs If String($oItm.className) = "fuel_price_entry ng-scope" Then ReDim $aItems[$iDim + 1][3] $aItems[$iDim][0] = $oItm $iDim += 1 EndIf Next If $iDim = 0 Then Return SetError(3, 0, 0) ; nothing found EndIf ; we have an array of container objects now For $iItem = 0 To $iDim - 1 ; we need to know the type of fuel $oDivs = _IETagNameGetCollection($aItems[$iItem][0], "DIV") For $oItm In $oDivs Switch String($oItm.className) Case "fuel_type" $aItems[$iItem][1] = StringRegExpReplace($oItm.innerText, "(.+?)(\v*.*?)", "$1") Case "price_field price_dirty" $aItems[$iItem][2] = String($oItm.innerText) EndSwitch Next Next ; reformat return array to remove objects found Local $aRet[$iDim][2] For $i = 0 To $iDim - 1 $aRet[$i][0] = $aItems[$i][1] ; type of fuel $aRet[$i][1] = $aItems[$i][2] ; price field/dirty (that just sounds bad) Next Return $aRet EndFunc Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer. Link to comment Share on other sites More sharing options...
Dizzy Posted December 29, 2014 Author Share Posted December 29, 2014 (edited) Hi SmOke_N, thanks for your help. Same for me with your code. I've inserted same consolewrites as checkpoints expandcollapse popup#include <IE.au3> #include <MsgBoxConstants.au3> Local $oIE = _IECreate("http://www.clever-tanken.de/tankstelle_details/20205", 0, 0) _IELoadWait($oIE) _myScrape($oIE) _IELoadWait($oIE) Func _myScrape(ByRef $oObj) ConsoleWrite("In Func" & @crlf) If Not IsObj($oObj) Then Return SetError(1, 0, 0) EndIf ConsoleWrite("No Error_1" & @crlf) ; step 1. <div class="fuel_price_entry ng-scope" Local $oDivs = _IETagNameGetCollection($oObj, "DIV") ConsoleWrite("Found oDivs : " & $oDivs & @crlf) If Not IsObj($oDivs) Then Return SetError(2, 0, 0) EndIf ConsoleWrite("No Error_2" & @crlf) ; find matching class objeects Local $aItems[0][3], $iDim = 0 ConsoleWrite("Found aItems : " & $aItems & @crlf) For $oItm In $oDivs If String($oItm.className) = "fuel_price_entry ng-scope" Then ReDim $aItems[$iDim + 1][3] $aItems[$iDim][0] = $oItm $iDim += 1 EndIf Next If $iDim = 0 Then Return SetError(3, 0, 0) ; nothing found EndIf ConsoleWrite("No Error_3" & @crlf) ; we have an array of container objects now For $iItem = 0 To $iDim - 1 ; we need to know the type of fuel $oDivs = _IETagNameGetCollection($aItems[$iItem][0], "DIV") ConsoleWrite("Found oDivs_2 : " & $iItem & @crlf) For $oItm In $oDivs Switch String($oItm.className) Case "fuel_type" $aItems[$iItem][1] = StringRegExpReplace($oItm.innerText, "(.+?)(\v*.*?)", "$1") Case "price_field price_dirty" $aItems[$iItem][2] = String($oItm.innerText) EndSwitch Next Next ; reformat return array to remove objects found Local $aRet[$iDim][2] For $i = 0 To $iDim - 1 $aRet[$i][0] = $aItems[$i][1] ; type of fuel $aRet[$i][1] = $aItems[$i][2] ; price field/dirty (that just sounds bad) ConsoleWrite("aRet[" & $i &"][0] :" & $aRet[$i][0] & @crlf) ConsoleWrite("aRet[" & $i &"][1] :" & $aRet[$i][1] & @crlf) Next Return $aRet EndFunc Result: >"C:\Program Files (x86)\AutoIt3\SciTE\AutoIt3Wrapper\AutoIt3Wrapper.exe" /run /prod /ErrorStdOut /in "C:temp_IE_Test6.au3" /UserParams +>10:10:30 Starting AutoIt3Wrapper v.14.801.2025.0 SciTE v.3.4.4.0 Keyboard:00000407 OS:WIN_7/Service Pack 1 CPU:X64 OS:X64 Environment(Language:0407) +> SciTEDir => C:Program Files (x86)AutoIt3SciTE UserDir => C:Program Files (x86)AutoIt3SciTEAutoIt3Wrapper >Running AU3Check (3.3.12.0) from:C:Program Files (x86)AutoIt3 input:C:temp_IE_Test6.au3 +>10:10:30 AU3Check ended.rc:0 >Running:(3.3.12.0):C:Program Files (x86)AutoIt3autoit3.exe "C:temp_IE_Test6.au3" --> Press Ctrl+Alt+Break to Restart or Ctrl+Break to Stop In Func No Error_1 Found oDivs : No Error_2 Found aItems : No Error_3 Found oDivs_2 : 0 Found oDivs_2 : 1 Found oDivs_2 : 2 Found oDivs_2 : 3 Found oDivs_2 : 4 aRet[0][0] :DieselMTS-K Preis aRet[0][1] : aRet[1][0] :Super E10MTS-K Preis aRet[1][1] : aRet[2][0] :Super E5MTS-K Preis aRet[2][1] : aRet[3][0] :SuperPlus 4 ct. aRet[3][1] : aRet[4][0] :Autogas aRet[4][1] : +>10:10:34 AutoIt3.exe ended.rc:0 +>10:10:34 AutoIt3Wrapper Finished. >Exit code: 0 Time: 4.4 I've tried to translate this to FF - has taken a moment to see that i have to use mozl ... expandcollapse popup#include "FF.au3" ;_FFConnect(default,Default,6000) Local $oFire= _FFStart("http://www.clever-tanken.de/tankstelle_details/20205") _myScrape($oFire) _FFQuit() Func _myScrape(ByRef $oObj) ConsoleWrite("In Func" & @crlf) If Not IsObj($oObj) Then Return SetError(1, 0, 0) EndIf ConsoleWrite("No Error_1" & @crlf) ; step 1. <div class="fuel_price_entry ng-scope" Local $oDivs = _FFObjGet($oObj, "DIV") ConsoleWrite("Found oDivs : " & $oDivs & @crlf) If Not IsObj($oDivs) Then Return SetError(2, 0, 0) EndIf ConsoleWrite("No Error_2" & @crlf) ; find matching class objeects Local $aItems[0][3], $iDim = 0 ConsoleWrite("Found aItems : " & $aItems & @crlf) For $oItm In $oDivs If String($oItm.className) = "fuel_price_entry ng-scope" Then ReDim $aItems[$iDim + 1][3] $aItems[$iDim][0] = $oItm $iDim += 1 EndIf Next If $iDim = 0 Then Return SetError(3, 0, 0) ; nothing found EndIf ConsoleWrite("No Error_3" & @crlf) ; we have an array of container objects now For $iItem = 0 To $iDim - 1 ; we need to know the type of fuel $oDivs = _FFObjGet($aItems[$iItem][0], "DIV") ConsoleWrite("Found oDivs_2 : " & $iItem & @crlf) For $oItm In $oDivs Switch String($oItm.className) Case "fuel_type" $aItems[$iItem][1] = StringRegExpReplace($oItm.innerText, "(.+?)(\v*.*?)", "$1") Case "price_field price_dirty" $aItems[$iItem][2] = String($oItm.innerText) EndSwitch Next Next ; reformat return array to remove objects found Local $aRet[$iDim][2] For $i = 0 To $iDim - 1 $aRet[$i][0] = $aItems[$i][1] ; type of fuel $aRet[$i][1] = $aItems[$i][2] ; price field/dirty (that just sounds bad) ConsoleWrite("aRet[" & $i &"][0] :" & $aRet[$i][0] & @crlf) ConsoleWrite("aRet[" & $i &"][1] :" & $aRet[$i][1] & @crlf) Next Return $aRet EndFunc Result: >"C:\Program Files (x86)\AutoIt3\SciTE\AutoIt3Wrapper\AutoIt3Wrapper.exe" /run /prod /ErrorStdOut /in "C:temp_IE_Test6_FF.au3" /UserParams +>10:04:33 Starting AutoIt3Wrapper v.14.801.2025.0 SciTE v.3.4.4.0 Keyboard:00000407 OS:WIN_7/Service Pack 1 CPU:X64 OS:X64 Environment(Language:0407) +> SciTEDir => C:Program Files (x86)AutoIt3SciTE UserDir => C:Program Files (x86)AutoIt3SciTEAutoIt3Wrapper >Running AU3Check (3.3.12.0) from:C:Program Files (x86)AutoIt3 input:C:temp_IE_Test6_FF.au3 +>10:04:33 AU3Check ended.rc:0 >Running:(3.3.12.0):C:Program Files (x86)AutoIt3autoit3.exe "C:temp_IE_Test6_FF.au3" --> Press Ctrl+Alt+Break to Restart or Ctrl+Break to Stop __FFStartProcess: ""C:Program Files (x86)Mozilla Firefoxfirefox.exe" -new-window "http://www.clever-tanken.de/tankstelle_details/20205" "-repl 4242 " _FFConnect: OS: WIN_7 WIN32_NT 7601 Service Pack 1 _FFConnect: AutoIt: 3.3.12.0 _FFConnect: FF.au3: 0.6.0.1b-14 _FFConnect: IP: 127.0.0.1 _FFConnect: Port: 4242 _FFConnect: Delay: 2ms _FFConnect: Socket: 528 _FFConnect: Browser: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:34.0) Gecko/20100101 Firefox/34.0 __FFSendJavaScripts: Sending functions to FireFox .......... done _FFLoadWait: . loaded in 10ms [object HTMLDocument] - {location: {...}, af_6098183008570310744: {...}, location: {...}, getElementsByName: function() {...}, getItems: function() {...}, open: function() {...}, close: function() {...}, ...} In Func __FFQuit: Closing FireFox ... +>10:04:37 AutoIt3.exe ended.rc:0 +>10:04:37 AutoIt3Wrapper Finished. >Exit code: 0 Time: 4.829 Whats wrong with this code? You wrote, you'll get it on FF - can you show me how? I'm not fixed on IE. Thanks Dizzy Edited December 29, 2014 by Dizzy Link to comment Share on other sites More sharing options...
Danp2 Posted December 29, 2014 Share Posted December 29, 2014 For starters, you need to switch from using IsObj() to checking @error because the FF UDF doesn't return objects like the IE UDF. Latest Webdriver UDF Release Webdriver Wiki FAQs Link to comment Share on other sites More sharing options...
Moderators SmOke_N Posted December 29, 2014 Moderators Share Posted December 29, 2014 I never ran the code, I just wrote it based on what I saw in the javascript spy tool with FF, but I've never used the FF.au3 stuff. I can't get the data to render in IE at all. So the fact that it's close, just means you need to check the class object I'm searching for the dollar amount and change them up with one of the others until it works. Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer. Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now