Monitoring changes in a website

KurogamineNox · October 15, 2010

[solution=Not Complete]

hey, I am making a new program that can monitor a status of a changing variable in a website.

I would like to monitor changes in my ranking on this website

http://rf.ogplanet.com/community/rankings.html

You would probably have to be logged in to acually see your ranking but ill post a picture here.

Posted Image

Id have the program check every hour for changes. If someone could give me an example using any website would be fine or steer me in the right direction to learn this if possible would be nice as well.

Edited October 16, 2010 by KurogamineNox

MrMitchell · October 15, 2010

If you view the page source do you see your ranking in plain text? If so, try using a combination of StringRegExp() or _StringBetween() along with the IE functions to open the page and extract the source.

Gets a little more complicated if you have to log in to the site every time or every so often...

Edited October 15, 2010 by MrMitchell

KurogamineNox · October 15, 2010

If you view the page source do you see your ranking in plain text? If so, try using a combination of StringRegExp() or _StringBetween() along with the IE functions to open the page and extract the source.

Gets a little more complicated if you have to log in to the site every time or every so often...

I think this site is a little bit more complicated than I thought. I think they have it update by itself from some coding maybe. Here is the code of the source of the site.

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>Rankings | RUMBLE FIGHTER</title>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
<meta name="subject" content="Rumble Fighter" />
<meta name="keywords" content="games, game, gamefactory, game factory, OGP, OG Planet, OGPlanet, OG, Planet, multiplayer, kid game, best kid game, free kid game, kid online game, kid game software, free online kid game, astro, astros, carats, carat, anime, anime picture, anime girl, anime game, full metal alchemist, naruto, naruto style, cute online game, cute game, cute characters, cartoon game, cartoon online game, cartoon style game, best game, video game, online video game, korean video game, korean game, most popular game, most popular online game, free fighting game, multiplayer fighting, fighting game, free fighting, fantasy fighting, fight game, fantasy online fight, fantasy online fighting, free fight game, free fighting game, martial arts, martial arts game, Rumble, Rumble Fighter, RumbleFighter, Online Fighting Game, Multyplayer fighting game, Beat'em Up, 3D Fighting, Exocore, Scroll, Sacred Scroll, Demon King, Fighting, Kungfu, Karate, Boxing, Wrestling, Taekwondo, Gem Fighter" />
<meta name="description" content="Rumble Fighter offers multiplayer online fighting. Are you ready to rumble?" />
<meta name="copyright" content="OGPlanet, OGplanet, Gretech, Nimonix, www.rf.ogplanet.com, 2008" />
<meta name="classification" content="online gaming, free online fighting game, online fighting game, Rumble Fighter" />
<meta name="author" content="OGPlanet - rf.ogplanet.com" />
<meta name="abstract" content="Rumble Fighter – Are you ready to rumble?" />
<link rel="stylesheet" type="text/css" href="http://ui.ogplanet.com/common/css/common.css" />
<link rel="stylesheet" type="text/css" href="http://static.ogplanet.com/static/RF/en/css/layout.css" media="screen" />
<script type="text/javascript" src="http://static.ogplanet.com/static/Common/js/jquery.js"></script>

<script type="text/javascript" language="javascript" src="http://ui.ogplanet.com/common/js/common.js"></script>
<script type="text/javascript" src="http://static.ogplanet.com/static/RF/js/include.js"></script>
<script type="text/javascript" src="http://static.ogplanet.com/static/Common/js/modulecache.js"></script>
<!--[if IE 6]>
    <link rel="stylesheet" type="text/css" href="http://static.ogplanet.com/static/RF/en/css/ie6.css" media="screen" />
<![endif]-->

</head>
<body>    
    <!-- universal top menu -->    
    <div id="gnb"></div>
    <script language="javascript" type="text/javascript">
    $(function(){
        gameGNB.setGNB({"auth":"login"});
    }); 
    </script>
    <div id="body_events">

    <div id="community_bg">
    <div id="container" style="">
        <div id="header_sub" style="">
            <script type="text/javascript">
                showFlashObject('5','http://static.ogplanet.com/static/RF/en/swf/sub.swf?menuCode=5', '1000', '225', 'best', 'transparent', '#fff');
            </script>
            <div id="facebook" style="">
                <script>setFacebookUI();</script>            
            </div><!-- end facebook -->
        </div><!-- end header -->

        
        <div style="clear:both;"></div>
        <div id="empty_box"style=""></div>
        <div id="sub_wrapper" style="">
            <div id="leftSide" style="">

                <!--
                <div class="leftPanel">
                    <div id="leftMenu_community">
                        <ul>
                            <li id="leftMenu_list"><a href="http://newforum.ogplanet.com/" id="forum" title="go to forum">&nbsp;</a></li>
                            <li id="leftMenu_list"><a href="rankings.html" id="rankingsActive" title="rankings">&nbsp;</a></li>
                            <li id="leftMenu_list"><a href="polls.html" id="polls" title="polls">&nbsp;</a></li>
                        </ul>
                    </div>
                </div>-->
                <div class="leftPanel">
                    <div id="leftMenu_community2">
                        <ul>

                            <li id="leftMenu_list"><a href="http://newforum.ogplanet.com/" id="forum" title="go to forum">&nbsp;</a></li>
                            <li id="leftMenu_list"><a href="rankings.html" id="rankingsActive" title="rankings">&nbsp;</a></li>
                            <li id="leftMenu_list"><a href="polls.html" id="polls" title="polls">&nbsp;</a></li>
                            <li id="leftMenu_list"><a href="stampCollection.html" id="stamp" title="stamp collection">&nbsp;</a></li>
                        </ul>
                    </div>
                </div>

                <!-- put last left menu -->

                <script>setLeftMenu();</script>
                <!-- put last left menu -->
                
            </div><!-- end leftSide -->
            <div id="sub_Main" style="">
                <div id="subContents" style="">
                    <div id="ranking_body" style="display:none;" >
                        <div id="ranking_head">
                        </div><!-- end page_head -->

                        
                        <h4 class="myrank">My Ranking</h4>
                        <p id="myrank_login" style="display:none;"><a href="https://id.ogplanet.com/ogp/login.og" id="login_msg">LOG IN</a> TO VIEW YOUR RANKING</p>
                        <p id="myrank_not_player" style="display:none">ARE YOU A RUMBLE FIGHTER PLAYER? PLAY RUMBLE FIGHTER TO BE ON THE RANKINGS TODAY!!</p>
                        
                        <div id="myrank_view" style="display:none">
                        </div>

                        <div id="page_info">

                            <span id="current_pg"><strong>1</strong> Page</span> / <span id="total_pg"><strong>5</strong> Pages</span>
                        </div>

                        <span class="rank_TH"></span><!-- rank table head -->
                        <table class="ranking_table" id="table1"><!-- second td:span class can be up, down, and same -->

                            <tbody></tbody>
                        </table>
                        <table class="ranking_table" id="table2" style="display:none;"><tbody></tbody></table>
                        <table class="ranking_table" id="table3" style="display:none;"><tbody></tbody></table>
                        <table class="ranking_table" id="table4" style="display:none;"><tbody></tbody></table>
                        <table class="ranking_table" id="table5" style="display:none;"><tbody></tbody></table>
                        <div id="paging"><!-- paging for pagenation-->
                            <ul>
                                
                                <li><a id="firstPage" style="cursor:pointer;" class="current">1</a></li>

                                <li><a id="secondPage" style="cursor:pointer;">2</a></li>
                                <li><a id="thirdPage" style="cursor:pointer;">3</a></li>
                                <li><a id="fourthPage" style="cursor:pointer;">4</a></li>
                                <li><a id="fifthPage" style="cursor:pointer;">5</a></li>
                                
                            </ul>
                        </div><!--// end paging -->
                    </div><!-- end ranking_body -->

                </div><!-- end subContents -->
            </div><!-- end subMain -->            
        </div><!-- end subwrapper -->        
        <div id="footer_middle_sub" style="">
            <a href="http://www.ogplanet.com/main.og" id="ogplanet" title="OGPlanet" target="_blank"></a>
            <a href="http://company.ogplanet.com" id="about" title="About OGPlanet" target="_blank"></a>
            <a href="#" id="userAgreement" title="User Agreement" onclick="javascript:popup('http://www.ogplanet.com/about.og?op=useragreement');"></a>
            <a href="#" id="privacy" title="Privacy Policy" onclick="javascript:popup('http://www.ogplanet.com/about.og?op=privacypolicy');"></a>
            <a href="http://rf.ogplanet.com/support/helpdesk.html" id="contact" title="Contact Us"></a>

        </div><!-- //end footer_middle -->
    </div><!--// end container -->    
    </div><!-- //end community_bg -->
    </div><!--end body_wrapper -->
    
    <div id="footer_sub" style="">               
    </div><!-- end footer_sub -->
    <script type="text/Javascript">
    $("#firstPage").click(function () {$('.ranking_table').hide();$('#current_pg').html('<strong>1</strong> Page');$('#table1').show();$('span#currentPage').html('<strong>1</strong> Page');$("#paging ul li a").removeClass("current");$(this).addClass("current");});
    $("#secondPage").click(function () {$('.ranking_table').hide();$('#current_pg').html('<strong>2</strong> Page');$('#table2').show();$('span#currentPage').html('<strong>2</strong> Page');$("#paging ul li a").removeClass("current");$(this).addClass("current");});
    $("#thirdPage").click(function () {$('.ranking_table').hide();$('#current_pg').html('<strong>3</strong> Page');$('#table3').show();$('span#currentPage').html('<strong>3</strong> Page');$("#paging ul li a").removeClass("current");$(this).addClass("current");});
    $("#fourthPage").click(function () {$('.ranking_table').hide();$('#current_pg').html('<strong>4</strong> Page');$('#table4').show();$('span#currentPage').html('<strong>4</strong> Page');$("#paging ul li a").removeClass("current");$(this).addClass("current");});
    $("#fifthPage").click(function () {$('.ranking_table').hide();$('#current_pg').html('<strong>5</strong> Page');$('#table5').show();$('span#currentPage').html('<strong>5</strong> Page');$("#paging ul li a").removeClass("current");$(this).addClass("current");});
    
    var module__ = new modulecache();
    module__.getJSON(module__.RANKLIST, {gameName:"rf", page:1, PerPage:100}, ranking);    
    module__.getJSON("/userrank.og", {gameName:"rf"}, userRank);

    function userRank(data) {

        if(data.user.isLogin == '0'){
            $('#myrank_login').show();
        } else if(data.user.isLogin == "1" && data.rank.UserRanking.length == 0) {
            $('#myrank_not_player').show();
        } else if(data.rank.UserRanking.length >= 1) {
            var myrank = $('#myrank_view');        
            progression = 'same';
            if(data.rank.UserRanking[0].Change == 0) {
                progression = 'same';
            } else if (data.rank.UserRanking[0].Change > 0) {
                progression = 'up';
            } else if (data.rank.UserRanking[0].Change < 0) {
                progression = 'down';
                data.rank.UserRanking[0].Change = -(data.rank.UserRanking[0].Change);
            }
            
            myrank.append(
                '<span id="myrank">'+data.rank.UserRanking[0].Rank+'</span>'
                +'<span id="'+progression+'"></span>'
                +'<span id="progression">'+data.rank.UserRanking[0].Change+'</span>'
                +'<span id="name">'+data.rank.UserRanking[0].CharacterName+'</span>'
                +'<span id="exp">'+data.rank.UserRanking[0].Exp+'</span>'
            );
            myrank.show();
        }
    }
        
    function ranking(data){
        var z = $('.ranking_table tbody');
        var tableNum;
        var levelNum;
        z.html('');

        rank = data.RankList.length;
        for(var i = 0; i < rank; i++) {
            if (i < 20){ z = $('#table1 tbody');}
            if (i >= 20 && i < 40){ z = $('#table2 tbody');}
            if (i >= 40 && i < 60){ z = $('#table3 tbody');}
            if (i >= 60 && i < 80){ z = $('#table4 tbody');}
            if (i >= 80 && i < 100){ z = $('#table5 tbody');}
            
            change = 'same';
            changed = '';
            top3ranking = '';
            if(data.RankList[i].Change == 0) {
                change = 'same';
            } else if (data.RankList[i].Change > 0) {
                change = 'up';            
                changed = data.RankList[i].Change;
            } else if (data.RankList[i].Change < 0) {
                change = 'down';
                changed = -(data.RankList[i].Change);
            } 
            
            if (data.RankList[i].Rank < 4) {
                if(data.RankList[i].Rank == 1){
                    top3ranking = 'top1';
                }else if(data.RankList[i].Rank == 2){
                    top3ranking = 'top2';
                }else if(data.RankList[i].Rank == 3){
                    top3ranking = 'top3';
                }
            }

            z.append( '<tr><td class="first"><span class="'+top3ranking+'">' + data.RankList[i].Rank + '</span></td>'
                            +'<td class="second"><span class="'+change+'">'+changed+'</span></td>'
                            +'<td class="third">'+data.RankList[i].nickName+'</td>'
                            +'<td class="fourth">'+data.RankList[i].Exp+'</td></tr>');
            
        }
        $('#ranking_body').show();
    }

    </script>
    <script type="text/javascript">
        var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www.");
        document.write(unescape("%3Cscript src='" + gaJsHost + "google-analytics.com/ga.js' type='text/javascript'%3E%3C/script%3E"));
    </script>
    <script type="text/javascript">
          try {
              var pageTracker = _gat._getTracker("UA-3525049-3");
              pageTracker._trackPageview();
          } catch(err) {}
    </script>

    
    <script type="text/javascript" language="javascript">
        function popup(URL) {
            day = new Date();
            id = day.getTime();
            eval("page" + id + " = window.open(URL, '" + id + "', 'toolbar=0,scrollbars=yes,location=0,statusbar=0,menubar=0,resizable=1,width=500,height=600');");
        }
    </script>

</body>
</html>

I think in the page source my ranking is displayed here.

function userRank(data) {

        if(data.user.isLogin == '0'){
            $('#myrank_login').show();
        } else if(data.user.isLogin == "1" && data.rank.UserRanking.length == 0) {
            $('#myrank_not_player').show();
        } else if(data.rank.UserRanking.length >= 1) {
            var myrank = $('#myrank_view');        
            progression = 'same';
            if(data.rank.UserRanking[0].Change == 0) {
                progression = 'same';
            } else if (data.rank.UserRanking[0].Change > 0) {
                progression = 'up';
            } else if (data.rank.UserRanking[0].Change < 0) {
                progression = 'down';
                data.rank.UserRanking[0].Change = -(data.rank.UserRanking[0].Change);
            }
            
            myrank.append(
                '<span id="myrank">'+data.rank.UserRanking[0].Rank+'</span>'
                +'<span id="'+progression+'"></span>'
                +'<span id="progression">'+data.rank.UserRanking[0].Change+'</span>'
                +'<span id="name">'+data.rank.UserRanking[0].CharacterName+'</span>'
                +'<span id="exp">'+data.rank.UserRanking[0].Exp+'</span>'
            );
            myrank.show();
        }
    }

Not sure if I can get it from this site. I was planning on doing more than one site to monitor but I thought id start with this. If I can get this site could someone still tell me how and use a website as an example? Or steer me to a page that could help me or something.

KurogamineNox · October 15, 2010

Does anyone know how to get the data from this? I am not sure if you could get data from a website maybe using autoit. Ive been looking at the

data.rank.UserRanking[0]

Hoping there would be a way to get that since I think that is where my ranking is stored maybe.

MrMitchell · October 15, 2010

What happens when you put this directly into the URL field after logging in to the page?

javascript:document.write(data.rank.UserRanking[0]);

KurogamineNox · October 15, 2010

What happens when you put this directly into the URL field after logging in to the page?

javascript:document.write(data.rank.UserRanking[0]);

Nothing, you think this is in a javascript format(I put that in and hit enter and the page didnt change or load or what ever. Just stayed there.)?

MrMitchell · October 16, 2010

Oops try this one, has to end with semicolon too:

javascript:document.write(data.rank.UserRanking[0].Rank);

KurogamineNox · October 16, 2010

No change again, I think ill only be able to see change is if I acually use a program to get the info.

#include <GUIConstants.au3>
#include <IE.au3>
#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <Constants.au3>
#include "resources.au3"


$GUI = GUICreate("Browser", 1000, 500)
$object = ObjCreate("Shell.Explorer.2")
$object_ctrl = GUICtrlCreateObj($object, 0, 0, 1000, 470)
GUISetSTate(@SW_SHOW)
_IENavigate($object, "http://rf.ogplanet.com")
$Rank = ""
$label = GUICtrlCreateLabel($Rank,0,470,100,50)
While 1
sleep(100)
$msg = GUIGetMsg()
    Select
        Case $msg = $GUI_EVENT_CLOSE
            Exit
    EndSelect
WEnd

This is based on as if I was already logged in. Ill fix the label stuff later as its kind of missplaced and lazy right now. I think that the page wont update it but the values are updated so I am thinking ill need to get the value and put it in a variable into my program to acually see if it works. These are just coming from my mind. *is still adding more coding as this is only a test program*

Kalin · October 16, 2010

No change again, I think ill only be able to see change is if I acually use a program to get the info.
#include <GUIConstants.au3>
#include <IE.au3>
#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <Constants.au3>
#include "resources.au3"


$GUI = GUICreate("Browser", 1000, 500)
$object = ObjCreate("Shell.Explorer.2")
$object_ctrl = GUICtrlCreateObj($object, 0, 0, 1000, 470)
GUISetSTate(@SW_SHOW)
_IENavigate($object, "http://rf.ogplanet.com")
$Rank = ""
$label = GUICtrlCreateLabel($Rank,0,470,100,50)
While 1
sleep(100)
$msg = GUIGetMsg()
    Select
        Case $msg = $GUI_EVENT_CLOSE
            Exit
    EndSelect
WEnd
This is based on as if I was already logged in. Ill fix the label stuff later as its kind of missplaced and lazy right now. I think that the page wont update it but the values are updated so I am thinking ill need to get the value and put it in a variable into my program to acually see if it works. These are just coming from my mind. *is still adding more coding as this is only a test program*

You sir just gave me an idea on a very important guide to make later on that can assist you and many others with these sort of problems.

MrMitchell · October 16, 2010

I was hoping that JS in the address bar would work, guess not...not really experienced in that area. That site is blocked here at work so I can't test with it, and even if it wasn't blocked I think I'm a little too lazy to register lol. Sorry couldn't help you, if something comes up I'll let ya know...good luck

KurogamineNox · October 16, 2010

Alright, Thanks for trying though. I appreciate it.

KurogamineNox · October 16, 2010

You sir just gave me an idea on a very important guide to make later on that can assist you and many others with these sort of problems.

Well, If it does help me with this sort of thing Ill be glad to see.

KurogamineNox · October 16, 2010

I was hoping that JS in the address bar would work, guess not...not really experienced in that area. That site is blocked here at work so I can't test with it, and even if it wasn't blocked I think I'm a little too lazy to register lol. Sorry couldn't help you, if something comes up I'll let ya know...good luck

Oh also I have like an extra account I do not even use which You could have used to experiment on, but yeah if its blocked at your place then its preaty much useless at this moment either way.

MrMitchell · October 16, 2010

Oh also I have like an extra account I do not even use which You could have used to experiment on, but yeah if its blocked at your place then its preaty much useless at this moment either way.

I'll take it, I had someone at home turn on my PC so now I got remote access to a PC that isn't blocked. This would be cool to experiment with...

KurogamineNox · October 16, 2010

I'll take it, I had someone at home turn on my PC so now I got remote access to a PC that isn't blocked. This would be cool to experiment with...

username nobodybrandon123

password brandon@nox

Account is useless to me as I found it out on youtube.

Finding abandon accounts is fun but that account is entirely useless for gameplay as everyone who accessed it already ripped off all the items and what not.

Edited October 16, 2010 by KurogamineNox

MrMitchell · October 16, 2010

Ok so this works, but it's weird...

#include <IE.au3>
#include <String.au3>

$oIE = _IEAttach("Rankings")
$text = _IEBodyReadHTML($oIE)
$aRanking = StringRegExp($text, "(?:<SPAN id=myrank>)(\d*)(?:</SPAN>)", 1)
$ranking = $aRanking[0]
ConsoleWrite($ranking & @CRLF)

Exit

KurogamineNox · October 16, 2010

Ok so this works, but it's weird...

#include <IE.au3>
#include <String.au3>

$oIE = _IEAttach("Rankings")
$text = _IEBodyReadHTML($oIE)
$aRanking = StringRegExp($text, "(?:<SPAN id=myrank>)(\d*)(?:</SPAN>)", 1)
$ranking = $aRanking[0]
ConsoleWrite($ranking & @CRLF)

Exit

Is that the exact coding of your program? I am trying to recreate what You have made to see how it works.

Hm seems like it worked. "Had to use my acual Internet Explorer browser rather than having it use the embeded one."

Edited October 16, 2010 by KurogamineNox

KurogamineNox · October 16, 2010

I do not see anything on it being weird though.

Edit: Wait, Not sure but maybe I can understand what you mean by weird.

Edited October 16, 2010 by KurogamineNox

KurogamineNox · October 16, 2010

A friend who also uses auto it has told me some more stuff.

Your coding works, but I can adapt your coding into using TCP functions. Sending the right Headers. Hm...I still wonder what you mean by weird.

(I need my program to be able to get the information without having to acually surf the page so the page doesnt have to load. He says I can get the Html Source coding with tcp and still use the same functions.)

MrMitchell · October 16, 2010

(I need my program to be able to get the information without having to acually surf the page so the page doesnt have to load. He says I can get the Html Source coding with tcp and still use the same functions.)

...and that's what I meant by weird Posted Image I don't like doing it this way, it's unreliable.

Monitoring changes in a website

Recommended Posts

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Create an account or sign in to comment

Create an account

Sign in

Recently Browsing 0 members