Any good way to speed up this script?

[Noko]

Basically i made a script for scraping the api that http://www.guildwarstrade.com/ has made public, but it does so at such a low rate for the moment. Do anyone see a way to speed up this script?

expandcollapse popup

#Region ;**** Directives created by AutoIt3Wrapper_GUI ****
#AutoIt3Wrapper_Res_requestedExecutionLevel=asInvoker
#EndRegion ;**** Directives created by AutoIt3Wrapper_GUI ****
;GWT data retreiver --> % profit
#include
#include
#include
;Temp[1] = Sell
;Temp[2] = Update
;$temp[3] = Buy

;Startup
$oExcel = _ExcelBookNew()
_ExcelWriteCell($oExcel,"Av Nikolas", 1,1)
_ExcelWriteCell($oExcel,"Bruker data mellom 68 og 31165 som id",2,1)
$url_Stub = "http://www.guildwarstrade.com/api/public/item?id="
$row_offset = 4-68
global $temp[10]

main()

func main()
;31165
for $i = 68 to 31165 step 1
retriver($i)

next
EndFunc


func retriver($id)
$string = InetRead($url_Stub & $id)
$string = BinaryToString($string)
if $string = "{}" Then
$row_offset = $row_offset - 1
Return
EndIf
$string = StringTrimRight($string,1)
$string = StringTrimLeft($string,1)
$temp = StringSplit($string,",")
$temp[1] = StringTrimLeft($temp[1],8)
$temp[3] = StringTrimLeft($temp[3],7)

Write($id,$temp)

EndFunc

func Write($id, $array)
_ExcelWriteCell($oExcel,$id,1,4)
_ExcelWriteCell($oExcel,$id,$row_offset + $id,1)
_ExcelWriteCell($oExcel,$temp[1],$row_offset + $id,2)
;_ExcelWriteCell($oExcel,$temp[2],$row_offset + $id,3)
_ExcelWriteCell($oExcel,$temp[3],$row_offset + $id,3)

$profit = $temp[1]*0.85-$temp[3]
_ExcelWriteCell($oExcel,$profit,$row_offset + $id,6)
if $temp[3] = 0 Then
_ExcelWriteCell($oExcel,0,$row_offset + $id,7)
Return
EndIf
$up_p = $temp[1]*0.85-$temp[3]
$dn_p = $temp[1]
$profitpercent = round((Round($up_p)/($dn_p)*100),1)
_ExcelWriteCell($oExcel,$profitpercent,$row_offset + $id,7)


EndFunc

[JLogan3o13]

Hi, Noko, welcome to the forum. Unfortunately, you seem to have missed the forum rules (located at the bottom right of every page), especially the part about game automation. Please review these, and you'll see why we do not give support on these types of questions.

Edit: As you seem to be just scraping data, I would say we'll invite a Mod to weigh in on this.

Edited January 23, 2013 by JLogan3o13

[Noko]

Yes, this is public avaiable data that the dev team (A-NET) allows to be scraped.

[Mat]

Interesting case. It is game automation if you go by our definition, but explicitly using an api provided, I think this is up to Melba to decide whether this particular grey area should be allowed.

Noko, you must understand we have particularly strong rules against game automation. Not because it is illegal in many cases, but because allowing for game automation makes the forum completely unmanageable.

In this case I imagine the downloading is the slow bit. You can try downloading multiple in parellel but the bottleneck is almost always your internet speed, so I doubt there is much to do to speed this up. You could write parellel to the download, but again I don't think there is anything too slow there, unless excel is being slow.

[Melba23]

Hi all,

The rules are quite clear: "•Launching, automation or script interaction with games or game servers, regardless of the game" - I see this as "interaction" (how else does it get the data?) and so not allowed. I also see no reason why using an "official" API should make any difference - the rules do not distinguish between "hacked" and "original" game software, both are out of bounds.

However, as this is an interesting case I am leaving the thread open for a while in case anyone has any comments either for or against this ruling. But please do not attempt to offer any help until I give the all-clear - if indeed I ever do.

M23

[Mat]

It's not interacting with the game, or even a game server. It's a web api interacting with a web server, you don't need the game installed to be doing this. It's very common code, get data from a website, parse it and do some logic on it, write data.

Like I said, we thought there was no grey area in the rules, but game related interaction with a web server is probably allowed if you go by the written word of the law as it stands.

Of course Melba, you ARE the word of the law, so that's why I left it up to you.

[kylomas]

Mat,

I respectfully disagree. This is automating gathering of game related data and the only logical conclusion to this effort is to gain advantage. Caveat,

I don't play games so am only judging this by what I see presented.

kylomas

[Melba23]

Guys,

Thanks for responding - exactly what I had hoped for.

So what exactly is the server with which this script is interacting? Is it associated with the official game server or is it run by an entirely separate entity? And surely the info garnered is for use in a game - and hence " game interaction" in some sense?

To be perfectly clear. my gut feeling is that allowing such scripts would be but the thin end of a very fat wedge and so I am very much against it. Any suggestions on how could we reword the rules to make it clear that such things are not permitted?

M23

Edit: Forget about the rules wording - "the Moderating team reserve the right to close any thread that they feel is contrary to the ethos of the forum" covers it nicely.

Edited January 24, 2013 by Melba23

[Mat]

Mat,

I respectfully disagree. This is automating gathering of game related data and the only logical conclusion to this effort is to gain advantage. Caveat,

I don't play games so am only judging this by what I see presented.

kylomas

Yes, I know. I am merely playing the devils advocate, to point out the gap in the forum rule wording. This is not covered by the forum rules in their current wording.

Melba, The website is the official *web*site. The information I imagine is like the in game stock market equivalent.

Edited January 24, 2013 by Mat

[kylomas]

Mat,

Yes, I ran the post. It creates a spreadsheet of value stats, though I did not let it complete the 30,000+ downloads.

[Melba23]

Hi all,

The website is the official *web*site

That is good enough for me - it is interaction with a "game server" and so prohibited under the current rules.

Thanks to all responders. And so we do arrive at a <click> ( © Jos)

M23