Jump to content

Download file from a website

aggarwal
 Share

Recommended Posts

aggarwal Seeker

aggarwal

Posted
Posted

Hello Everyone!!

So I am trying to automate this website(bea.gov). I am able to sail through all the forms and land on the final page of the website. Now I want to download the table that is at that page. They have given a download option but what I think(I am not the best person for website) is there is a image which is in a form of form and takes the data from the table below. Anyways the code for the download part of the fragment is attached below.

Any help on it will be greatly appreciated!!

Thanks

Abhishek Aggarwal

post-69943-0-50642700-1331850821_thumb.j

Link to comment
Share on other sites
aggarwal Seeker

aggarwal

Posted
  • Author
Posted

ohh ma bad. I did not realize I did print screen for two screens. Here goes the code of the download part-

FORM NAME="excel" action="/pdq/SurveyOutputServlet" METHOD="POST" target="_blank" >
<B>Download:</B>
<input id="download_xls" type="image" src="/images/buttons/download_button_xls.gif"  alt="Download as an Excel File" value="Download as an Excel File" style="vertical-align: middle">
<input type="hidden" name="request_action" value="get_data">
<input type="hidden" name="reformat" value="true">
<input type="hidden" name="from_results_page" value="true">
<input type="hidden" name="years_option" value="specific_years">
<input type="hidden" name="delimiter" value="comma">
<input type="hidden" name="output_type" value="multi">
<input type="hidden" name="periods_option" value="all_periods">
<input type="hidden" name="output_view" value="data|"/>
<input type="hidden" name="to_year" value="2010"/>
<input type="hidden" name="from_year" value="2001"/>
<input type="hidden" name="output_format" value="excel">
<input type="hidden" name="original_output_type" value="default">
<input type="hidden" name="series_id" value="ENU190005051013"/>
<input type="hidden" name="series_id" value="ENU190015051013"/>
<input type="hidden" name="series_id" value="ENU190035051013"/>
<input type="hidden" name="series_id" value="ENU190055051013"/>
<input type="hidden" name="series_id" value="ENU190075051013"/>
<input type="hidden" name="series_id" value="ENU190095051013"/>
<input type="hidden" name="series_id" value="ENU190115051013"/>
<input type="hidden" name="series_id" value="ENU190135051013"/>
<input type="hidden" name="series_id" value="ENU190155051013"/>
<input type="hidden" name="series_id" value="ENU190175051013"/>
<input type="hidden" name="series_id" value="ENU190195051013"/>
<input type="hidden" name="series_id" value="ENU190215051013"/>
<input type="hidden" name="series_id" value="ENU190235051013"/>
<input type="hidden" name="series_id" value="ENU190255051013"/>
<input type="hidden" name="series_id" value="ENU190275051013"/>
<input type="hidden" name="series_id" value="ENU190295051013"/>
<input type="hidden" name="series_id" value="ENU190315051013"/>
<input type="hidden" name="series_id" value="ENU190335051013"/>
<input type="hidden" name="series_id" value="ENU190355051013"/>
<input type="hidden" name="series_id" value="ENU190375051013"/>
<input type="hidden" name="series_id" value="ENU190395051013"/>
<input type="hidden" name="series_id" value="ENU190415051013"/>
<input type="hidden" name="series_id" value="ENU190435051013"/>
<input type="hidden" name="series_id" value="ENU190455051013"/>
<input type="hidden" name="series_id" value="ENU190475051013"/>
<input type="hidden" name="series_id" value="ENU190495051013"/>
<input type="hidden" name="series_id" value="ENU190515051013"/>
<input type="hidden" name="series_id" value="ENU190535051013"/>
<input type="hidden" name="series_id" value="ENU190555051013"/>
<input type="hidden" name="series_id" value="ENU190575051013"/>
<input type="hidden" name="series_id" value="ENU190595051013"/>
<input type="hidden" name="series_id" value="ENU190615051013"/>
<input type="hidden" name="series_id" value="ENU190635051013"/>
<input type="hidden" name="series_id" value="ENU190655051013"/>
<input type="hidden" name="series_id" value="ENU190675051013"/>
<input type="hidden" name="series_id" value="ENU190695051013"/>
<input type="hidden" name="series_id" value="ENU190715051013"/>
<input type="hidden" name="series_id" value="ENU190735051013"/>
<input type="hidden" name="series_id" value="ENU190755051013"/>
<input type="hidden" name="series_id" value="ENU190775051013"/>
<input type="hidden" name="series_id" value="ENU190795051013"/>
<input type="hidden" name="series_id" value="ENU190815051013"/>
<input type="hidden" name="series_id" value="ENU190835051013"/>
<input type="hidden" name="series_id" value="ENU190855051013"/>
<input type="hidden" name="series_id" value="ENU190875051013"/>
<input type="hidden" name="series_id" value="ENU190895051013"/>
<input type="hidden" name="series_id" value="ENU190915051013"/>
<input type="hidden" name="series_id" value="ENU190935051013"/>
<input type="hidden" name="series_id" value="ENU190955051013"/>
<input type="hidden" name="series_id" value="ENU190975051013"/>
<input type="hidden" name="series_id" value="ENU190995051013"/>
<input type="hidden" name="series_id" value="ENU191015051013"/>
<input type="hidden" name="series_id" value="ENU191035051013"/>
<input type="hidden" name="series_id" value="ENU191055051013"/>
<input type="hidden" name="series_id" value="ENU191075051013"/>
<input type="hidden" name="series_id" value="ENU191095051013"/>
<input type="hidden" name="series_id" value="ENU191115051013"/>
<input type="hidden" name="series_id" value="ENU191135051013"/>
<input type="hidden" name="series_id" value="ENU191155051013"/>
<input type="hidden" name="series_id" value="ENU191175051013"/>
<input type="hidden" name="series_id" value="ENU191195051013"/>
<input type="hidden" name="series_id" value="ENU191215051013"/>
<input type="hidden" name="series_id" value="ENU191235051013"/>
<input type="hidden" name="series_id" value="ENU191255051013"/>
<input type="hidden" name="series_id" value="ENU191275051013"/>
<input type="hidden" name="series_id" value="ENU191295051013"/>
<input type="hidden" name="series_id" value="ENU191315051013"/>
<input type="hidden" name="series_id" value="ENU191335051013"/>
<input type="hidden" name="series_id" value="ENU191355051013"/>
<input type="hidden" name="series_id" value="ENU191375051013"/>
<input type="hidden" name="series_id" value="ENU191395051013"/>
<input type="hidden" name="series_id" value="ENU191415051013"/>
<input type="hidden" name="series_id" value="ENU191435051013"/>
<input type="hidden" name="series_id" value="ENU191455051013"/>
<input type="hidden" name="series_id" value="ENU191475051013"/>
<input type="hidden" name="series_id" value="ENU191495051013"/>
<input type="hidden" name="series_id" value="ENU191515051013"/>
<input type="hidden" name="series_id" value="ENU191535051013"/>
<input type="hidden" name="series_id" value="ENU191555051013"/>
<input type="hidden" name="series_id" value="ENU191575051013"/>
<input type="hidden" name="series_id" value="ENU191595051013"/>
<input type="hidden" name="series_id" value="ENU191615051013"/>
<input type="hidden" name="series_id" value="ENU191635051013"/>
<input type="hidden" name="series_id" value="ENU191655051013"/>
<input type="hidden" name="series_id" value="ENU191675051013"/>
<input type="hidden" name="series_id" value="ENU191695051013"/>
<input type="hidden" name="series_id" value="ENU191715051013"/>
<input type="hidden" name="series_id" value="ENU191735051013"/>
<input type="hidden" name="series_id" value="ENU191755051013"/>
<input type="hidden" name="series_id" value="ENU191775051013"/>
<input type="hidden" name="series_id" value="ENU191795051013"/>
<input type="hidden" name="series_id" value="ENU191815051013"/>
<input type="hidden" name="series_id" value="ENU191835051013"/>
<input type="hidden" name="series_id" value="ENU191855051013"/>
<input type="hidden" name="series_id" value="ENU191875051013"/>
<input type="hidden" name="series_id" value="ENU191895051013"/>
<input type="hidden" name="series_id" value="ENU191915051013"/>
<input type="hidden" name="series_id" value="ENU191935051013"/>
<input type="hidden" name="series_id" value="ENU191955051013"/>
<input type="hidden" name="series_id" value="ENU191975051013"/>
</FORM>

Link to comment
Share on other sites
  • 2 weeks later...
qsek Universalist

qsek

Posted
Posted (edited)

Some people dont even...

edit: found the code in the screen in another topic about the same issue :oops:

and here:

and here:

Edited by qsek
Teamspeak 3 User Viewer - Quick and functional TS3 Query script, which shows online users.Cached Screenshot Deleter - Deletes older Fraps Screenshots if they exceed a specified limit.Unresolved Topics:Intercept and modify dragdrop text behaviour in scite
Link to comment
Share on other sites
aggarwal Seeker

aggarwal

Posted
  • Author
Posted

also the rest are different as they have a direct download link attached to them! but here what happens after clicking download(thats what my guess is) that it reads the data from some other source and give it out after reading as a form!! So dont know how to go about it

Link to comment
Share on other sites
aggarwal Seeker

aggarwal

Posted
  • Author
Posted

Here is my code so far - 

#include <IE.au3>
$oIE = _IECreate("http://data.bls.gov/cgi-bin/dsrv?en")
$oForm = _IEFormGetCollection($oIE, 2)
$oSelect = _IEFormElementGetObjByName($oForm, "industry_code")
_IEFormElementOptionSelect($oSelect, "1013 Manufacturing", 1, "byText")
_IEFormSubmit ($oForm)
$oForm = _IEFormGetCollection($oIE, 2)
$oSelect = _IEFormElementGetObjByName($oForm, "area_code")
_IEFormElementOptionSelect($oSelect, "19000 Iowa -- Statewide", 1, "byText")
$a=19001
_IEFormSubmit ($oForm)
$oForm = _IEFormGetCollection($oIE, 2)
$oSelect = _IEFormElementGetObjByName($oForm, "owner_code")
_IEFormElementOptionSelect($oSelect, "5 Private", 1, "byText")
_IEFormSubmit ($oForm)
$oForm = _IEFormGetCollection($oIE, 2)
$oSelect = _IEFormElementGetObjByName($oForm, "type_code")
_IEFormElementOptionSelect($oSelect, "5 Average Annual Pay", 1, "byText")
_IEFormSubmit ($oForm)
$oForm = _IEFormGetCollection($oIE, 2)
$oSelect = _IEFormElementGetObjByName($oForm, "size_code")
_IEFormElementOptionSelect($oSelect, "0 All establishment sizes", 1, "byText")
_IEFormSubmit ($oForm)
$oForm = _IEFormGetCollection($oIE, 2)
_IEFormSubmit ($oForm)
_IEFormImageClick($oIE, "More Formatting Options", "alt")
$oForm = _IEFormGetObjByName($oIE, "format")
$oSelect = _IEFormElementGetObjByName($oForm, "output_type")
_IEFormElementOptionSelect($oSelect, "Multi-series table", 1, "byText")
$oSelect = _IEFormElementGetObjByName($oForm, "delimiter")
_IEFormElementOptionSelect($oSelect, "tab delimited", 1, "byText")
_IEFormSubmit ($oForm)
sleep(5000)

I have tried clicking on the picture with image click and all but every time it shows no match!!

Link to comment
Share on other sites
  • Moderators
Melba23 Universalist

Melba23

Posted
  • Moderators
Posted

aggarwal,

If you open yet another thread on this topic you will find them all locked - 4 is quite enough! :oops:

M23

Public_Domain.png.2d871819fcb9957cf44f4514551a2935.png Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind

Open spoiler to see my UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Link to comment
Share on other sites
  • Moderators
Melba23 Universalist

Melba23

Posted
  • Moderators
Posted

aggarwal,

Perhaps no-one knows the answer! :bye:

But do not even think about opening another thread. :oops:

M23

Public_Domain.png.2d871819fcb9957cf44f4514551a2935.png Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind

Open spoiler to see my UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Link to comment
Share on other sites
DaleHohm Universalist

DaleHohm

Posted
Posted

The thing to click is this:

<input style="vertical-align: middle;" id="download_xls" alt="Download as an Excel File" value="Download as an Excel File" src="/images/buttons/download_button_xls.gif" type="image">

Hace you looked at _IEFormImageClick?

Dale

p.s. You are having trouble here because you have framed your question very poorly. Try to simplify things and ask very specific questions. I skipped over your post many times because you made it so hard for me to understand what you were asking.

Free Internet Tools: DebugBar, AutoIt IE Builder, HTTP UDF, MODIV2, IE Developer Toolbar, IEDocMon, Fiddler, HTML Validator, WGet, curl

MSDN docs: InternetExplorer Object, Document Object, Overviews and Tutorials, DHTML Objects, DHTML Events, WinHttpRequest, XmlHttpRequest, Cross-Frame Scripting, Office object model

Automate input type=file (Related)

Alternative to _IECreateEmbedded? better: _IECreatePseudoEmbedded  Better Better?

IE.au3 issues with Vista - Workarounds

SciTe Debug mode - it's magic: #AutoIt3Wrapper_run_debug_mode=Y

Doesn't work needs to be ripped out of the troubleshooting lexicon. It means that what you tried did not produce the results you expected. It begs the questions 1) what did you try?, 2) what did you expect? and 3) what happened instead?

Reproducer: a small (the smallest?) piece of stand-alone code that demonstrates your trouble

Link to comment
Share on other sites
DaleHohm Universalist

DaleHohm

Posted
Posted

Looking at the site again, you'd be better off choosing "More Formatting Options", then select to download as comma delimited text. It then puts your results right on the page in a <pre> block that you can retrieve and parse.

Dale

Free Internet Tools: DebugBar, AutoIt IE Builder, HTTP UDF, MODIV2, IE Developer Toolbar, IEDocMon, Fiddler, HTML Validator, WGet, curl

MSDN docs: InternetExplorer Object, Document Object, Overviews and Tutorials, DHTML Objects, DHTML Events, WinHttpRequest, XmlHttpRequest, Cross-Frame Scripting, Office object model

Automate input type=file (Related)

Alternative to _IECreateEmbedded? better: _IECreatePseudoEmbedded  Better Better?

IE.au3 issues with Vista - Workarounds

SciTe Debug mode - it's magic: #AutoIt3Wrapper_run_debug_mode=Y

Doesn't work needs to be ripped out of the troubleshooting lexicon. It means that what you tried did not produce the results you expected. It begs the questions 1) what did you try?, 2) what did you expect? and 3) what happened instead?

Reproducer: a small (the smallest?) piece of stand-alone code that demonstrates your trouble

Link to comment
Share on other sites
aggarwal Seeker

aggarwal

Posted
  • Author
Posted (edited)

Hi Dale

Thanks A lot for the clarrification and I will keep things in mind from now on. But As you said I tried using _IEFormImageClick but to no success. I always ended up getting No Match message. And I want to download the data in my computer every quaterly basis so that I can use it for the next step and I want no human interaction involved in it. So I still think the best option would be to click on that line you pointed if we can.

Abhishek Aggarwal

Edited by aggarwal
Link to comment
Share on other sites
DaleHohm Universalist

DaleHohm

Posted
Posted

You show no code where you are trying to _IEFormImageClick the download_xls button.

I still suggest you'll have better luck with the more formatting options section.

Dale

Free Internet Tools: DebugBar, AutoIt IE Builder, HTTP UDF, MODIV2, IE Developer Toolbar, IEDocMon, Fiddler, HTML Validator, WGet, curl

MSDN docs: InternetExplorer Object, Document Object, Overviews and Tutorials, DHTML Objects, DHTML Events, WinHttpRequest, XmlHttpRequest, Cross-Frame Scripting, Office object model

Automate input type=file (Related)

Alternative to _IECreateEmbedded? better: _IECreatePseudoEmbedded  Better Better?

IE.au3 issues with Vista - Workarounds

SciTe Debug mode - it's magic: #AutoIt3Wrapper_run_debug_mode=Y

Doesn't work needs to be ripped out of the troubleshooting lexicon. It means that what you tried did not produce the results you expected. It begs the questions 1) what did you try?, 2) what did you expect? and 3) what happened instead?

Reproducer: a small (the smallest?) piece of stand-alone code that demonstrates your trouble

Link to comment
Share on other sites
aggarwal Seeker

aggarwal

Posted
  • Author
Posted

You show no code where you are trying to _IEFormImageClick the download_xls button.

I still suggest you'll have better luck with the more formatting options section.

Dale

Thanks a lot for the help. I was able to use more formatting option and retrieve the table data with _IETableGetCollection, _IETableWriteToArray and writting them to an excel file.

Thanks A lot

Abhishek

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share
Go to topic listing
×
×
  • Create New...