Jump to content

Getting a data set from a webpage


Recommended Posts

I'm attempting to make a script which will pull the relevant pieces of data from a Patreon post I have finished making, and then place that data into a new post with some of the info altered so that I can quickly make additional posts without having to manually type in things such as the date/time/tags/etc. 

My problem is a fundamental cluelessness of where to begin. I am happy to learn how to do it myself but I have no idea of what terms to use when searching to find what I am trying to accomplish. Could anyone point me in the right direction?

A breakdown of my plans:
1. Write initial post, input title, body text, tags, schedule it for the proper date and time manually.
2. Use autoit to copy the post title, tags, date and time
3. Open a new tab using the URL for the new post page, paste the title, tags and set the date and time, press the button that saves the post/sets it for scheduling.

Is autoit a suitable tool for this task? Can anyone give me some search terms or some basic fundamental concepts of what I should search for to accomplish this?
Any direction is appreciated!

Link to post
Share on other sites

I have found that doing a CTRL+A does copy all of the relevant data that I need and am able to paste that data into a notepad file in order to parse through it. But I'm wondering if I'm on the right path or if this is a ridiculously inefficient method of solving my issue.

Link to post
Share on other sites
16 hours ago, EagerLearner said:

Patreon post

What this is?

16 hours ago, EagerLearner said:

Write initial post, input title, body text, tags, schedule it for the proper date and time manually

Is it the same kind of 'post' type ?

 

Edited by mLipok

Signature beginning:
Please remember: "AutoIt"..... *  Wondering who uses AutoIt and what it can be used for ? * Forum Rules *
ADO.au3 UDF * POP3.au3 UDF * XML.au3 UDF * for other useful stuff click the following button:

Spoiler

Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind. 

My contribution (my own projects): * Debenu Quick PDF Library - UDF * Debenu PDF Viewer SDK - UDF * Acrobat Reader - ActiveX Viewer * UDF for PDFCreator v1.x.x * XZip - UDF * AppCompatFlags UDF * CrowdinAPI UDF * _WinMergeCompare2Files() * _JavaExceptionAdd() * _IsBeta() * Writing DPI Awareness App - workaround * _AutoIt_RequiredVersion() * Chilkatsoft.au3 UDF * TeamViewer.au3 UDF * JavaManagement UDF * VIES over SOAP * WinSCP UDF * GHAPI UDF - modest begining - comunication with GitHub REST APIErrorLog.au3 UDF - A logging Library * Include Dependency Tree (Tool for analyzing script relations) *

My contribution to others projects or UDF based on  others projects: * _sql.au3 UDF  * POP3.au3 UDF *  RTF Printer - UDF * XML.au3 UDF * ADO.au3 UDF SMTP Mailer UDF * Dual Monitor resolution detection * * 2GUI on Dual Monitor System * _SciLexer.au3 UDF * SciTE - Lexer for console pane

Useful links: * Forum Rules * Forum etiquette *  Forum Information and FAQs * How to post code on the forum * AutoIt Online Documentation * AutoIt Online Beta Documentation * SciTE4AutoIt3 getting started * Convert text blocks to AutoIt code * Games made in Autoit * Programming related sites * Polish AutoIt Tutorial * DllCall Code Generator * 

Wiki: Expand your knowledge - AutoIt Wiki * Collection of User Defined Functions * How to use HelpFile * Good coding practices in AutoIt * 

OpenOffice/LibreOffice/XLS Related: WriterDemo.au3 * XLS/MDB from scratch with ADOX

IE Related:  * How to use IE.au3  UDF with  AutoIt v3.3.14.x * Why isn't Autoit able to click a Javascript Dialog? * Clicking javascript button with no ID * IE document >> save as MHT file * IETab Switcher (by LarsJ ) * HTML Entities * _IEquerySelectorAll() (by uncommon) * IE in TaskSchedulerIE Embedded Control Versioning (use IE9+ and HTML5 in a GUI) * PDF Related:How to get reference to PDF object embeded in IE *

I encourage you to read: * Global Vars * Best Coding Practices * Please explain code used in Help file for several File functions * OOP-like approach in AutoIt * UDF-Spec Questions *  EXAMPLE: How To Catch ConsoleWrite() output to a file or to CMD *

I also encourage you to check awesome @trancexx code:  * Create COM objects from modules without any demand on user to register anything. * Another COM object registering stuffOnHungApp handlerAvoid "AutoIt Error" message box in unknown errors  * HTML editor

winhttp.au3 related : * https://www.autoitscript.com/forum/topic/206771-winhttpau3-download-problem-youre-speaking-plain-http-to-an-ssl-enabled-server-port/

"Homo sum; humani nil a me alienum puto" - Publius Terentius Afer
"Program are meant to be read by humans and only incidentally for computers and execute" - Donald Knuth, "The Art of Computer Programming"
:naughty:  :ranting:, be  :) and       \\//_.

Anticipating Errors :  "Any program that accepts data from a user must include code to validate that data before sending it to the data store. You cannot rely on the data store, ...., or even your programming language to notify you of problems. You must check every byte entered by your users, making sure that data is the correct type for its field and that required fields are not empty."

Signature last update: 2021-10-18

Link to post
Share on other sites

I'm not sure if I'm allowed to share URLs, but it's Patreon dot com - commonly used by creators to find support for their art. Somewhat like a blog that people pay to access. Here is an example image of a post that is scheduled to be posted soon. 

btVhYZb.jpg

 

1 hour ago, mLipok said:

Is it the same kind of 'post' type ?

 

Yes, it is identical. I make a full post which contains the image gallery for the members of my site, and then I make a duplicate post which is set to "public" so that users who are not members are able to have some idea of what they are paying for upon signup. I'm trying to speed up the process of making these preview posts since I have to make several a week. I just would like to have most of the relevant data filled out instead of manually copy pasting and setting up things like the date and time since the preview posts are always released 5 minutes after the initial post. 

Link to post
Share on other sites
#include <INet.au3> ; needed for get source (hmtl)
#include <String.au3> ; needed for stringbetween
#include <MsgBoxConstants.au3>
#include <File.au3>
#include <Process.au3>


;Grabbing All Patreon Data & Entering it into a Notepad
AllPageData()
;Open new tab to make new post
OpenNewPost()

Func AllPageData()
    ;
    Send("^a")
    Send("^c")
    ; Store current clipboard data in a variable.
    Local $allText = ClipGet()

    ; Display the data returned by ClipGet.
    MsgBox($MB_SYSTEMMODAL, "", "The following data is stored in the clipboard: " & @CRLF & $allText)

EndFunc   ;==>AllPageData

Func OpenNewPost()
    ;grabbing the windows process name and converting to "chrome.exe/firefox.exe" format
    Local $iPid = WinGetProcess("[ACTIVE]", "")
    Local $currentWindow = _ProcessGetName($iPid)

    ;open the new post page using the active browser window
    ShellExecute($currentWindow,"https://www.patreon.com/posts/new")
EndFunc

Here is where I've gotten so far. I wanted to keep my code dynamic and use whichever browser window I was currently in, as I use multiple browsers to keep multiple logins for multiple accounts. I have it currently working to open the proper URL within whatever browser is active at the moment...but now I need to click this button and everything I find is for using IE commands as far as I can tell. 

Is there a way to simply find the button in the active window and click it without having to specify which browser?

Here is the button I'm trying to click
PiN0Ldj.jpg


I don't know anything about HTML but when I do an "inspect element" and hover this button, this is the text returned, hoping the answer is in here somewhere. Please let me know if I'm grabbing the wrong portion of text.

<div id="post_type_button_image_file" tabindex="0" role="button" class="sc-925rxp-1 jIGhQS"><div class="sc-crrsfI ejVMUJ"><span class="sc-iqHYGH fpJpwB"><svg viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><path clip-rule="evenodd" d="M14 17.333A4.667 4.667 0 1 0 14 8a4.667 4.667 0 0 0 0 9.333z" data-fill="1" data-stroke="1" fill-rule="evenodd" stroke-linecap="round" stroke-linejoin="round"></path><path clip-rule="evenodd" d="M18.667 6.667l-.965-1.93A1.333 1.333 0 0 0 16.51 4h-5.018c-.505 0-.967.285-1.193.737l-.965 1.93h-6C2.597 6.667 2 7.264 2 8v10.667C2 19.403 2.597 20 3.333 20h17.334c.736 0 1.333-.597 1.333-1.333V8c0-.736-.597-1.333-1.333-1.333h-2z" data-stroke="1" fill-rule="evenodd" stroke-linecap="round" stroke-linejoin="round"></path><path d="M5 9.333A.333.333 0 1 1 5 10a.333.333 0 0 1 0-.667M6 6.667V5.333" data-stroke="2" stroke-linecap="round" stroke-linejoin="round"></path></svg></span></div><div class="sc-jrAGrp xPZGK"><span color="dark" class="sc-gsTCUz dZTKNA">Images</span></div></div>


Also I'm just now realizing I may have posted under the wrong sub, so mods if you can move me I would much appreciate it!

Link to post
Share on other sites

All what you was asking in first post, is possible to do in AutoIt with WebDriver UDF

 

Edited by mLipok

Signature beginning:
Please remember: "AutoIt"..... *  Wondering who uses AutoIt and what it can be used for ? * Forum Rules *
ADO.au3 UDF * POP3.au3 UDF * XML.au3 UDF * for other useful stuff click the following button:

Spoiler

Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind. 

My contribution (my own projects): * Debenu Quick PDF Library - UDF * Debenu PDF Viewer SDK - UDF * Acrobat Reader - ActiveX Viewer * UDF for PDFCreator v1.x.x * XZip - UDF * AppCompatFlags UDF * CrowdinAPI UDF * _WinMergeCompare2Files() * _JavaExceptionAdd() * _IsBeta() * Writing DPI Awareness App - workaround * _AutoIt_RequiredVersion() * Chilkatsoft.au3 UDF * TeamViewer.au3 UDF * JavaManagement UDF * VIES over SOAP * WinSCP UDF * GHAPI UDF - modest begining - comunication with GitHub REST APIErrorLog.au3 UDF - A logging Library * Include Dependency Tree (Tool for analyzing script relations) *

My contribution to others projects or UDF based on  others projects: * _sql.au3 UDF  * POP3.au3 UDF *  RTF Printer - UDF * XML.au3 UDF * ADO.au3 UDF SMTP Mailer UDF * Dual Monitor resolution detection * * 2GUI on Dual Monitor System * _SciLexer.au3 UDF * SciTE - Lexer for console pane

Useful links: * Forum Rules * Forum etiquette *  Forum Information and FAQs * How to post code on the forum * AutoIt Online Documentation * AutoIt Online Beta Documentation * SciTE4AutoIt3 getting started * Convert text blocks to AutoIt code * Games made in Autoit * Programming related sites * Polish AutoIt Tutorial * DllCall Code Generator * 

Wiki: Expand your knowledge - AutoIt Wiki * Collection of User Defined Functions * How to use HelpFile * Good coding practices in AutoIt * 

OpenOffice/LibreOffice/XLS Related: WriterDemo.au3 * XLS/MDB from scratch with ADOX

IE Related:  * How to use IE.au3  UDF with  AutoIt v3.3.14.x * Why isn't Autoit able to click a Javascript Dialog? * Clicking javascript button with no ID * IE document >> save as MHT file * IETab Switcher (by LarsJ ) * HTML Entities * _IEquerySelectorAll() (by uncommon) * IE in TaskSchedulerIE Embedded Control Versioning (use IE9+ and HTML5 in a GUI) * PDF Related:How to get reference to PDF object embeded in IE *

I encourage you to read: * Global Vars * Best Coding Practices * Please explain code used in Help file for several File functions * OOP-like approach in AutoIt * UDF-Spec Questions *  EXAMPLE: How To Catch ConsoleWrite() output to a file or to CMD *

I also encourage you to check awesome @trancexx code:  * Create COM objects from modules without any demand on user to register anything. * Another COM object registering stuffOnHungApp handlerAvoid "AutoIt Error" message box in unknown errors  * HTML editor

winhttp.au3 related : * https://www.autoitscript.com/forum/topic/206771-winhttpau3-download-problem-youre-speaking-plain-http-to-an-ssl-enabled-server-port/

"Homo sum; humani nil a me alienum puto" - Publius Terentius Afer
"Program are meant to be read by humans and only incidentally for computers and execute" - Donald Knuth, "The Art of Computer Programming"
:naughty:  :ranting:, be  :) and       \\//_.

Anticipating Errors :  "Any program that accepts data from a user must include code to validate that data before sending it to the data store. You cannot rely on the data store, ...., or even your programming language to notify you of problems. You must check every byte entered by your users, making sure that data is the correct type for its field and that required fields are not empty."

Signature last update: 2021-10-18

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...