_FileDragDrop.au3 works for Notepad++ but not for Firefox

[MakzNovice]

I have a simple code that takes file "C:\Temp\File1.txt" and pastes it in drop area of specific firefox location ( Note : The drag drop works manually meaning firefox window supports drag and drop)  . The return value is true with no error indicating that operation successful.

The results are not getting updated in firefox window even after exe execution completion and sleep time 20secs.

Attached my code snippet and _FileDragDrop.au3 + text file it works 

#include "_FileDragDrop.au3"
Opt("WinTitleMatchMode", 2) ;1=start, 2=subStr, 3=exact, 4=advanced, -1 to -4=Nocase
Opt("MouseCoordMode", 0) ;1=absolute, 0=relative, 2=client

$File1 = "E:\AutoIt\DragDropFile\File1.txt"
$File2 = "E:\AutoIt\DragDropFile\File2.txt"
If ProcessExists("Firefox.exe") Then
    $hWnd = WinGetHandle("Mozilla Firefox") ; My FF tab title substring
    WinActivate($hWnd)
Else
    MsgBox(64,"Error","No Firefox window found",10)
EndIf


$hNPPlus=WinGetHandle("[CLASS:Notepad++]")
WinActivate($hNPPlus)
$iRet = _FileDragDrop($hNPPlus, $File1, 75, 274)
ConsoleWrite("Return: "&$iRet&" @error="&@error&", @extended="&@extended&@CRLF)
Sleep(20000)
;~ Above code to add in Notepad++ works fine

;~ Cordinate from drop file 1 - 75, 274
$iRet = _FileDragDrop($hWnd, $File1, 75, 274,'|',False)
ConsoleWrite("Return: "&$iRet&" @error="&@error&", @extended="&@extended&@CRLF)
Sleep(20000)
;~ Above code to add file in firefox with title Mozilla firefox DOES NOT work

Exit

 

Repose from code -->

Press Ctrl+Alt+Break to Restart or Ctrl+Break to Stop
Return: True @error=0, @extended=0

Return: True @error=0, @extended=0

 

_FileDragDrop.au3 5.25 kB · 177 downloads File1.txt 14 B · 153 downloads

Edited January 21, 2022 by MakzNovice
Typo in snippet

[mLipok]

Take a look for: 

_WD_SelectFiles()

in WebDriver UDF.

[MakzNovice]

@mLipok

I am trying the webdriver UDF as given in sample for firefox, I am stuck at very basic step though..

1. I have my firefox session page already launched, how can I gets associate it Session created by _WD_CreateSession?

_WD_Navigate - opens new page, if I pass second argument as URL, which requires login and everything else( not what I desire)

I tried $hWnd = WinGetHandle($sTitleString) and then _WD_Navigate($sSession,$hWnd) but it fails too.. 

_WD_Attach - looks for page by title created in session and fails

 

My aim is to read the page that I already have, but all the _WD_* operations seem to need session id created using _WD_CreateSession.

I want to know how can I read existing firefox webpage and be able to call methods like _WD_SelectFiles or _WD_FindElements

Any snippet would be of great help here

 

2. In _WD_SelectFiles($sSession, $sStrategy, $sSelector, $sFilename) , what types of drop are supported

As per documentation of method -   $sSelector - Value to find. Should point to element of type '< input type="file" >'

The information that I have about by drop location by developer tools is - <div id='divFiledrop' class='dropArea'....

does the class gets supported by this method ?

 

Any response will be highly appreciated

Edited January 24, 2022 by MakzNovice

[mLipok]

  On 1/24/2022 at 9:41 AM, MakzNovice said:

I have my firefox session page already launched, how can I gets associate it Session created by _WD_CreateSession?

Expand  

_WD_CreateSession() create new session not the one you have already launched manually.

  On 1/24/2022 at 9:41 AM, MakzNovice said:

_WD_Navigate - opens new page _WD_Attach - looks for page by title created in session and fails

Expand  

_WD_Attach() can attach only to sessions started with WebDriver not manually.

 

[MakzNovice]

@mLipok

So is there a way to get handle of existing page or no option to get it?

[SOLVE-SMART]

Hi MakzNovice,

  On 1/24/2022 at 11:25 AM, MakzNovice said:

So is there a way to get handle of existing page or no option to get it?

Expand  

When you mean you would like to use your local and manually opened Firefox browser, then basically no, not as far as I understood it.

• You start with a new WebDriver instance (other then your manually opened Firefox browser).
• You do all the navigation and probably clicking steps to reach your target drop area like you described in the first post.
• Then you are at the point to do your drag and drop actions by WebDriver functions.

Please correct me if I am wrong @mLipok or @Danp2, but stealing a browser session into the opened WebDriver session isn't possible, am I right?
At least not without specific security-attacks or something ... I don't know.

Best regards
Sven

________________
Stay innovative!

Edited January 24, 2022 by SOLVE-SMART

[SOLVE-SMART]

Hi MakzNovice again,

  On 1/24/2022 at 9:41 AM, MakzNovice said:

2. In _WD_SelectFiles($sSession, $sStrategy, $sSelector, $sFilename) , what types of drop are supported

As per documentation of method -   $sSelector - Value to find. Should point to element of type '< input type="file" >'

The information that I have about by drop location by developer tools is - <div id='divFiledrop' class='dropArea'....

does the class gets supported by this method ?

Expand  

Please have a look into wd_demo.au3, function DemoUpload. I guess this will answer or at least help you with your question 😀 .

Best regards
Sven

________________
Stay innovative!

[MakzNovice]

Well if that is the case it seems , approach of Webdriver is going to be less useful for my case.

Because the first page is login credentials and which in my case I may not have, if others launches the executable.

Looking forward from response/confirmation from @mLipok or @Danp2

Is there any other way to achieve my requirement to upload files to existing webpage in firefox?

Edited January 24, 2022 by MakzNovice

[SOLVE-SMART]

Hmm, get it,

  On 1/24/2022 at 12:44 PM, MakzNovice said:

Because the first page is login credentials and which in my case I may not have, if user launches the executable.

Expand  

Okay then maybe a non elegant way by mouse actions (moves) 🥴 in case you can not ask the user to enter credentials in forehand.

Just thinking about what you would do manually. You would click your file within a folder and "drag and drop" it to the Firefox window (into the drop area). When you arrange the Browser window size and position, this shouldn't be very flaky when it comes to execution on different monitor resolutions. But still ugly in my opinion.

Best regards
Sven

________________
Stay innovative!

[mLipok]

  On 1/24/2022 at 12:44 PM, MakzNovice said:

Because the first page is login credentials and which in my case I may not have, if others launches the executable.

Expand  

So you have no right to do there anything?
 

[MakzNovice]

@SOLVE-SMART

That is what I tried in my first script, mouse clicks, but it won't work on firefox webpage

Really need to find a way as good as this _WD_SelectFiles.. if somehow can just crack a way to get handle of existing webpage... 

[MakzNovice]

@mLipok

It will work on my machine with my credentials, but would be problematic if other users use the script.

Also, I tried next steps manually and it takes almost 8-9 more steps to reach the page I need too...

Needless to say the page is dynamic, clicks are to be decided by other user actions... So its tough to automate logically till this page.

[mLipok]

  On 1/24/2022 at 1:13 PM, MakzNovice said:

It will work on my machine with my credentials, but would be problematic if other users use the script.

Expand  

You should do one of the two things:
1. provide a way to put login/password by user to your program and thus to webpage.
2. open browser, display message to login manually, wait for login.... do the stuff.
 

  On 1/24/2022 at 1:13 PM, MakzNovice said:

Also, I tried next steps manually and it takes almost 8-9 more steps to reach the page I need too...

Expand  

and where is the problem ?

  On 1/24/2022 at 1:13 PM, MakzNovice said:

Needless to say the page is dynamic, clicks are to be decided by other user actions... So its tough to automate logically till this page.

Expand  

automate it tooo or: 2. open browser, display message to login manually, wait for login, wait for manually provided stuff .... do the automation stuff.

 

[MakzNovice]

@mLipok

Sorry I just confirmed one thing, I do not have option retrieve/ask for user credentials..

I am automating very small process of bigger tool, which has user already logged in and then this a part of internal operation.

- But seems webdriver approach is not going to work since we can't get existing page details..

Sorry again for my poor English which did not clarify that user credentials can't be requested.

[mLipok]

so only InternetExplorer.....

 

[MakzNovice]

@mLipok

I wish we could use IE..

But as you know due to various issue the only browser allowed is Firefox. 

[Danp2]

  On 1/24/2022 at 9:41 AM, MakzNovice said:

1. I have my firefox session page already launched, how can I gets associate it Session created by _WD_CreateSession?

Expand  

You cant unless the existing firefox instance was launched with Marionette enabled. See the following for more details --

  On 1/24/2022 at 9:41 AM, MakzNovice said:

In _WD_SelectFiles($sSession, $sStrategy, $sSelector, $sFilename) , what types of drop are supported As per documentation of method -   $sSelector - Value to find. Should point to element of type '< input type="file" >' The information that I have about by drop location by developer tools is - <div id='divFiledrop' class='dropArea'....does the class gets supported by this method ?

Expand  

As you noted, _WD_SelectFiles is designed to work with a specific type of element, so it likely will not work with your target.

 

[SOLVE-SMART]

Hi MakzNovice,

here is a working example of pure mouse and window drag and drop usage which works also for Firefox.

  Reveal hidden contents

expandcollapse popup

; opt --------------------------------------------------------------------------
Opt('MouseCoordMode', 0)
Opt('MustDeclareVars', 1)
Opt('WinTitleMatchMode', 2)



; declaration ------------------------------------------------------------------
Global $sTitle                 = 'FileUploadDemo'
Global $sTargetFolderOfFiles   = @TempDir & '\' & $sTitle & '\'

Global $iFolderWindowWidth     = 500
Global $iFolderWindowHeight    = 500

Global $iSearchRectangleLeft   = 166
Global $iSearchRectangleTop    = 196
Global $iSearchRectangleRight  = 472
Global $iSearchRectangleBottom = 443
Global $vSearchColor           = 0xCCE8FF

Global $iTargetXPosition       = 3200
Global $iTargetYPosition       = 492



; processing -------------------------------------------------------------------
DirCreate($sTargetFolderOfFiles)

_writeFile($sTargetFolderOfFiles & 'file1.txt', 'Demo purpose 1')
_writeFile($sTargetFolderOfFiles & 'file2.txt', 'Demo purpose 2')
_writeFile($sTargetFolderOfFiles & 'file3.txt', 'Demo purpose 3')

_openFolder($sTargetFolderOfFiles)
_setExplorerFolderWindowPosition()
_selectFirstFileInOpenedFolder()
_dragAndDropByPixelColor()



; functions --------------------------------------------------------------------
Func _writeFile($sFile, $sText)
    Local Const $iUtf8WithoutBomAndOverwriteCreationMode = 256 + 2 + 8

    Local $hFile = FileOpen($sFile, $iUtf8WithoutBomAndOverwriteCreationMode)
    FileWrite($hFile, $sText)
    FileClose($hFile)
EndFunc

Func _openFolder($sFolderPath = @DesktopDir)
    ShellExecute($sFolderPath)
EndFunc

Func _setExplorerFolderWindowPosition()
    WinActivate($sTitle)
    WinWaitActive($sTitle, '', 2)
    WinMove($sTitle, '', 0, 0, $iFolderWindowWidth, $iFolderWindowHeight)
EndFunc

Func _selectFirstFileInOpenedFolder()
    Send('{DOWN}')
    Sleep(100)
    Send('{UP}')
EndFunc

Func _dragAndDropByPixelColor()
    Sleep(1000)

    Local $aCoordinates   = PixelSearch($iSearchRectangleLeft, $iSearchRectangleTop, $iSearchRectangleRight, $iSearchRectangleBottom, $vSearchColor)
    Local $iOffset        = 5
    Local $iFromXPosition = $aCoordinates[0] + $iOffset
    Local $iFromYPosition = $aCoordinates[1] + $iOffset

    _mouseDragAndDrop($iFromXPosition, $iFromYPosition, $iTargetXPosition, $iTargetYPosition)
EndFunc

Func _mouseDragAndDrop($iFromX, $iFromY, $iToX, $iToY, $iSpeed = 30)
    ;~ MouseClickDrag() isn't working, that's why I add this function

    Local $sMouseKey = 'primary'

    MouseMove($iFromX, $iFromY, $iSpeed)
    MouseDown($sMouseKey)

    MouseMove($iToX, $iToY, $iSpeed)
    MouseUp($sMouseKey)
EndFunc

 

I don't like it and would stick with WebDriver, but this could help 😅 . I guess you have to improve and adjust this example to fit your needs, but it works.

By the way @Danp2, thanks for the hint about Marionette. I wasn't aware of this by now - interesting 👍 .

Best regards
Sven

________________
Stay innovative!

Edited January 24, 2022 by SOLVE-SMART

[MakzNovice]

@SOLVE-SMART

Yes it indeed fit my requirement, quick and effective fix for long term problem.

I had to change my program to meet my needs as you mentioned.

I made it slightly more robust by skipping pixel search and instead using approach of source and target Top-Left corner coordinates (WinGetPos) + relative target location to finalize my X and Y.

Also confirmed that Firefox window will be 100% zoom. so chances of error are reduced. 

My code for reference

expandcollapse popup

#include <MsgBoxConstants.au3>

Opt('MouseCoordMode', 0)
Opt('MustDeclareVars', 1)
Opt('WinTitleMatchMode', 2)

; declaration ------------------------------------------------------------------
Global $sTitle                 = 'FileUploadDemo'
Global $sTargetFolderOfFiles   = @TempDir & '\' & $sTitle & '\'

Global $iFolderWindowWidth     = 200
Global $iFolderWindowHeight    = 200

Global $iFileExplorerFile1X     = 61
Global $iFileExplorerFile1Y     = 114

Global $iFileExplorerFile2X     = 61
Global $iFileExplorerFile2Y     = 134

;~ Cordinates for
Global $iTargetXPosition1       = 75
Global $iTargetYPosition1       = 274

Global $iTargetXPosition2       = 175
Global $iTargetYPosition2       = 274

Global $sTitleString = "Mozilla Firefox"
Global $hWndExplorer, $hWndFF

; processing -------------------------------------------------------------------
DirCreate($sTargetFolderOfFiles)

_writeFile($sTargetFolderOfFiles & 'file1.txt', 'upload file 1')
_writeFile($sTargetFolderOfFiles & 'file2.txt', 'upload file 2')

_openFolder($sTargetFolderOfFiles)
_setExplorerFolderWindowPosition()
_setFireFoxWindowActive()

_selectFirstFileInOpenedFolder()
_dragAndDropByPixelColor()
;~ WinClose($hWndExplorer) ; Explorer window will be closed after work is done

; functions --------------------------------------------------------------------
Func _writeFile($sFile, $sText)
    Local Const $iUtf8WithoutBomAndOverwriteCreationMode = 256 + 2 + 8
    Local $hFile = FileOpen($sFile, $iUtf8WithoutBomAndOverwriteCreationMode)
    FileWrite($hFile, $sText)
    FileClose($hFile)
EndFunc

Func _openFolder($sFolderPath = @DesktopDir)
    ShellExecute($sFolderPath)
EndFunc

Func _setFireFoxWindowActive()
    If ProcessExists("Firefox.exe") Then
    $hWndFF = WinGetHandle($sTitleString)
    WinActivate($hWndFF)
Else
    MsgBox(64,"Error","No Firefox window found, closing script")
    Exit
EndIf
EndFunc

Func _setExplorerFolderWindowPosition()
    $hWndExplorer = WinGetHandle($sTitle)
    WinActivate($sTitle)
    WinWaitActive($sTitle, '', 2)
    WinMove($sTitle, '', 0, 0, $iFolderWindowWidth, $iFolderWindowHeight)
EndFunc

Func _selectFirstFileInOpenedFolder()
    WinActivate($hWndExplorer)
    Send('{DOWN}')
    Sleep(100)
    Send('{UP}')
EndFunc

Func _dragAndDropByPixelColor()
    Sleep(1000)

    ;~ PixelSearch is not suitable for my need, lets use WinGetPos
    ;Local $aCoordinates1   = PixelSearch($iSearchRectangleLeft, $iSearchRectangleTop, $iSearchRectangleRight, $iSearchRectangleBottom, $vSearchColor)
    
    ;MsgBox(64, "", "X and Y are: " & $aCoordinates[0] & "," & $aCoordinates[1])
    ;Local $iOffset        = 5
    WinActivate($hWndExplorer)
    Local $aPosExplorer = WinGetPos($hWndExplorer)

    ;Display the array values returned by WinGetPos.
    MsgBox($MB_SYSTEMMODAL, "Explorer details", "X-Pos: " & $aPosExplorer[0] & @CRLF & _
            "Y-Pos: " & $aPosExplorer[1] & @CRLF & _
            "Width: " & $aPosExplorer[2] & @CRLF & _
            "Height: " & $aPosExplorer[3]& @CRLF, 1)
    $iFileExplorerFile1X = $aPosExplorer[0] + $iFileExplorerFile1X
    $iFileExplorerFile1Y = $aPosExplorer[1] + $iFileExplorerFile1Y

    Sleep(1000)
    WinActivate($hWndFF)
    Local $aPos = WinGetPos($hWndFF)

    ; Display the array values returned by WinGetPos.
    MsgBox($MB_SYSTEMMODAL, "FireFox Window details", "X-Pos: " & $aPos[0] & @CRLF & _
            "Y-Pos: " & $aPos[1] & @CRLF & _
            "Width: " & $aPos[2] & @CRLF & _
            "Height: " & $aPos[3]& @CRLF, 1)

    $iTargetXPosition1 = $aPos[0] + $iTargetXPosition1
    $iTargetYPosition1 = $aPos[1] + $iTargetYPosition1
    WinActivate($hWndExplorer)
    _mouseDragAndDrop($iFileExplorerFile1X, $iFileExplorerFile1Y, $iTargetXPosition1, $iTargetYPosition1)

EndFunc

Func _mouseDragAndDrop($iFromX, $iFromY, $iToX, $iToY, $iSpeed = 30)
    ;~ MouseClickDrag() isn't working, that's why I add this function

    Local $sMouseKey = 'primary'

    MouseMove($iFromX, $iFromY, $iSpeed)
    MouseDown($sMouseKey)

    MouseMove($iToX, $iToY, $iSpeed)
    MouseUp($sMouseKey)
EndFunc

I will keep hunting for better solution including webdriver approach, but this is good workaround for me.

Any further support will be appreciated.

[SOLVE-SMART]

Hi @MakzNovice,

  On 1/25/2022 at 5:22 AM, MakzNovice said:

Yes it indeed fit my requirement, quick and effective fix for long term problem.

Expand  

That's great 😀, I am glad it was helpful.
Don't be shy or don't hesitate to press one of the fancy buttons here in case you want to.



I am not focused in getting any praise, it's more a gentle way of saying thanks for your invested time 😊 .

More important:

  On 1/25/2022 at 5:22 AM, MakzNovice said:

I made it slightly more robust [...]

Expand  

I hoped that you will do this 👍 . Nice that you could get rid of the PixelSearch mechanism.

  On 1/25/2022 at 5:22 AM, MakzNovice said:

I will keep hunting for better solution including webdriver approach, but this is good workaround for me.

Expand  

Yes please do so, because the WebDriver approach is such a flexible way of doing browser interactions which will help in many cases.

Best regards
Sven

________________
Stay innovative!

Edited January 25, 2022 by SOLVE-SMART