Sign in to follow this  
Followers 0
wraithdu

Deadlock with Subclassed Button and Mouse Hook

4 posts in this topic

#1 ·  Posted (edited)

Can anyone shed some light as to why the following code deadlocks when pressing the button? I'm trying to write a toggle button library... it's way harder than it sounds. I think to get everything working I need to subclass the buttons to handle some messages, but whenever I subclass the button and click, the app deadlocks. Without subclassing, it does not. Nevermind the very incomplete code, it's just a quick reproducer. Tested with 3.3.5.3 on Win7 32-bit.

#NoTrayIcon
#include <WindowsConstants.au3>
#include <WinAPI.au3>
#include <Constants.au3>

Global $pressed = 0
Global $gui = GUICreate("Test", 400, 400)
Global $b1 = GUICtrlCreateButton("Test Button 1", 10, 10)
Global $hb1 = GUICtrlGetHandle($b1)

; subclass the button
Global $ButtonProc = DllCallbackRegister("_ButtonProc", "long_ptr", "hwnd;uint;wparam;lparam")
Global $defproc = _WinAPI_SetWindowLong($hb1, $GWL_WNDPROC, DllCallbackGetPtr($ButtonProc))
; set mouse hook
Global $MouseProc = DllCallbackRegister("_MouseProc", "long_ptr", "int;wparam;lparam")
Global $NextHook = _WinAPI_SetWindowsHookEx($WH_MOUSE, DllCallbackGetPtr($MouseProc), 0, _WinAPI_GetCurrentThreadId())

GUISetState()

Do
Until GUIGetMsg() = -3

_Exit()

Func _Exit()
    _WinAPI_UnhookWindowsHookEx($NextHook)
    _WinAPI_SetWindowLong($hb1, $GWL_WNDPROC, $defproc)
    DllCallbackFree($ButtonProc)
    DllCallbackFree($MouseProc)
    Exit
EndFunc

Func _ButtonProc($hWnd, $Msg, $wParam, $lParam)
    Return _WinAPI_CallWindowProc($defproc, $hWnd, $Msg, $wParam, $lParam)
EndFunc

Func _MouseProc($nCode, $wParam, $lParam)
    Local $ret = 0, $point, $hwnd, $tagMHS = "long;long;hwnd;uint;ulong_ptr"

    If $nCode < 0 Then Return _WinAPI_CallNextHookEx($NextHook, $nCode, $wParam, $lParam)
    Switch $wParam
        Case $WM_LBUTTONDOWN
            ; get hovered control hwnd
            $point = DllStructCreate($tagMHS, $lParam)
            $hwnd = DllStructGetData($point, 3)
            If $hwnd = $hb1 Then
                _SendMessage($hb1, 0xF3, 1) ; pressed state
                Return 1 ; block normal processing
            EndIf
        Case $WM_LBUTTONUP
            ; get hovered control hwnd
            $point = DllStructCreate($tagMHS, $lParam)
            $hwnd = DllStructGetData($point, 3)
            If $hwnd = $hb1 Then
                $pressed = Number(Not $pressed) ; toggle
                _SendMessage($hb1, 0xF3, $pressed) ; set new state
                Return 1
            EndIf
    EndSwitch
    Return _WinAPI_CallNextHookEx($NextHook, $nCode, $wParam, $lParam)
EndFunc
Edited by wraithdu

Share this post


Link to post
Share on other sites



wraithdu,

I hesitate to offer advice, but.......... :D

If you are looking to use the MouseProc to toggle the button state, do you not have to block the actual button press by the mouse within the subclassed ButtonProc? So you would need something like this:

Func _ButtonProc($hWnd, $Msg, $wParam, $lParam)
    If $Msg = ####### Then ; eat the actual button press by the mouse
        Return 0
    Else
        Return _WinAPI_CallWindowProc($defproc, $hWnd, $Msg, $wParam, $lParam)
    EndIf
EndFunc

Quite what it is you would have to trap I am not sure - what do buttons send when they are pressed? WM_COMMAND? WM_NOTIFY?

Or am I talking complete spherical objects as I usually do when it gets this deep? :huggles:

M23


Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

#3 ·  Posted (edited)

LOL, a little. The BN_CLICKED notification is sent to the parent window via WM_COMMAND. But by that point it's too late. What I'm dealing with here is the button's own window procedure that controls the drawing state. The mouse hook blocks the button from getting clicks and manually sets the pushed state via the BM_SETSTATE message.

All of this can be done with just the mouse hook. The subclassing is needed to handle keyboard navigation, specifically the button receiving WM_KILLFOCUS. That's right, the button gets WM_KILLFOCUS, the parent window gets BN_KILLFOCUS via WM_COMMAND. Notice the difference, WM = Window Message (does something), BN = Button Notification (something has happened, too late for you to do anything about it).

So yeah, I've got to get inside the button to stop it from screwing with the view state. The problem is all the things that the WM_KILLFOCUS message does -

Removes the focus rectangle from a button. For push buttons and default push buttons, the focus rectangle is invalidated. If the button has the mouse capture, the capture is released, the button is not clicked, and any push state is removed.

I've tried handling BN_KILLFOCUS and BN_CLICKED in WM_COMMAND to reset the pushed state based on saved values, but tabbing through a control that is in the pushed state first sends a BN_CLICKED notification before the BN_KILLFOCUS notification, thus resetting any saved values I might have had for the state of that button. By the time I got the BN_KILLFOCUS, I had no idea what the previous state was. So, I have to get at it earlier in the button's window procedure.

Or... I have to install a keyboard hook as well, and handle Enter/Space/Tab in some appropriate way. I'd be much better off if it didn't deadlock for whatever reason. But you're essentially right, I have to block stuff in the button window procedure. But as you can see, simply passing along the messages is deadlocking the app. This has to be solved first.

Edited by wraithdu

Share this post


Link to post
Share on other sites

wraithdu,

Only small cubes, rather than spheres then.....I am getting better! :D

M23


Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0