Subclassing Toolbar (need help with missing NMTBCUSTOMDRAW structure)

WildByDesign · Thursday at 11:53 AM

I'm at the beginning stages of trying to figure out how to subclass a toolbar (ToolbarWindow32) for potential use in GUIDarkTheme UDF.

AutoIt does not have a tag structure for $tagNMTBCUSTOMDRAW and I am stuck there. I could generally follow some of the other examples from StructureConstants.au3, however, I don't know how to handle the HBRUSH in the structure because none of the examples have it.

Link: NMTBCUSTOMDRAW structure (C++)

typedef struct _NMTBCUSTOMDRAW {
  NMCUSTOMDRAW nmcd;
  HBRUSH       hbrMonoDither;
  HBRUSH       hbrLines;
  HPEN         hpenLines;
  COLORREF     clrText;
  COLORREF     clrMark;
  COLORREF     clrTextHighlight;
  COLORREF     clrBtnFace;
  COLORREF     clrBtnHighlight;
  COLORREF     clrHighlightHotTrack;
  RECT         rcText;
  int          nStringBkMode;
  int          nHLStringBkMode;
  int          iListGap;
} NMTBCUSTOMDRAW, *LPNMTBCUSTOMDRAW;

I'll share some more details of the subclassing part soon. But for now, here is my current testing example to get started if anyone can help with the structure. Thank you.

#include <WinAPITheme.au3>
#include <ToolbarConstants.au3>
#include <GUIConstantsEx.au3>
#include <GuiToolbar.au3>
#include <StructureConstants.au3>
#include <WinAPIConstants.au3>
#include <WindowsNotifsConstants.au3>
#include <WindowsStylesConstants.au3>
#include <WinAPIShellEx.au3>

Global $g_hToolbar
Global $g_iItem ; Command identifier of the button associated with the notification.
Global Enum $e_idNew = 1000, $e_idOpen, $e_idSave, $e_idHelp

; NMTBCUSTOMDRAW:  https://learn.microsoft.com/en-us/windows/win32/api/commctrl/ns-commctrl-nmtbcustomdraw
; C++ structure
#cs
typedef struct _NMTBCUSTOMDRAW {
  NMCUSTOMDRAW nmcd;
  HBRUSH       hbrMonoDither;
  HBRUSH       hbrLines;
  HPEN         hpenLines;
  COLORREF     clrText;
  COLORREF     clrMark;
  COLORREF     clrTextHighlight;
  COLORREF     clrBtnFace;
  COLORREF     clrBtnHighlight;
  COLORREF     clrHighlightHotTrack;
  RECT         rcText;
  int          nStringBkMode;
  int          nHLStringBkMode;
  int          iListGap;
} NMTBCUSTOMDRAW, *LPNMTBCUSTOMDRAW;
#ce

; Structure for NM_CUSTOMDRAW notification
Global Const $tagNMTBCUSTOMDRAW = $tagNMHDR & ";" & _                                  ; Contains NM_CUSTOMDRAW / NMHDR header among other things
                                "dword dwDrawStage;" & _                               ; Current drawing stage (CDDS_*)
                                "handle hdc;" & _                                      ; Device Context Handle
                                "long left;long top;long right;long bottom;" & _       ; Drawing rectangle
                                "dword_ptr dwItemSpec;" & _                            ; Item index or other info (depending on the control)
                                "uint uItemState;" & _                                 ; State Flags (CDIS_SELECTED, CDIS_FOCUS etc.)
                                "lparam lItemlParam"                                   ; lParam set by the item (e.g., via LVITEM.lParam)

Example()

Func Example()
    Local $hGUI, $aSize

    ; Create GUI
    $hGUI = GUICreate("Toolbar", 600, 400)
    $g_hToolbar = _GUICtrlToolbar_Create($hGUI)
    $aSize = _GUICtrlToolbar_GetMaxSize($g_hToolbar)

    ;_WinAPI_SetWindowTheme($g_hToolbar, 'DarkMode_DarkTheme')

    ; add subclass for toolbar
    Local $hSubClass = DllCallbackRegister(WM_NOTIFY, "lresult", "hwnd;uint;wparam;lparam;uint_ptr;dword_ptr")
    _WinAPI_SetWindowSubclass($g_hToolbar, DllCallbackGetPtr($hSubClass), 1)

    GUISetState(@SW_SHOW)

    ; Add standard system bitmaps
    _GUICtrlToolbar_AddBitmap($g_hToolbar, 1, -1, $IDB_STD_LARGE_COLOR)

    ; Add buttons
    _GUICtrlToolbar_AddButton($g_hToolbar, $e_idNew, $STD_FILENEW)
    _GUICtrlToolbar_AddButton($g_hToolbar, $e_idOpen, $STD_FILEOPEN)
    _GUICtrlToolbar_AddButton($g_hToolbar, $e_idSave, $STD_FILESAVE)
    _GUICtrlToolbar_AddButtonSep($g_hToolbar)
    _GUICtrlToolbar_AddButton($g_hToolbar, $e_idHelp, $STD_HELP)

    ; Loop until the user exits.
    Do
    Until GUIGetMsg() = $GUI_EVENT_CLOSE

    _WinAPI_RemoveWindowSubclass($g_hToolbar, DllCallbackGetPtr($hSubClass), 1)
    DllCallbackFree($hSubClass)
EndFunc   ;==>Example

Func WM_NOTIFY($hWnd, $iMsg, $wParam, $lParam, $iID, $pData)
  Local Static $bHover, $tPoint
  Switch $iMsg
    Case $WM_NOTIFY
      Local $tNMHDR = DllStructCreate($tagNMHDR, $lParam)
      Switch $tNMHDR.Code
        Case $NM_CUSTOMDRAW
          If $tNMHDR.hWndFrom = $pData Then
            Local $tCustDraw = DllStructCreate($tagNMTBCUSTOMDRAW, $lParam)
            Switch $tCustDraw.dwDrawStage
              Case $CDDS_PREPAINT
                Return $CDRF_NOTIFYITEMDRAW
              Case $CDDS_ITEMPREPAINT
                Return $CDRF_SKIPDEFAULT
            EndSwitch
          EndIf
      EndSwitch
  EndSwitch
  Return _WinAPI_DefSubclassProc($hWnd, $iMsg, $wParam, $lParam)
EndFunc   ;==>WM_NOTIFY

Edited Thursday at 11:54 AM by WildByDesign

WildByDesign · Thursday at 12:00 PM

From the win32-darkmodelib C++ library:

Link: Handles custom draw notifications for a toolbar control.

What we need to handle is really quite simple from what I can tell.

/**
 * @brief Handles custom draw notifications for a toolbar control.
 *
 * Processes `NMTBCUSTOMDRAW` messages to provide custom color painting
 * at each stage of the custom draw cycle:
 * - **CDDS_PREPAINT**: Fills the toolbar background and requests item-level drawing.
 * - **CDDS_ITEMPREPAINT**: Applies custom item painting via @ref prepaintToolbarItem.
 * - **CDDS_ITEMPOSTPAINT**: Paints dropdown arrows glyphs via @ref postpaintToolbarItem.
 *
 * @param[in]   hWnd        Handle to the toolbar control.
 * @param[in]   uMsg        Should be `WM_NOTIFY` with custom draw type (forwarded to default subclass processing).
 * @param[in]   wParam      Message parameter (forwarded to default subclass processing).
 * @param[in]   lParam      Pointer to `NMTBCUSTOMDRAW`.
 * @return `LRESULT` containing draw flags or the result of default subclass processing.
 *
 * @see prepaintToolbarItem()
 * @see postpaintToolbarItem()
 */
[[nodiscard]] static LRESULT darkToolbarNotifyCustomDraw(
	HWND hWnd,
	UINT uMsg,
	WPARAM wParam,
	LPARAM lParam
) noexcept
{
	switch (auto* lptbcd = reinterpret_cast<LPNMTBCUSTOMDRAW>(lParam);
		lptbcd->nmcd.dwDrawStage)
	{
		case CDDS_PREPAINT:
		{
			::FillRect(lptbcd->nmcd.hdc, &lptbcd->nmcd.rc, dmlib::getDlgBackgroundBrush());
			return CDRF_NOTIFYITEMDRAW | CDRF_NOTIFYPOSTPAINT;
		}

		case CDDS_ITEMPREPAINT:
		{
			return prepaintToolbarItem(lptbcd);
		}

		case CDDS_ITEMPOSTPAINT:
		{

Link: Applies custom drawing to a toolbar items (buttons) during `CDDS_ITEMPOSTPAINT.

/**
 * @brief Applies custom drawing to a toolbar items (buttons) during `CDDS_ITEMPOSTPAINT.
 *
 * Paints arrow glyph with custom color over system black "down triangle" for button with style `BTNS_DROPDOWN`.
 * Triggered by `CDRF_NOTIFYPOSTPAINT` from @ref prepaintToolbarItem.
 *
 * Logic:
 * - Retrieves the drop-down rectangle via `TB_GETITEMDROPDOWNRECT`.
 * - Selects the toolbar font and draws a centered arrow glyph with custom text color.
 *
 * @param[in] lptbcd Reference to `LPNMTBCUSTOMDRAW`.
 * @return `CDRF_DODEFAULT` to let default text/icon drawing proceed normally.
 *
 * @note Only applies to iconic buttons.
 *
 * @see prepaintToolbarItem()
 * @see darkToolbarNotifyCustomDraw()
 */
[[nodiscard]] static LRESULT postpaintToolbarItem(const LPNMTBCUSTOMDRAW& lptbcd) noexcept
{
	TBBUTTONINFOW tbi{};
	tbi.cbSize = sizeof(TBBUTTONINFOW);
	tbi.dwMask = TBIF_IMAGE;
	::SendMessage(lptbcd->nmcd.hdr.hwndFrom, TB_GETBUTTONINFO, lptbcd->nmcd.dwItemSpec, reinterpret_cast<LPARAM>(&tbi));

	if (tbi.iImage == I_IMAGENONE)
	{
		return CDRF_DODEFAULT;
	}

	RECT rcArrow{};
	const auto idx = ::SendMessage(lptbcd->nmcd.hdr.hwndFrom, TB_COMMANDTOINDEX, lptbcd->nmcd.dwItemSpec, 0);
	::SendMessage(lptbcd->nmcd.hdr.hwndFrom, TB_GETITEMDROPDOWNRECT, static_cast<WPARAM>(idx), reinterpret_cast<LPARAM>(&rcArrow));
	rcArrow.left += 1;
	rcArrow.bottom -= dmlib_dpi::scale(3, lptbcd->nmcd.hdr.hwndFrom);

	::SetBkMode(lptbcd->nmcd.hdc, TRANSPARENT);
	::SetTextColor(lptbcd->nmcd.hdc, dmlib::getTextColor());

	const auto hFont = dmlib_paint::GdiObject{ lptbcd->nmcd.hdc, lptbcd->nmcd.hdr.hwndFrom };
	static constexpr UINT dtFlags = DT_CENTER | DT_VCENTER | DT_SINGLELINE | DT_NOCLIP | DT_NOPREFIX;
	::DrawText(lptbcd->nmcd.hdc, dmlib_glyph::kTriangleDown, -1, &rcArrow, dtFlags);

	return CDRF_DODEFAULT;
}

Link: Applies custom drawing to a toolbar items (buttons) during `CDDS_ITEMPREPAINT`

/**
 * @brief Applies custom drawing to a toolbar items (buttons) during `CDDS_ITEMPREPAINT`
 *
 * Handles color assignment and background painting for toolbar buttons during the
 * `CDDS_ITEMPREPAINT` stage of `NMTBCUSTOMDRAW`. Applies appropriate brushes, pens,
 * and background drawing depending on the button state:
 * - **Hot**: Uses hot background and edge styling.
 * - **Checked**: Uses control background and standard edge styling.
 * - **Drop-down**: Calculates and paints iconic split-button drop arrow.
 *
 * Also configures transparency and color usage for text, hot-tracking, and background fills.
 * Ensures hot/checked states are visually overridden by custom color highlights.
 *
 * @param[in,out] lptbcd Reference to the toolbar's custom draw structure.
 * @return Flags to control draw behavior (`TBCDRF_USECDCOLORS`, `TBCDRF_NOBACKGROUND`, `CDRF_NOTIFYPOSTPAINT`).
 *
 * @note This function clears `CDIS_HOT`/`CDIS_CHECKED` to allow manual visual overrides.
 *
 * @see postpaintToolbarItem()
 * @see darkToolbarNotifyCustomDraw()
 */
[[nodiscard]] static LRESULT prepaintToolbarItem(LPNMTBCUSTOMDRAW& lptbcd) noexcept
{
	// Set colors

	lptbcd->hbrMonoDither = dmlib::getBackgroundBrush();
	lptbcd->hbrLines = dmlib::getEdgeBrush();
	lptbcd->hpenLines = dmlib::getEdgePen();
	lptbcd->clrText = dmlib::getDarkerTextColor();
	lptbcd->clrTextHighlight = dmlib::getTextColor();
	lptbcd->clrBtnFace = dmlib::getBackgroundColor();
	lptbcd->clrBtnHighlight = dmlib::getCtrlBackgroundColor();
	lptbcd->clrHighlightHotTrack = dmlib::getHotBackgroundColor();
	lptbcd->nStringBkMode = TRANSPARENT;
	lptbcd->nHLStringBkMode = TRANSPARENT;

	// Get styles and rectangles

	const bool isHot = (lptbcd->nmcd.uItemState & CDIS_HOT) == CDIS_HOT;
	const bool isChecked = (lptbcd->nmcd.uItemState & CDIS_CHECKED) == CDIS_CHECKED;

	RECT rcItem{ lptbcd->nmcd.rc };
	RECT rcDrop{};

	TBBUTTONINFOW tbi{};
	tbi.cbSize = sizeof(TBBUTTONINFOW);
	tbi.dwMask = TBIF_IMAGE | TBIF_STYLE;
	::SendMessage(lptbcd->nmcd.hdr.hwndFrom, TB_GETBUTTONINFO, lptbcd->nmcd.dwItemSpec, reinterpret_cast<LPARAM>(&tbi));

	const bool isIcon = tbi.iImage != I_IMAGENONE;
	const bool isDropDown = ((static_cast<WORD>(tbi.fsStyle) & BTNS_DROPDOWN) == BTNS_DROPDOWN) && isIcon; // has 2 "buttons"
	if (isDropDown)
	{
		const auto idx = ::SendMessage(lptbcd->nmcd.hdr.hwndFrom, TB_COMMANDTOINDEX, lptbcd->nmcd.dwItemSpec, 0);
		::SendMessage(lptbcd->nmcd.hdr.hwndFrom, TB_GETITEMDROPDOWNRECT, static_cast<WPARAM>(idx), reinterpret_cast<LPARAM>(&rcDrop));

		rcItem.right = rcDrop.left;
	}

	static const int roundness = dmlib::isAtLeastWindows11() ? dmlib_paint::kWin11CornerRoundness + 1 : 0;

	// Paint part

	if (isHot) // hot must have higher priority to overwrite checked state
	{
		if (!isIcon)
		{
			::FillRect(lptbcd->nmcd.hdc, &rcItem, dmlib::getHotBackgroundBrush());
		}
		else
		{
			dmlib_paint::paintRoundRect(lptbcd->nmcd.hdc, rcItem, dmlib::getHotEdgePen(), dmlib::getHotBackgroundBrush(), roundness, roundness);
			if (isDropDown)
			{
				dmlib_paint::paintRoundRect(lptbcd->nmcd.hdc, rcDrop, dmlib::getHotEdgePen(), dmlib::getHotBackgroundBrush(), roundness, roundness);
			}
		}

		lptbcd->nmcd.uItemState &= ~static_cast<UINT>(CDIS_CHECKED | CDIS_HOT); // clears states to use custom highlight
	}
	else if (isChecked)
	{
		if (!isIcon)
		{
			::FillRect(lptbcd->nmcd.hdc, &rcItem, dmlib::getCtrlBackgroundBrush());
		}
		else
		{
			dmlib_paint::paintRoundRect(lptbcd->nmcd.hdc, rcItem, dmlib::getEdgePen(), dmlib::getCtrlBackgroundBrush(), roundness, roundness);
			if (isDropDown)
			{
				dmlib_paint::paintRoundRect(lptbcd->nmcd.hdc, rcDrop, dmlib::getEdgePen(), dmlib::getCtrlBackgroundBrush(), roundness, roundness);
			}
		}

		lptbcd->nmcd.uItemState &= ~static_cast<UINT>(CDIS_CHECKED); // clears state to use custom highlight
	}

	LRESULT retVal = TBCDRF_USECDCOLORS;
	if ((lptbcd->nmcd.uItemState & CDIS_SELECTED) == CDIS_SELECTED)
	{
		retVal |= TBCDRF_NOBACKGROUND;
	}

	if (isDropDown)
	{
		retVal |= CDRF_NOTIFYPOSTPAINT;
	}

	return retVal;
}

Nine · Thursday at 01:27 PM

Here one way :

; From Nine
#include <GUIConstants.au3>
#include <WinAPI.au3>
#include <StructureConstants.au3>
#include <GuiToolbar.au3>
#include <WinAPITheme.au3>

Opt("MustDeclareVars", True)

Global Const $tagNMTBCUSTOMDRAW = $tagNMHDR & ";dword dwDrawStage;handle hdc;" & $tagRECT & ";dword_ptr dwItemSpec;uint uItemState;lparam lItemlParam;" & _
    "ptr hbrMonoDither;ptr hbrLines;ptr hpenLines;dword clrText;dword clrMark;dword clrTextHighlight;dword clrBtnFace;dword clrBtnHighlight;dword clrHighlightHotTrack;" & _
    "long TextLeft;long TextTop;long TextRight;long TextBottom;int nStringBkMode;int nHLStringBkMode;int iListGap;"
Global Const $TBCDRF_NOBACKGROUND = 0x400000
Global Const $TBCDRF_HILITEHOTTRACK = 0x20000

ConsoleWrite($tagNMTBCUSTOMDRAW & @CRLF)

Example()

Func Example()
  Local Enum $e_idNew = 1000, $e_idOpen, $e_idSave, $e_idHelp
  Local $hGUI = GUICreate("Toolbar", 600, 400)
  Local $hToolbar = _GUICtrlToolbar_Create($hGUI)

  GUIRegisterMsg($WM_NOTIFY, WM_NOTIFY)
  GUISetState()

  _GUICtrlToolbar_AddBitmap($hToolbar, 1, -1, $IDB_STD_LARGE_COLOR)
  _GUICtrlToolbar_AddButton($hToolbar, $e_idNew, $STD_FILENEW)
  _GUICtrlToolbar_AddButton($hToolbar, $e_idOpen, $STD_FILEOPEN)
  _GUICtrlToolbar_AddButton($hToolbar, $e_idSave, $STD_FILESAVE)
  _GUICtrlToolbar_AddButtonSep($hToolbar)
  _GUICtrlToolbar_AddButton($hToolbar, $e_idHelp, $STD_HELP)

  Do
  Until GUIGetMsg() = $GUI_EVENT_CLOSE

  _GUICtrlToolbar_Destroy($hToolbar)
EndFunc   ;==>Example

Func WM_NOTIFY($hWnd, $iMsg, $wParam, $lParam)
  Local $tTool = DllStructCreate($tagNMTBCUSTOMDRAW, $lParam)
  If _WinAPI_GetClassName($tTool.hWndFrom) <> "ToolbarWindow32" Or $tTool.Code <> $NM_CUSTOMDRAW Then Return $GUI_RUNDEFMSG
  If $tTool.dwDrawStage = $CDDS_PREPAINT Then
    Local $hBrush = _WinAPI_CreateSolidBrush(0x606060)
    Local $tRect = DllStructCreate($tagRECT, DllStructGetPtr($tTool, "left"))
    _WinAPI_FillRect($tTool.hdc, $tRect, $hBrush)
    _WinAPI_DeleteObject($hBrush)
    Return $CDRF_NOTIFYITEMDRAW
  ElseIf $tTool.dwDrawStage = $CDDS_ITEMPREPAINT And BitAND($tTool.uItemState, $CDIS_HOT) Then
    $tTool.clrHighlightHotTrack = BitAND($tTool.uItemState, $CDIS_SELECTED) ? 0xA0A0A0 : 0x808080
    Return $TBCDRF_HILITEHOTTRACK
  EndIf

  Return $GUI_RUNDEFMSG
EndFunc   ;==>WM_NOTIFY

Edited Thursday at 03:08 PM by Nine
better code

WildByDesign · Thursday at 03:30 PM

1 hour ago, Nine said:

Here one way :

This is fantastic, thank you!

Looking at your $tagNMTBCUSTOMDRAW, I see now that the brushes are Pointers. I definitely never would have got that, so I appreciate it.

Now I can also see why the win32-darkmodelib project also has code for painting over the line just above the toolbar. Similar to the white line that has to be painted over with a dark mode menubar. I assume that this white line is likely non-client area as well.

This gives me something to work with and try some things and learn from. And if something good comes out of it, maybe it can eventually be added to the GUIDarkTheme UDF.

WildByDesign · Thursday at 04:46 PM

Adding the following below GUISetState fixes the issue with the white line above the toolbar:

_GUICtrlToolbar_SetColorScheme($hToolbar, 0x606060, 0x606060)

Block all input without UAC	Save/Retrieve Images to/from Text	Monitor Management (VCP commands)
Tool to search in text (au3) files	Date Range Picker	Virtual Desktop Manager
Sudoku Game 2020	Overlapped Named Pipe IPC	HotString 2.0 - Hot keys with string
x64 Bitwise Operations	Multi-keyboards HotKeySet	Recursive Array Display
Fast and simple WCD IPC	Multiple Folders Selector	Printer Manager
GIF Animation (cached) Debug Messages Monitor UDF	Screen Scraping Round Corner GUI UDF	Multi-Threading Made Easy Interface Object based on Tag

Sign In

Subclassing Toolbar (need help with missing NMTBCUSTOMDRAW structure)

Recommended Posts

WildByDesign

WildByDesign

Nine

WildByDesign

WildByDesign

Create an account or sign in to comment

Create an account

Sign in

Browse

AutoIt Resources

Release

Beta