WinRT - UI Gallery (Xaml Islands)

[MattyD]

In order not to pollute other threads, here's one dedicated to Windows.UI controls and examples.

These are for use with Xaml Islands, NOT WinUI3 - although porting them across shouldn't be too demanding. Both methods have their pros and cons - but I'd say the XAML Island approach is bit more robust at this point in time.

Don't forget to update your manifests to make this work! (background/details in this thread)

Download

PS. More than happy for this to be a community thing if anyone else wishes to contribute

[MattyD]

Hi All,

I've been working some more on this over the last day or two, and it seems there's issues with XAML controls that require keyboard input. (i.e. textboxes weren't behaving!)... I might not have this 100% right, but my understanding is:

A message pump basically looks something like this:

while (true)
{
  while (PeekMessage( &msg, NULL, 0U, 0U, PM_REMOVE ))
  {
      TranslateMessage(&msg);
      DispatchMessage(&msg);
  }
 //do other stuff
}

• PeekMessage/GetMessage retrieves a message (GetMessage waits for a message, peek will return immediately)
• TranslateMessage does some pre-processing on kb input.
• Then DispatchMessage sends messages on to the appropriate window proc.

For xaml islands to properly handle the kb input, we're supposed to pass messages to the XAML host container using:
IDesktopWindowXamlSourceNative2_PreTranslateMessage($pDesktopWinXamlSrc).  And as the function name implies, this should be done before TranslateMessage. If PreTranslateMessage() fails, then the message is not meant for a XAML control and we should do a translate/dispatch as per normal.

Now the API calls for messaging are pretty straight forward - but the problem is that the message pump baked into Autoit. I think, internally the "do other stuff" part of the loop probably includes the execution of  a portion of our source script... Regardless of my take on AutoIt's internals, it seems impossible to trap every message before it has been dispatched.

I don't know how feasible it is to call PreTranslateMessage from custom window procs after messages have been dispatched. This would be a horrible approach, but I guess beggars can't be choosers...

[WildByDesign]

Thank you for continuing this journey, Matty.

I still have a lot of interest in this. I feel like this can potentially be great for AutoIt. But at the same time, I also understand that this is a tremendous amount of work for one person to do. I feel bad that I personally don't have the skills or understanding yet to be of much help.

I've been reading a bunch of MS documentation and that has probably confused my understanding more. I was wondering if you can help clear it up a bit for me.

Are the controls created by either Xaml Islands or WinUI 3 essentially the same (at least visually on the surface)?

And is the main difference essentially the mechanism on which to get there (there, being what is seen on the surface)?

[MattyD]

1 hour ago, WildByDesign said:

Are the controls created by either Xaml Islands or WinUI 3 essentially the same (at least visually on the surface)?

yeah, visually very similar - but not identical. 

1 hour ago, WildByDesign said:

And is the main difference essentially the mechanism on which to get there (there, being what is seen on the surface)?

Yes and no - XAML Islands are the older tech, and all the current MS development seems to be around WinUI3. But so far I'm having more success with XAML Islands. 

The hosting of the XAML source object is obviously quite different with the windowing class etc. in WinUI3.   But yes, on the control front WinUI3 is essentially an update on the same controls. The layout concepts with grids and stackpanels etc. all work the same way...   You will find some things moved around though - MS has obviously rationalised many of the classes' interfaces. for e.g. the functions on Windows.UI.Xaml.Controls.IButtonWithFlyout moves to Micorsoft.UI.Xaml.Controls.IButton in WinUI3 etc.

For WinUI3, The first thing I need to get to grips with (I think) is around setting up "resources".  Basically a bunch of nested dictionaries etc that contain stuff that a control can draw on.  At least that's probably where I need to start - I'm hoping it will fix some of the weirdness I'm seeing over there. Resources are a thing in the old paradigm too, but these seem to be pre-populated for the most part.  For example, ATM all your controls turn invisible in WinUI3 when in dark mode because none of the dark mode pallets are defined... (And there's some other issues that I need to work through as well)

2 hours ago, WildByDesign said:

Thank you for continuing this journey, Matty.

pleasure mate. thanks very much for the support  Nothing seems to be straight forward with this stuff, so motivation comes and goes! 

[MattyD]

Ok, so it seems the pretranslate function is used to handle input from the arrow/tab keys and some other things.. but there is more to this puzzle.

The window that we're calling $hIsland is a: Windows.UI.Composition.DesktopWindowContentBridge.  This window has a child, and the WM messages for BK/Mouse input seem to end up there. The child window is of the class: Windows.UI.Input.InputSite.WindowClass.

From some testing, I think we want to be sending WM_CHAR messages to this window.  - It kindof works anyway, I have to mouse over the island for the new characters to appear, but we're moving in the right direction!

WM_CHAR messages should be produced automatically by TranslateMessage.  As it parses WM_KEY* messages, we should by rights be getting new WM_CHAR messages in our queue. If I hijack the Input window's WndProc, I can see there's WM_KEYDOWN/UP messages coming in - but no WM_CHAR messages.  I guess a TranslateMessage was never performed here? Not sure why this would be the case, but its my best guess ATM..

As a workaraoud, you'd think we could subclass that Input window, fudge our own WM_CHAR translation, then forward messages to the original handler. But this plan falls down at the last hurdle.  Manually calling the original window proc throws a fatal Access Denied error .   So not sure where to go from here....

[WildByDesign]

23 hours ago, MattyD said:

Yes and no - XAML Islands are the older tech, and all the current MS development seems to be around WinUI3. But so far I'm having more success with XAML Islands. 

There is definitely some pros and cons to both. As I understand it, XAML Islands can be beneficial for that fact that they can more easily be added to our existing GUI's.

23 hours ago, MattyD said:

For WinUI3, The first thing I need to get to grips with (I think) is around setting up "resources". 

One thing that I always found interesting about your WinUI3 examples is that they use exactly zero CPU while running. I've always become accustomed to seeing AutoIt GUI's using some small amount of CPU for their GUIGetMsg loop or While loops for OnEvent mode. So that was pretty neat to see.

I see benefits to the XAML Islands approach and the WinUI3 approach as well.

Hypothetically speaking... let's assume for a moment that you got some sort of superhuman powers, unlimited time, etc. Let's assume also that you had a full understanding of how both XAML Islands and WinUI3 work entirely.

Given those hypothetical circumstances:

• Would you create a UDF for XAML Islands or WinUI3?
  • Or a UDF that somehow combines both?

I've actually been thinking about this for a few months now.

For example, if the community here was to come together on this and help. Obviously it's a significant amount of work for just one person. But what I've been pondering is which approach (XAML Islands or WinUI3) deserves the most attention. I think about which would be most beneficial for AutoIt in, for example, 2 years from now.

I also think about how much support and development Microsoft will put into both of those as time goes on. With XAML Islands being a bit older, I wonder if they will continue supporting and improving them. We all know how they like to jump from one thing to the next.

Anyway, sorry but that was mostly putting my thoughts in writing. To be perfectly clear, I personally am tossed (50/50) between the two.

Edited Monday at 02:28 PM by WildByDesign

[MattyD]

12 hours ago, WildByDesign said:

also think about how much support and development Microsoft will put into both of those as time goes on. With XAML Islands being a bit older, I wonder if they will continue supporting and improving them. We all know how they like to jump from one thing to the next.

Yep I agree with this - WinUI3 is by far the better option from this standpoint. 

But at the end of the day, we're just trying to get something to work - So the older approach was looking pretty appealing up until a few days ago!

As a side, I think we might need to change our terminology around this too - after a bit of reading, it looks like XAML Islands for WinUI3 controls will become a thing at some point as well...

Edited yesterday at 03:06 AM by MattyD