Jump to content

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Find out more here. X
X


Photo

How to automate Java windows/controls?


  • Please log in to reply
25 replies to this topic

#1 Jon

Jon

    Up all night to get lucky

  • Administrators
  • 10,614 posts

Posted 15 March 2007 - 07:25 AM

So I had a look at a couple of Java programs. Notably the Java Control Panel that most people will have. And it's like a total black box. Using Window Spy++ I could see the main window but even though the window was full of buttons and checkboxes and stuff - there were no child windows!

I thought automating Java controls would be a case of finding out which messages it accepts to do what - but if it doesn't even present a child window to send to I'm a bit stumped. Spy++ couldn't even see any unknown/registered messages when you click on things.

Is there a Java SDK type thing for programming these sorts of apps - that might shown how they are control internally and help out. I mean, there must be an internal command for selecting a checkbox.

Ideas welcome.







#2 Manadar

Manadar

         

  • MVPs
  • 10,842 posts

Posted 16 March 2007 - 01:06 PM

Here's what the professionals say about Java Applets.

AutoMate is able to find the Java Applet window but recognizes nothing inside it. The Java controls are basically "invisible" to AutoMate. From Windows' point of view the Java Applet looks like a window containing one large bitmap. Because of this, most of AutoMate's interactivity actions (i.e. 'Move Mouse to Object,' 'Press,' or 'Set Text' actions) will not work within a Java Applet window.

Solutions
The only possibility for AutoMate to interact with controls within a Java Applet Window is to set the focus to the Java Window and use the 'Send Keystrokes' action to tab through the fields to set and get data or press buttons.


http://www.networkautomation.com/automate/...23252F9A5553FE0

Edit: What Larry said completely contradicts this information.

Edited by Manadar, 16 March 2007 - 01:07 PM.


#3 Jon

Jon

    Up all night to get lucky

  • Administrators
  • 10,614 posts

Posted 16 March 2007 - 01:13 PM

Not quite. Junit looks to be a general java testing thing. But you need to use some addins to test gui stuff through a awk.robot java class. This class can mess around with a few buttons but appears to have no c++/external interface - so it's of no use :whistle:

#4 Manadar

Manadar

         

  • MVPs
  • 10,842 posts

Posted 16 March 2007 - 01:28 PM

I've seen a few programs that decompile the .class files, that could at least give you information about the Java applet. This probably should not be considered an option...

My guess is that if you should succeed, Jon, you would be the first ever.

#5 big_daddy

big_daddy

  • Moderators
  • 2,591 posts

Posted 16 March 2007 - 02:22 PM

By the sounds of it this may be worth taking a look at.

IBM Rational Functional Tester

Edited by big_daddy, 16 March 2007 - 02:23 PM.


#6 TimothyGirard

TimothyGirard

    Seeker

  • New Members
  • 1 posts

Posted 04 May 2007 - 07:42 PM

What if we were able to capture user defined images and then OCR any text. Most text within these UI's (Flash and Java) is usually in standard fonts.

this would be a very powerfull ability for AutoIt. The ability to define a "virtual" object that could be manipulated via standard functions would no doubt push a large group of app automators (Like me) to AutoIt a lot faster. Until I can do that easily in AutoIT, I'm stuck in QuickTest Pro.

#7 DaleHohm

DaleHohm

    Think of IE as an API...

  • MVPs
  • 5,981 posts

Posted 04 May 2007 - 11:15 PM

I have always been on the lookout for such tools and methods, but had never found anything other than trying to exploit methods that might be exposed by the applet to Javascript.

I did recently find a couple of resources dedicated to discussions of this but I have not had a chance to dig into them to see if there is anything promising.

This Wiki has a bunch of tool pointers: TestingGUIApplications and this Yahoo! group: java-gui-testing · Java GUI Testing

Dale

Edit: also seeing the source may help you... Google the DJ Java Decompiler

Edited by DaleHohm, 04 May 2007 - 11:16 PM.

Free Internet Tools: DebugBar, AutoIt IE Builder, HTTP UDF, MODIV2, IE Developer Toolbar, IEDocMon, Fiddler, HTML Validator, WGet, curl

MSDN docs: InternetExplorer Object, Document Object, Overviews and Tutorials, DHTML Objects, DHTML Events, WinHttpRequest, XmlHttpRequest, Cross-Frame Scripting, Office object model

 

Automate input type=file (Related)

Alternative to _IECreateEmbedded? better: _IECreatePseudoEmbedded  Better Better?

IE.au3 issues with Vista - Workarounds

 

SciTe Debug mode - it's magic: #AutoIt3Wrapper_run_debug_mode=Y
Doesn't work needs to be ripped out of the troubleshooting lexicon. It means that what you tried did not produce the results you expected. It begs the questions 1) what did you try?, 2) what did you expect? and 3) what happened instead?

Reproducer: a small (the smallest?) piece of stand-alone code that demonstrates your trouble


#8 b990595

b990595

    Seeker

  • New Members
  • 3 posts

Posted 16 May 2007 - 09:52 AM

I would be nice to read java windows, maybe this can help ...

Sun have developed a Java Access Bridge, which contains DLLs to read Java windows ..
http://java.sun.com/products/accessbridge/

API documentation is here ..
http://java.sun.com/products/accessbridge/docs/api.html

JavaFerret is a simple app, that reads java screens ...
http://java.sun.com/products/accessbridge/...javaferret.html

-Rasmus, Denmark

#9 Xenobiologist

Xenobiologist

    Xx Code~Mega xX

  • MVPs
  • 4,849 posts

Posted 07 September 2007 - 06:49 AM

Hi,

any progress on this?

We use this : IBM Rational Functional Tester

But it is written in Java.

So long,

Mega

Scripts & functions Organize Includes Let Scite organize the include files *new

Yahtzee The game "Yahtzee" (Kniffel, DiceLion)

LoginWrapper Secure scripts by adding a query (authentication)

_RunOnlyOnThis UDF Make sure that a script can only be executed on ... (Windows / HD / ...)

Internet-Café Server/Client Application Open CD, Start Browser, Lock remote client, etc.

MultipleFuncsWithOneHotkey Start different funcs by hitting one hotkey different times


#10 MadMike

MadMike

    Seeker

  • Active Members
  • 7 posts

Posted 10 September 2007 - 02:51 PM

Sorry I'm not an experienced developer just yet but I'm very interested if anyone has made any progress on this as well.

I've lookep up a load of how to automate Java windows/controls and this Java Access Bridge seems the best way.

I've been trying to use the Java Access Bridge and the Java Ferret and I'm not having much luck.
Just have to use DllCall to the WindowsAccessBridge.dll with the Java Access Bridge then I can start making calls to the classes within the program.
If it's too easy, then its not worth doingMike P. O'Keeffe BEng, Hardware Design EngineerLanguages: Perl, C, Expect, Visual Basic, Assembly

#11 ValeryVal

ValeryVal

    Dig the way

  • Active Members
  • PipPipPipPipPipPip
  • 453 posts

Posted 11 September 2007 - 05:17 AM

I added ALisp
here

Apart from other things you can create by newLisp's tools java-based GUI.
For doing it newLisp has special guiserver.jar (GUI server) and guiserver.lsp.

Creation of java-based gui is the following process:

1. Start GuiServer demon

java.exe -jar "c:/Program Files/newlisp/guiserver.jar" 47011

You will see the demon's window worked (black window)

2. Start ALisp.au3

3. Evaluate by ALisp the expression:

(load "c:/Program Files/newlisp/guiserver.lsp")

4. Now you can evaluate lines (step by step or all as one) of some example from newlisp/guiserver folder.
ie button-demo.lsp

(gs:init)
(gs:set-trace true)
(gs:frame 'ButtonDemo 100 100 400 300 "Click on button or color panel")
(gs:set-resizable 'ButtonDemo nil)
(gs:panel 'ColorPanel 360 200)
(gs:set-background 'ColorPanel '(0 1 0) 0.2)
(gs:button 'aButton 'button-action "color")
(gs:set-flow-layout 'ButtonDemo "center" 2 15)
(gs:add-to 'ButtonDemo 'ColorPanel 'aButton)
(gs:set-visible 'ButtonDemo true)
(gs:mouse-event 'ColorPanel 'mouse-action)

;;;; define actions
(define (button-action id)
(gs:set-color 'ColorPanel (random) (random) (random)))

(define (mouse-action id type x y button cnt mods)
(gs:set-text 'ButtonDemo (format "%8s %3d:%3d %d %d %2d" type x y button cnt mods))
)

;(gs:listen)

Thus, you will create really java-based GUI having two handlers - button-action and mouse-action.
If you uncomment last line then GUI will independent window which handles button and mouse events.
This is a two-side body - ALisp & java window.
Maybe it is simplest platform for experiment with interaction between AutoIt and java, I think.

Remark. You should to install newlisp from this site, I think

BTW. ALisp (with help newlisp) shows how you can build simplest java-based bridge from Windows into Linux/Mac OS X/Solaris/BSD's platform
The point of world view

#12 ValeryVal

ValeryVal

    Dig the way

  • Active Members
  • PipPipPipPipPipPip
  • 453 posts

Posted 11 September 2007 - 05:29 AM

Feed back from java-based Gui can be implemented by AutoItX.
See, example of AutoItX using (not tested)

here

However there are other more fast method, I think

;-)
The point of world view

#13 Antiec

Antiec

    Wayfarer

  • Active Members
  • Pip
  • 52 posts

Posted 14 September 2007 - 12:43 PM

Hi!

I had to automate a Java program at my work. I found two kinda nice java-based automation programs (marathon & abbot), but marathon wouldn't work with our huge program (it did work with some smaller applications) and I couldn't really do what I wanted with abbot. So I found the already mentioned Java Access Bridge. I didn't/don't know how to use it with autoit, but I managed to automate our program with JAB & C++, mostly by customizing Java Ferret though. I know a bit about how JAB works and I might be able to answer some questions if you have any. And maybe I'll try to use it with autoit too (I'd really love it).

Though... Marathon seems to be pretty nice program for the job. And I couldn't find a way to use button IDs (like name etc) straight, but I had to run the parent/child tree's (there's java monkey for finding the right routes). And somehow my program stalls when there's a new dialog window, but that might not happen with a new code.

Yours,
Lassi

Edit: And actually I didn't use the DLLs, but the source code. Anyways, atleast I think I have somekind of image how the calls should work... gotta test it.

Edited by Antiec, 14 September 2007 - 02:48 PM.


#14 Rocc

Rocc

    Seeker

  • New Members
  • 1 posts

Posted 30 January 2008 - 12:31 AM

Perhaps this web site might help.
http://jacareto.sourceforge.net/

#15 junkew

junkew

    Universalist

  • Active Members
  • PipPipPipPipPipPip
  • 659 posts

Posted 11 March 2008 - 08:17 PM

And maybe when somebody comes around to read
http://java.sun.com/docs/books/jni/
it can be programmed in C (and with samples it looks like it could be done in AutoIT) but definitely is some effort.

#16 cbadal

cbadal

    Seeker

  • Active Members
  • 10 posts

Posted 13 March 2008 - 04:41 PM

For automating Java you would really need something that understands Java.

Java swing controls are all drawn in Java drawing functions (these functions eventually call native platform dependent drawing functions).
This makes it nice for Java because they only need to write the drawing code once (and only update the low level platform dependent drawing functions in the actual JVM code).
This also means that their controls don't adhere to the Windows API (except for perhaps a top level Window, or some container).

The very old Java AWT toolkit uses native controls and if you came across a Java GUI that was written in AWT, you may be able to automate it with AutoIt.

#17 daluu

daluu

    Wayfarer

  • Active Members
  • Pip
  • 81 posts

Posted 16 March 2009 - 07:32 PM

What about:

The free FEST framework/tool for Java? Though it's more developer & unit test centric. Works for Java applets too. Not sure how one can use it with AutoIt though.

http://code.google.com/p/fest/
http://fest.easytesting.org/swing/wiki/pmw...ESTAppletViewer

#18 Xenobiologist

Xenobiologist

    Xx Code~Mega xX

  • MVPs
  • 4,849 posts

Posted 16 March 2009 - 09:17 PM

Hi,

another tool which works quiet well is Loadrunner from HP.

Mega

Scripts & functions Organize Includes Let Scite organize the include files *new

Yahtzee The game "Yahtzee" (Kniffel, DiceLion)

LoginWrapper Secure scripts by adding a query (authentication)

_RunOnlyOnThis UDF Make sure that a script can only be executed on ... (Windows / HD / ...)

Internet-Café Server/Client Application Open CD, Start Browser, Lock remote client, etc.

MultipleFuncsWithOneHotkey Start different funcs by hitting one hotkey different times


#19 bo8ster

bo8ster

    Universalist

  • Active Members
  • PipPipPipPipPipPip
  • 1,189 posts

Posted 16 March 2009 - 10:26 PM

Thanks guys for all the links, really helpful!
Post your code because code says more then your words can. SciTe Debug mode - it's magic: #AutoIt3Wrapper_run_debug_mode=Y. Use Opt("MustDeclareVars", 1)Brett F's Learning To Script with AutoIt V3Valuater's AutoIt 1-2-3, Class... is now in SessionContribution: Get SVN Rev Number, Control Handle under mouse, A Presentation using AutoIt, Log ConsoleWrite output in Scite

#20 vixLT6B

vixLT6B

    Newbie

  • Banned (NOT IN USE)
  • 4 posts

Posted 19 March 2009 - 10:47 PM

I am not sure, but look at this: [link removed]

Edited by SmOke_N, 19 March 2009 - 11:36 PM.





0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users