Where I work we use Rational Functional Tester but we keep running into a lot of serious problems with it. And IBM support has been horrible to boot.

I’ve been interested in using FunFX with Cucumber. I like the idea of abstracting the testing implementation details from the description of what should be tested.

For libraries I’ve been using FlexUnit 4 which has been great. My only complaint is that the documentation seems to be non-existent. I typically have to read through the code base to figure out how to get something to work and the code is almost completely uncommented.

I’ve read through the documentation of FlexMonkey but I haven’t tried it yet. I’m not a big fan of recording and replaying interactions because it seems brittle. That’s the reason why I’m interested in trying FunFX/Cucumber.

Flex testing frameworks have come along way in the last year or two which is great. I think the biggest issue is getting people to really start using the tools available. There’s a stigma about Flex that you simply don’t test it because that’s the way it’s always been. That needs to change.

hmmm, is the silence telling us something …

We’ve been testing manually too but just downloaded flex monkey.

So far all swfs we load show up using target swf window but our does not; just a gray screen …

We’ll try the forums and look over the docs again.

Looks promising though.

Thank you for the comment. The silence is from scheduling a blog and then take the wife out of town (no internet). Let me know if you are still having problems.

Hi Jon,

Testing can become complex and expensive, so maybe the silence just shows that Gorilla Logic is addressing a complex issue here!

I’m working at the moment for a test team of quite a big enterprise AIR application and because we use FlexMonkey I thought my lessons learned could be interesting for the FlexMonkey community. The application is built with Flex(AIR)/JEE/GraniteDS and my customer wants to have about 150 automated end-to-end functional tests for the first RC. Next to this, there are also about 400 manual tests for this first delivery. In the end we will speak about more than 1200 tests where about a third automated. The goal is that the CI runs the automation test at least once a week.

The customer made the decision for FlexMonkey and our job as Flex Consultants was to help making the test environment that they already built more mature. So we helped them to use FlexMonkey with AIR using the MonkeyLink and the Flex mixin hook to connect the client to FlexMonkey and started to automate test cases. All in all this worked very smooth! I can definitively say that I like FlexMonkey and I would like to thank Gorilla Logic for the contribution to the Flex community! Of course, as always there are short comings. With this project we where facing following troubles:

They started using FlexMonkey for recording and playback but missed reusability features in the FlexMonkey client. So they decided early on to use FlexMonkey only for recording and to build their own playback environment. To work around the short comings, they started to divide and externalise the xml files FlexMonkey writes (the test steps which then translate into the monkey commands) and they started to write a mechanism to load and parse these files from within the Fluint test methods while generating the FlexMonkey commands at runtime. So they moved away from the unit test actionscript classes to a more generic solution, where the test crew writes the test in mxml modules and the tests are loaded at runtime, running reusable steps from the xml files.

The approach works, but we loose a lot of the FlexMonkey/Fluint out-of-box functionality! The test crew also has to do much copy/paste with files and code they don’t understand, which leads to errors. I have to admit that the gained flexibility could lead to an even more integrated solution, but to me it feels quite over-engineered.

But also the Flex automation framework provides its own troubles and it’s not always easy to find a good way or place to add automation identifiers. FlexMonkey helps with this, but our app is build with a Transaction Script approach (without much code reuse) and the test crew has a lot of troubles to find identifiers that work or has to use different approaches for almost identical situations.

Using the label’s is not a bad idea but our application uses multiple languages which means that a recording in French can not be used for a playback in Dutch or English. Because many view components are created at runtime, we have to use the parent identifiers in many steps we want to automate, but finding the right strategy to add dynamic automation names is hard and never really future prove.

Maybe the thing we missed most in FlexMonkey was an easier way to reuse recorded test steps directly from within the client. Think for example about a login step you have to do over and over again before you can start your actual test. Personally, I think that generating the unit tests from FlexMonkey is the better approach than building a whole playback environment from scratch. Or at least, using instead the generated unit tests as the driver of such a solution. Also more freedom in structuring the FlexMonkey project would make a great difference, being able to have several reusable xml files and the ability to generate the unit tests based on this more complex project structure would help much to avoid java developers from starting to over-engineer to early.

There are a lot of things you cannot do from within AIR of course, so we use Ant (and the FlexMonkey Ant Tasks) to compile our mxml modules and run the tests, but setting up the test data (dbunit integration) or starting and stopping the AIR client where problems that had to be solved. Also writing the reports to test management tools could be interesting add-ons. Also here I can say that any form of out-of-the-box help would be much appreciated!

As for the automation names, I hope that Adobe will further invest in the automation framework with Flex 4 and that FlexMonkey will pick these things up. Maybe this is even the most troublesome part for larger applications. How do you deal with dynamically generated view components etc?

I have no experience with Mercury, Selenium and all the rest, but I use Fluint/FlexUnit 4 and a TDD approach as often as I can. It would be interesting to here if the other tools already do a better job, but somehow I have my doubts. To me, FlexMonkey is definitively a great win for the community, so I would like to close with the questions if Gorilla Logic plans to further develop FlexMonkey and if yes, if there is already a road map available? And of course, how can the Flex community help to make FlexMonkey better?

Cheers & Success,
Roland

Hi guys,

In my Company, Neotys, we have developed a load testing tool that supports Flex and AIR apps. We are Adobe partners.

So NeoLoad can create scenarios to test your FLEX and applications’ behavior under stress and validate their performances, while pinpointing any weaknesses. It brings you support of: AIR applications on user desktop, FLEX Messaging Service, allowing data push by polling or streaming requests, externalized object transport, Flash Remoting objects content in FLEX messages, AMFX, the AMF serialized XML format.

You can download and try it from our website (www.neotys.com). We’ll be very happy to get your feedback.

Best rgds,

Paul