Wednesday, September 19, 2007

How does your portal QA compare ?

QA has always been an advantage of JBoss Portal over other open-source portals: we've had strong coverage of databases and LDAP servers with the JBoss Portal 2.x series in addition to specification implementation and unit tests. However, we're not stopping there. We're always looking for ways to improve JBoss Portal's quality, and one way to accomplish this is better testsuite coverage. Here's an overview of our QA work:


  • Testsuite coverage

    We have continuous run of portal 2.6 testsuite and trunk testsuite against JBoss 4.2.0.GA. Results can be seen at 2.6-testuite and trunk-testsuite. We plan to add JBoss AS 4.2.1.GA to the matrix as well.

  • Database Compatibility

    We've been continuously running our testsuite against following the most popular database products for over a year now:


    1. Oracle 9i
    2. Oracle 10g
    3. MySql4
    4. MySql5
    5. MS SQL Server
    6. PostgreSQL 8
    7. Hypsersonic (HSQLDB)

    The result of these test runs can be seen at DB-tests.


  • LDAP compatibility

    LDAP support has been added in the 2.6 release and we've also had continuous runs of identity module tests with the following LDAP servers:

    1. OpenLDAP
    2. RedHat DS (of course)
    3. Sun DS
    4. Open DS

    The test results can be seen at ldap-tests.


  • Modules detachment

    I won't speak here about the detachment of modules since Julien covered it nicely in another post already. It is the last item that has been added to the mix, just a couple of days ago. The following modules have been detached from the trunk source tree so far and they are used in portal as thirdparty libraries:


    Basically it is about modularizing the code base and make it container agnostic. As a result, we now have continuous runs of these modules test suites against JBoss AS 4.2.0.GA, JBoss AS 4.2.1.GA and Tomcat 6. We plan to add JBoss 5 and Tomcat 5.5 very soon to the matrix.

  • Interoperability with other portal servers

    This effort is still in its early stages. We have started doing some deeper testing of WSRP interoperability with BEA portal server. We plan to try to make it part of a continuous run and include more vendors as well. This might prove tricky and the lack of a TCK for WSRP makes interoperability testing rather dauting.

A portal is a key integration product for enterprise applications and it should be tested with the different products it integrates with. If you need a portal and you worry about how it will behave in your environment, a continuously run, well covering test suite is a major factor for your peace of mind. QA is a necessary evil and, as you can see, we are serious about it. How does your current portal compare (if you're not already using JBoss Portal, that is ^_^)?

9 comments:

Michael Neale said...

Don't forget Derby database - also known as JavaDB !

Anton said...

Do some of these tests run from the wbe browser, or are they more based on the java foundations?

If they run from the browser ( on portlets ) what testing toolkit do you use?

Julien Viet said...

For the portlet container, the tests do not rely on web browser although the tests use the HTTP protocol in order to provide the more realistic test conditions.

We had to develop our own test framework for that purpose. At runtime we dynamically deploy war files in sequence, each war contains a fixed number of unit tests under the form of portlets. The testing occurs with a sequence of portlets contained in the war file and the portlet cooperates in order to get the test done.

The client side of the test is a runner which is based on commons HTTP client and reacts to the cooperation of the server side portlets of the test.

Drop me a line at julien at jboss dot com, if you want to know more about it.

Prabhat Jha said...

Adding to Julien's comment, we are planning to do browser compatibility testing using Selenium. Selenium seemed to have met most (not all) of our need during proof of concept.

Raj said...

None of the links works for me :( say test-suits, DB Tests etc

Is it available for public?

Thanks,
Rajesh

Prabhat Jha said...

Our CI system was down for maintenance today. You just happen to catch us on wrong time. Talk about bad coincidence. ;-) It should be up now.

Raj said...

Thanks Prabhat, it works now ..btw what are the tools which you use in testing . I hope you are using selenium for functional testing? Is the test suites open sourced?

Thanks,
Raj

Prabhat Jha said...

Hi Raj,

As I mentioned in my earlier comment, I am planning to use Selenium for functional testing. I had done a POC and it seemed like Selenium would serve the purpose.

Yes, all of our tests are open source. Please see our anonsvn repo at http://anonsvn.jboss.org/repos/portal/

Raj said...

Thank you prabhat!

- Raj