Wednesday, July 1, 2009

Run your junit tests in parallel (with spring 3.0 and maven)

Swimming in a sea of patches...



Note; Most of the content in this post has been surpassed by a newer entry.

If you're willing to apply a few patches, you can now run your junit tests in parallel. There are two main use-cases for this; one is to make your regular unit tests run faster, the other is for io-bound tests, where you want to run several in parallel because you're waiting for external systems.

Just to whet your appetite, here are some numbers from a testcase (running 1000 instances of a simple unit-test-class):

Legend:
SC = SingleThreadedComputer (Computer, Junit47-snapshot class)
PC = ParallelComputer (ParallelComputer, Junit4.7-snapshot class)
CPC = ConfigurableParallelComputer (My project, see references)
C=Classes (one thread per class)
M=Methods (one thread per method)






































Core 2 duo 3.14GhzIntel i7 920
SC203451
PC(Classes, Method, Unlimited threads)406806
CPC(Methods,16 threads)218214
CPC(Classes, Unlimited threads)203116
CPC(Classes,Methods, Unlimited threads)266147

Running time in ms, 1000 simple tests

The table shows that for most dual-core solutions, running with parallel threads is currently never any faster than running non-threaded. Switching to the brand-new Intel i7 (4 hyperthreaded cores) has a marked effect, where simple unit test performance is doubled.

The lack of punch does, of course, change the moment your tests start doing IO (Integration tests, selenium tests and other web-tests).



The juicy stuff
svn co https://svn.openqa.org/svn/selenium-rc

Second build with mvn clean install: 102 seconds.
With 80 threads class-parallel on the i7: 37 seconds.

Class-parallel works easily on most tests.

A sea of patches


Getting all of this to work was not without its problems, though. To get this running, I had to patch just about half the libraries in use in my current stack. Furthermore, there's a huge difference between getting it to run and getting it to run without hiccups - every time. These patches insure that your infrastructure works as it should, but are your tests concurrent ?

Patches involved


See below for how to apply these patches in a simple manner.
Spring (3.0 trunk):
http://jira.springframework.org/browse/SPR-5863
AspectJ: Patch trunk with this patch (if using spring) (This patch is included in version 1.6.6)
https://bugs.eclipse.org/bugs/show_bug.cgi?id=281654
Junit4.7:
Several patches needed for thread safety:
http://github.com/KentBeck/junit/issues#issue/16
http://github.com/KentBeck/junit/issues#issue/18
Maven: You need a patched version of surefire that supports junit 4.7:
http://jira.codehaus.org/browse/SUREFIRE-555
You'll also probably want to be using the ConfigurableParallelComputer instead of the ParallelComputer that's included with JUnit.
git://github.com/krosenvold/configurable-parallel-computer.git

With these patches in place you're smooth sailing ;)

Applying the patches


Note: If you did this on the pre rc1 version of this patchset, I recommend that you start from a fresh copy of everything this time.
If you're using maven you could try this sequence:

git clone git://github.com/krosenvold/spring-framework-concurrent-junit-patch.git
git clone git://github.com/krosenvold/junit.git
git clone git://github.com/krosenvold/configurable-parallel-computer.git
git clone git://github.com/krosenvold/spring-test.git
git clone git://github.com/krosenvold/spring-concurrency-test.git
cd junit
ant
cd ../spring-framework-concurrent-junit-patch
./installJunit
cd ../spring-test
mvn install
cd ../configurable-parallel-computer
mvn install
cd ../spring-framework-concurrent-junit-patch
./checkoutSurefire
./buildSureFire
./applySureFirePatch
./buildSureFire




cd ../spring-concurrency-test
mvn install # Test project to verify that everything is ok


Your new versions


JUnit should be modified to 4.7-CONCURRENT
spring-test should have version number 3.0.0.RC1-CONCURRENT
Surefire should be modified to 2.5-SNAPSHOT
You must update your aspectj version to 1.6.6 in your project. Spring does a transitive
include on 1.6.5

You can see the version number for the spring artifacts when building spring. Probably something like 3.0.0.BUILD-20091005102426

Maven surefire settings



<plugin>
<artifactid>maven-surefire-plugin</artifactid>
<version>2.5-SNAPSHOT</version>
<configuration>
<parallel>classes</parallel>
<useUnlimitedThreads>true</useUnlimitedThreads>
<threadCount>80</threadCount>
<perCoreThreadCount>true</perCoreThreadCount>
</configuration>
</plugin>


UnlimitedThreads knocks out threadCount and perCoreThreadCount.

Look at the various "install" scripts to see the version numbers of the artifacts.
Simple, eh ?

And while we're at it: Half of the work with patching these projects has been about fighting with version control systems and custom made build scripts. So please; switch to GIT and maven: I can build any maven project without reading as much as a readme file. Maven allows me as a casual bystander to *easily* make contributions; I never need to read that 4 page "how to build" manual! Git is the only version control tool that respects my right/duty to patch, change and upgrade other projects. It allows me to do all these things without having to *ask* anyone. Repeat after me: Git and maven encourages free collaboration; a key idea of open source, right ?

http://twitter.com/krosenvold

50 comments:

  1. i have go through your article and bit eager to know what will need to use ANT instead of Maven. share some screenshots also if possible
    for selenium hands on practice Best selenium training institute in chennai

    ReplyDelete
  2. Very nice post here thanks for it .I always like and such a super contents of these post.Excellent and very cool idea and great content of different kinds of the valuable information's.

    machine learning training in chennai
    machine learning course in Chennai
    machine learning training center in chennai
    best training insitute for machine learning
    Android training in Chennai
    PMP training in chennai

    ReplyDelete
  3. Thank you so much for your information,its very useful and helful to me.Keep updating and sharing. Thank you.
    RPA training in chennai | UiPath training in chennai


    ReplyDelete

  4. After reading this web site I am very satisfied simply because this site is providing comprehensive knowledge for you to audience.
    Thank you to the perform as well as discuss anything incredibly important in my opinion. We loose time waiting for your next article writing in addition to I beg one to get back to pay a visit to our website in


    Selenium training in Chennai
    Selenium training in Bangalore
    Selenium training in Pune
    Selenium Online training

    ReplyDelete

  5. Get the most advanced RPA Course by RPA Professional expert. Just attend a FREE Demo session about how the RPA Tools get work.
    For further details call us @ 9884412301 | 9600112302

    RPA training in chennai | RPA training in chennai

    ReplyDelete
  6. The article is so informative. This is more helpful for our
    software testing training online
    best selenium online training in chennai. Thanks for sharing

    ReplyDelete

  7. Just seen your Article, it amazed me and surpised me with god thoughts that eveyone will benefit from it. It is really a very informative post for all those budding entreprenuers planning to take advantage of post for business expansions. You always share such a wonderful articlewhich helps us to gain knowledge .Thanks for sharing such a wonderful article, It will be deinitely helpful and fruitful article.
    Thanks
    DedicatedHosting4u.com

    ReplyDelete
  8. Great. It is good to constantly coming up with creative ideas. Provides much needed knowledge. goal oriented blog posts and always tried to find creative ways to meet goals.

    Thanks
    Online affiliates

    ReplyDelete
  9. Your post is very good. I got to learn a lot from your post. Thank you for sharing your article for us. it is amazing post
    what is seo
    types of seo

    ReplyDelete
  10. This is a fantastic idea! I like it a lot because it's super easy for the audience to see the value of opting in. wonderful and amazing post very use full your post thanks for sharing your article
    Android Application development
    Web application

    ReplyDelete
  11. If you want best smart phones under 10000-15000 then go on. https://www.arecious.com/

    ReplyDelete
  12. If you want best home theater under 5000-10000 then go on. https://www.arecious.com/

    ReplyDelete
  13. If you want best bluetooth speakers with affordable price rate then go on https://www.arecious.com/

    ReplyDelete
  14. If you are looking for sony bluetooth speakers then go on https://www.arecious.com/

    ReplyDelete
  15. If you are looking for Best Gyms in Ghaziabad then click on the given link.

    ReplyDelete
  16. While composing the last paper of your last B.Com test, a plenty of considerations experience your psyche. A great deal of these contemplations incorporate plans about resting for a whole week or celebrating for a whole week, contingent upon your individual inclinations. Be that as it may, trust us, none of that is really going to happen in light of the fact that when you complete your tests your folks, relatives, neighbours, and even your Facebook companions will begin getting some information about your feasible arrangements. What's more, don't mistake them for your gathering or dozing plans since they are alluding to your vocation Career after B com plans. In the present focused world, you are offered with many profession improving courses. On the off chance that you are not happy with the profession or course you decided for yourself at that point there are some present moment yet high worth – low speculation courses accessible in the market.

    ReplyDelete
  17. Thanks for provide great informatic and looking beautiful blog, really nice required information & the things i never imagined and i would request, wright more blog and blog post like that for us. Thanks you once agianMarriage certificate in delhi
    Marriage certificate in ghaziabad
    Marriage registration in gurgaon
    Marriage registration in noida
    special marriage act
    Marriage certificate online
    Marriage certificate in mumbai
    Marriage certificate in faridabad
    Marriage certificate in bangalore
    Marriage certificate in hyderabad thanks once again to all.

    ReplyDelete