To celebrate the release of spring rc1, I have updated the patches and the procedure for building custom versions of junit, surefire and spring3. Spring had about 500 commits since I created the initial patchset, and they were no longer applying cleanly.
The instructions in my original blogpost now reflect the revised procedure, and in about 20 minutes you can have it all up & running.
The original instructions can be found here.
Monday, October 5, 2009
Memo to self: How to patch an open source CVS/SVN project with GIT
Most CVS/SVN open source projects will prefer patches to be submitted as follows:
1. First patch contains a failing test
2. Second patch contains bugfix/improvement and possibly additional testcases and maybe also changes to the initial failing test case
The smart way to do this with git is as follows:
1. Clone project, if it's a SVN project use github's git-svn import (*much* perferred to using git svn clone yourself)
2. Create an initial feature branch masterForGit, where you add stuff that is nice for working this project in a git environment; typically a .gitignore file and any local scripts you need to be able to work with the project. Make sure this project compiles well and does not add unwanted files to git before proceeding (git does not handle empty directories same way as svn so you may need to correct this in this branch)
3. Create a feature branch failingTest based on masterForGit. In this branch you create the original failing
test what will be submitted as proof.
4. Create a feature branch bugFix that is branched off failingTest. In this branch you fix the bug and add any subsequent test cases.
5. When you're all done, you typically do diff between failingTest and masterForGit to get the patch file for "proof" bug. Then you diff bugFix and failingTest to get fix-patch. Since you created the "masterForGit" branch initially, none of the patches get polluted with temp-stuff you did locally.
And, if you end up maintaining this patchset, you can just update from svn through github and merge through all of your branches. Easy.
1. First patch contains a failing test
2. Second patch contains bugfix/improvement and possibly additional testcases and maybe also changes to the initial failing test case
The smart way to do this with git is as follows:
1. Clone project, if it's a SVN project use github's git-svn import (*much* perferred to using git svn clone yourself)
2. Create an initial feature branch masterForGit, where you add stuff that is nice for working this project in a git environment; typically a .gitignore file and any local scripts you need to be able to work with the project. Make sure this project compiles well and does not add unwanted files to git before proceeding (git does not handle empty directories same way as svn so you may need to correct this in this branch)
3. Create a feature branch failingTest based on masterForGit. In this branch you create the original failing
test what will be submitted as proof.
4. Create a feature branch bugFix that is branched off failingTest. In this branch you fix the bug and add any subsequent test cases.
5. When you're all done, you typically do diff between failingTest and masterForGit to get the patch file for "proof" bug. Then you diff bugFix and failingTest to get fix-patch. Since you created the "masterForGit" branch initially, none of the patches get polluted with temp-stuff you did locally.
And, if you end up maintaining this patchset, you can just update from svn through github and merge through all of your branches. Easy.
Thursday, October 1, 2009
Status of concurrency patches
AspectJ 1.6.6 is released and contains the patch I submitted there. 1 patch down.
The main patch for spring is scheduled for release 3.1. The patch no longer applies cleanly, but I will be updating it fairly soon, probably some time over the weekend. I may also try to publish a forked org.springframework.test to some maven repo by the time 3.0 goes release.
Surefire is another story. I am not sure anyone is actively maintaining this plugin any more. Anyone know any committers ? I have been considering forking the surefire plugin permanently and remove cruft, it sure has a lot of it.
As for the patches to junit itself they are submitted but still not accepted. There may be possible workarounds for this that I'll can consider when I revise the patches.
The main patch for spring is scheduled for release 3.1. The patch no longer applies cleanly, but I will be updating it fairly soon, probably some time over the weekend. I may also try to publish a forked org.springframework.test to some maven repo by the time 3.0 goes release.
Surefire is another story. I am not sure anyone is actively maintaining this plugin any more. Anyone know any committers ? I have been considering forking the surefire plugin permanently and remove cruft, it sure has a lot of it.
As for the patches to junit itself they are submitted but still not accepted. There may be possible workarounds for this that I'll can consider when I revise the patches.
Sunday, July 5, 2009
Are four cores required to surpass one ?
My recent post about running unit tests in parallel has led to some interesting food for thought:
When running regular unit tests, running with 2 cores seems to be no faster than one core. 4 cores (and arguably a newer CPU architecture) had a significant impact, where performance nearly doubled. The hyperthreaded quad-core i7 920 was *slower* than a 3Ghz core 2 duo when running single threaded, but nearly twice as fast in a multithreaded scenario. What's happening ?
I looked into the issue with jprofiler, but there is no obvious lock contention or other clearly visible bottleneck. The only explanation I can think of so far is this:
The test I was running was extremely simplistic, and there was no real cpu requirement in running the test methods. So the overhead of creating thread pools and all that actually amounts to just as much computation as running the tests. Likely ?
After my summer vacation, I will try to find out. I probably need to run with more realistic test cases. I figured I'd build surefire with threads, because thats a fairly heavy build. In the mean time, your thoughts are welcome.
From legoland,
Kristian
When running regular unit tests, running with 2 cores seems to be no faster than one core. 4 cores (and arguably a newer CPU architecture) had a significant impact, where performance nearly doubled. The hyperthreaded quad-core i7 920 was *slower* than a 3Ghz core 2 duo when running single threaded, but nearly twice as fast in a multithreaded scenario. What's happening ?
I looked into the issue with jprofiler, but there is no obvious lock contention or other clearly visible bottleneck. The only explanation I can think of so far is this:
The test I was running was extremely simplistic, and there was no real cpu requirement in running the test methods. So the overhead of creating thread pools and all that actually amounts to just as much computation as running the tests. Likely ?
After my summer vacation, I will try to find out. I probably need to run with more realistic test cases. I figured I'd build surefire with threads, because thats a fairly heavy build. In the mean time, your thoughts are welcome.
From legoland,
Kristian
Wednesday, July 1, 2009
Run your junit tests in parallel (with spring 3.0 and maven)
Swimming in a sea of patches...
Note; Most of the content in this post has been surpassed by a newer entry.
If you're willing to apply a few patches, you can now run your junit tests in parallel. There are two main use-cases for this; one is to make your regular unit tests run faster, the other is for io-bound tests, where you want to run several in parallel because you're waiting for external systems.
Just to whet your appetite, here are some numbers from a testcase (running 1000 instances of a simple unit-test-class):
Legend:
SC = SingleThreadedComputer (Computer, Junit47-snapshot class)
PC = ParallelComputer (ParallelComputer, Junit4.7-snapshot class)
CPC = ConfigurableParallelComputer (My project, see references)
C=Classes (one thread per class)
M=Methods (one thread per method)
| Core 2 duo 3.14Ghz | Intel i7 920 | ||
| SC | 203 | 451 | |
| PC(Classes, Method, Unlimited threads) | 406 | 806 | |
| CPC(Methods,16 threads) | 218 | 214 | |
| CPC(Classes, Unlimited threads) | 203 | 116 | |
| CPC(Classes,Methods, Unlimited threads) | 266 | 147 |
Running time in ms, 1000 simple tests
The table shows that for most dual-core solutions, running with parallel threads is currently never any faster than running non-threaded. Switching to the brand-new Intel i7 (4 hyperthreaded cores) has a marked effect, where simple unit test performance is doubled.
The lack of punch does, of course, change the moment your tests start doing IO (Integration tests, selenium tests and other web-tests).
The juicy stuff
svn co https://svn.openqa.org/svn/selenium-rc
Second build with mvn clean install: 102 seconds.
With 80 threads class-parallel on the i7: 37 seconds.
Class-parallel works easily on most tests.
A sea of patches
Getting all of this to work was not without its problems, though. To get this running, I had to patch just about half the libraries in use in my current stack. Furthermore, there's a huge difference between getting it to run and getting it to run without hiccups - every time. These patches insure that your infrastructure works as it should, but are your tests concurrent ?
Patches involved
See below for how to apply these patches in a simple manner.
Spring (3.0 trunk):
http://jira.springframework.org/browse/SPR-5863
AspectJ: Patch trunk with this patch (if using spring) (This patch is included in version 1.6.6)
https://bugs.eclipse.org/bugs/show_bug.cgi?id=281654
Junit4.7:
Several patches needed for thread safety:
http://github.com/KentBeck/junit/issues#issue/16
http://github.com/KentBeck/junit/issues#issue/18
Maven: You need a patched version of surefire that supports junit 4.7:
http://jira.codehaus.org/browse/SUREFIRE-555
You'll also probably want to be using the ConfigurableParallelComputer instead of the ParallelComputer that's included with JUnit.
git://github.com/krosenvold/configurable-parallel-computer.git
With these patches in place you're smooth sailing ;)
Applying the patches
Note: If you did this on the pre rc1 version of this patchset, I recommend that you start from a fresh copy of everything this time.
If you're using maven you could try this sequence:
git clone git://github.com/krosenvold/spring-framework-concurrent-junit-patch.git
git clone git://github.com/krosenvold/junit.git
git clone git://github.com/krosenvold/configurable-parallel-computer.git
git clone git://github.com/krosenvold/spring-test.git
git clone git://github.com/krosenvold/spring-concurrency-test.git
cd junit
ant
cd ../spring-framework-concurrent-junit-patch
./installJunit
cd ../spring-test
mvn install
cd ../configurable-parallel-computer
mvn install
cd ../spring-framework-concurrent-junit-patch
./checkoutSurefire
./buildSureFire
./applySureFirePatch
./buildSureFire
cd ../spring-concurrency-test
mvn install # Test project to verify that everything is ok
Your new versions
JUnit should be modified to 4.7-CONCURRENT
spring-test should have version number 3.0.0.RC1-CONCURRENT
Surefire should be modified to 2.5-SNAPSHOT
You must update your aspectj version to 1.6.6 in your project. Spring does a transitive
include on 1.6.5
You can see the version number for the spring artifacts when building spring. Probably something like 3.0.0.BUILD-20091005102426
Maven surefire settings
<plugin>
<artifactid>maven-surefire-plugin</artifactid>
<version>2.5-SNAPSHOT</version>
<configuration>
<parallel>classes</parallel>
<useUnlimitedThreads>true</useUnlimitedThreads>
<threadCount>80</threadCount>
<perCoreThreadCount>true</perCoreThreadCount>
</configuration>
</plugin>
UnlimitedThreads knocks out threadCount and perCoreThreadCount.
Look at the various "install" scripts to see the version numbers of the artifacts.
Simple, eh ?
And while we're at it: Half of the work with patching these projects has been about fighting with version control systems and custom made build scripts. So please; switch to GIT and maven: I can build any maven project without reading as much as a readme file. Maven allows me as a casual bystander to *easily* make contributions; I never need to read that 4 page "how to build" manual! Git is the only version control tool that respects my right/duty to patch, change and upgrade other projects. It allows me to do all these things without having to *ask* anyone. Repeat after me: Git and maven encourages free collaboration; a key idea of open source, right ?
http://twitter.com/krosenvold
Subscribe to:
Posts (Atom)
