Item4893: Test suite does not work with standard release

I realize that using the test suite on a standard release is not the way most developers work, but nevertheless it seems a reasonable thing that one might want to do.

For example, one might want to develop plugins against the current release, rather than the develop branch. In doing so, one may want to also run the TWikiSuite to ensure one did not break anything.

If one wants to set up unit tests there are a number of obstacles:

• The UnitTestContrib is not available on twiki.org, but must be retrieved from svn. There is no documentation how to retrieve just a subdirectory and not the whole release.
• The unit tests themselves are not available on twiki.org, but must be retrieved from svn.
• The required CPAN libraries are not documented.
• The file lib/MANIFEST is required to run many test suites. This is not obvious from the error message without actually studying the failing source code. Why this is needed is unclear. This file is not in the standard release but has to be found on svn.
• The unit tests then also require files that are not present in the release. For example, a number of tests rely on TWiki/LoginManager/TemplateLogin.pm which is not in the standard release. This becomes problematic as adding such files may cause problems.

The long and short of my whining is that I think we should maintain the unit tests that held when a release was made. (Unit tests may become outdated as develop progresses, for example, or as in the current case, they may not work on the current release due to newly added dependencies.)

-- TWiki:Main/ThomasWeigert - 26 Oct 2007

We have always made a complete copy of the MAIN directory when we created a release branch. And we always copy the entire release branch when we create a tag of each exact release (+ all the ,v files that are added only to tags).

So what additional things do you propose Thomas?

-- TWiki:Main.KennethLavrsen - 27 Oct 2007

Thomas is right, we need to be working in that direction. I moved the unit tests to a contrib as the first step in the process that would lead to the tests being runnable against a release, rather than always requiring SVN. It's just a question of time. However I'm reclassifying this as an Enhancement.

-- TWiki:Main.CrawfordCurrie - 27 Oct 2007

When I created the Release01x00 branch on SVN I copied only core, default plugin, the Unit test contrib, and other topic based test contrib and the build contrib.

So it is possible now to checkout a small reasonable Release01x00 branch and do your plugin development under this.

It is also possible using pseudo-install combined with the right environment variable to pseudo link individual plugins that are in the trunk from the Release01x00.

I not sure what more we need. Being able to both build, run test suites and link plugins from outside the release branch seems to give pretty good options for a developer now.

And I do not believe a plugin needs to be checked into SVN to be able to run unit tests for it.

So I am not sure if there is much left to action on this one

-- KennethLavrsen - 08 Mar 2009