Item8386: MailInContrib unit tests fails

The MailInContrib unit tests fail on my computer. I tested on the release1x0 branch.

Dependency on order of files in directory

testTopicPathTo (which I used to diagnose the problem) writes two messages to a temporary maildir directory, calls the contrib's processInbox to process those messages and then checks that the topic has the correct content. This check fails - I can see that the topic has the messages in the wrong order. However, MailInContrib is behaving correctly; processInbox processes the second message before the first because Email::Folder returns them in that order.

As far as I can see, the fault is that MailInContrib's unit tests create files in a new directory and expects Email::Folder to return them in the order that they were created. On my machine, this is not the case. Email::Folder::Maildir gets the list of files with readdir, which returns them in a different order on my machine.

Here is a perl script that illustrates how readdir returns the files in a different order to the order that they were created.

Here is the output I get:

Creating new directory
Creating new files:
   * apple.txt
   * celery.txt
   * banana.txt

Plain 'ls':
apple.txt
banana.txt
celery.txt

'ls -U':
banana.txt
celery.txt
apple.txt

Perl's readdir:
banana.txt
celery.txt
apple.txt
..
.

Removing files
Removing directory

My /tmp is on an ext3 filesystem, on ubuntu 8.10.

-- MichaelTempest - 11 Jan 2010

I discovered that the order of files in a directory on ext3 is based on hash of the filename and a "secret" that is different for each filesystem So the order of a set of files should be consistent on a given ext3 filesystem, but will differ between various ext3 filesystems (See here and here). So if I understand this correctly, there is thus a 50% chance of these tests passing on any given ext3 filesystem, but the results will be consistent on that filesystem (because the filenames and the filesystem-specific secret do not change, so the hashes do not change).

To test this, I modified sendTestMail to include the process ID in mail filenames: "$box->{folder}new/mail".$$.$nbr
instead of: "$box->{folder}new/mail$nbr"

I now find that the test passes half the time, and fails half the time.

-- MichaelTempest - 19 Jan 2010

Inconsistent expected results

I tried splitting up the unit tests so that each unit test uses a single mail message. I still encountered problems.

testOnTopicNoSpam expects the following to clear the message from the topic text ...

    $t =~
s/\* \*no valid topic\*: Message 1 text here\s*-- $this->{users_web}.AllyGator -\s+\d+\s+\w+\s+\d+\s+-\s+\d+:\d+//s;

... which matches the "normal" template in the MailInContribTemplate topic.

However, testTopicPathTo and testTopicPathSubject expect this to clear the message ...

    $this->assert( 0, $t )
      unless $t =~ s/^Message 1 text here\s*//s;
    $this->assert( 0, $t )
      unless $t =~
s/^_$this->{users_web}\.MoleInnaHole\s*\@\s*\d+\s+\w+\s+\d+\s+-\s+\d+:\d+_\s*//s;

... which matches the default template in _saveTopic in MailInContrib.pm. If I run these tests, I see that MailInContrib used the "normal" template from MailInContribTemplate, not the default in _saveTopic.

I do not understand why the expected results differ. I can see that testTopicPathTo and testTopicPathSubject might pass if the contrib is not installed (but I have not been able to test that). I do not see how all of these tests could pass at the same time, yet it is reported that they do pass, so obviously I am missing something . So if nothing else, perhaps the tests lack some documentation?

testUserTemplate fails

This test fails when I run it on the release branch because MailInContrib uses the default template in _saveTopic, and not the specified user template. This is because Foswiki cannot find the user template topic. I put debug code into lib/Foswiki/Templates.pm. It gave this output:

no MailInContribUser template at /home/www/fw_1x0/core/templates/TemporaryMailInContribTestsUsersWeb/MailInContribUser..tmpl, /home/www/fw_1x0/core/templates/MailInContribUser..tmpl, SkinMailInContribUserTemplate, SkinMailInContribUserTemplate, /home/www/fw_1x0/core/templates/TemporaryMailInContribTestsUsersWeb/MailInContribUser.tmpl, /home/www/fw_1x0/core/templates/MailInContribUser.tmpl, MailInContribUserTemplate, MailInContribUserTemplate

Note that the list includes topics in the System web, but not in the TemporaryMailInContribSuiteSystemWeb web. I think that is causing the test to fail.

Tests do not run on trunk

The unit tests do not run on trunk because they make assumptions about Foswiki internals that are true for the 1x0 release branch, but not on trunk. The following call in setup() fails because $this->{twiki}->{store} is undef:

    $this->{twiki}->{store}->createWeb( $this->{twiki}->{user},
        $this->{system_web}, $Foswiki::cfg{SystemWebName} );

I can get the tests to run if I replace that statement with the code that follows. The tests do not all pass, but at least they run.

    $this->{twiki} ||= $this->{session};
    Foswiki::Func::createWeb($this->{system_web}, $Foswiki::cfg{SystemWebName});

-- MichaelTempest - 17 Jan 2010

Tricky. I don't have any problem with Email::Folder on my system. But as for the createWeb you are right, and the templates.... well, it looks like someone did a half-job on supporting user templates for the incoming mail (maybe some of their work was lost in translation) but it wasn't loading user templates (problem 1). problem 2 was in the testcase; it switches the system web to a copy/derivative, but the {TemplatePath}, which is expanded when localSite.cfg loads, was still using System - so it wasn't finding the right templates.

The unit tests pass for me in trunk. Note that the one minor change in the core code is only needed if you want to define user templates; otherwise it ought to all work.

-- CrawfordCurrie - 18 Jan 2010

I added to the description of the Email::Folder/file-order problem

-- MichaelTempest - 19 Jan 2010

I changed the tests to be independent of the actual file order - so I hope they still pass on your machine If they do pass, then I think this task may be closed. If the tests fail, I will revert the change I made.

-- MichaelTempest - 21 Jan 2010

Indeed they do. Many thanks.

-- CrawfordCurrie - 22 Jan 2010

Commenting is disabled while not logged in

• test_file_order.pl.txt: Perl script to reproduce the order-swapping problem