Item9418: Uploading files causes fatal error

Try to attach a file to a topic

the result is

[Sat Jul 31 23:37:28 2010] [error] [client 192.168.1.11] Can't use an undefined value as a symbol reference at /var/www/foswiki/core/lib/Foswiki/Request/Upload.pm line 86., referer: http://merlin.lavrsen.dk/trunk/bin/attach/Sandbox/TestTopic6
[Sat Jul 31 23:37:28 2010] [error] [client 192.168.1.11]  at /var/www/foswiki/core/lib/Foswiki/Request/Upload.pm line 86, referer: http://merlin.lavrsen.dk/trunk/bin/attach/Sandbox/TestTopic6
[Sat Jul 31 23:37:28 2010] [error] [client 192.168.1.11] \tFoswiki::Request::Upload::handle('Foswiki::Request::Upload=HASH(0x9bc1c6c)') called at /var/www/foswiki/core/lib/Foswiki/Request.pm line 626, referer: http://merlin.lavrsen.dk/trunk/bin/attach/Sandbox/TestTopic6
[Sat Jul 31 23:37:28 2010] [error] [client 192.168.1.11] \tFoswiki::Request::upload('Foswiki::Request=HASH(0x9baaa60)', 'filepath') called at /var/www/foswiki/core/lib/Foswiki/UI/Upload.pm line 152, referer: http://merlin.lavrsen.dk/trunk/bin/attach/Sandbox/TestTopic6
[Sat Jul 31 23:37:28 2010] [error] [client 192.168.1.11] \tFoswiki::UI::Upload::_upload('Foswiki=HASH(0x9bc80c4)') called at /var/www/foswiki/core/lib/Foswiki/UI/Upload.pm line 87, referer: http://merlin.lavrsen.dk/trunk/bin/attach/Sandbox/TestTopic6
[Sat Jul 31 23:37:28 2010] [error] [client 192.168.1.11] \tFoswiki::UI::Upload::upload('Foswiki=HASH(0x9bc80c4)') called at /var/www/foswiki/core/lib/Foswiki/UI.pm line 310, referer: http://merlin.lavrsen.dk/trunk/bin/attach/Sandbox/TestTopic6
[Sat Jul 31 23:37:28 2010] [error] [client 192.168.1.11] \tFoswiki::UI::__ANON__() called at /usr/lib/perl5/vendor_perl/5.8.8/Error.pm line 415, referer: http://merlin.lavrsen.dk/trunk/bin/attach/Sandbox/TestTopic6
[Sat Jul 31 23:37:28 2010] [error] [client 192.168.1.11] \teval {...} called at /usr/lib/perl5/vendor_perl/5.8.8/Error.pm line 407, referer: http://merlin.lavrsen.dk/trunk/bin/attach/Sandbox/TestTopic6
[Sat Jul 31 23:37:28 2010] [error] [client 192.168.1.11] \tError::subs::try('CODE(0x9b82200)', 'HASH(0x9bc7e78)') called at /var/www/foswiki/core/lib/Foswiki/UI.pm line 429, referer: http://merlin.lavrsen.dk/trunk/bin/attach/Sandbox/TestTopic6
[Sat Jul 31 23:37:28 2010] [error] [client 192.168.1.11] \tFoswiki::UI::_execute('Foswiki::Request=HASH(0x9baaa60)', 'CODE(0x9baa778)', 'upload', 1) called at /var/www/foswiki/core/lib/Foswiki/UI.pm line 277, referer: http://merlin.lavrsen.dk/trunk/bin/attach/Sandbox/TestTopic6
[Sat Jul 31 23:37:28 2010] [error] [client 192.168.1.11] \tFoswiki::UI::handleRequest('Foswiki::Request=HASH(0x9baaa60)') called at /var/www/foswiki/core/lib/Foswiki/Engine/CGI.pm line 30, referer: http://merlin.lavrsen.dk/trunk/bin/attach/Sandbox/TestTopic6
[Sat Jul 31 23:37:28 2010] [error] [client 192.168.1.11] \tFoswiki::Engine::CGI::run('Foswiki::Engine::CGI=HASH(0x9841c60)') called at /var/www/foswiki/core/bin/upload line 24., referer: http://merlin.lavrsen.dk/trunk/bin/attach/Sandbox/TestTopic6

-- KennethLavrsen - 31 Jul 2010

I have done a binary svn checkout and located that the checkin that broke this is SVN 8174.

A checkin related to charsets and textarea. Not related to attaching at all.

I bet it is again a Perl 5.8 vs 5.10 thing.

Crawford that 8174 checkin was yours.

I run with $Foswiki::cfg{Site}{CharSet} = 'iso-8859-1';

If I revert the part of SVN 8174 in Foswiki.pm then I can attach again. It is the code line

CGI::charset( $Foswiki::cfg{Site}{CharSet} );

that breaks things.

Probably this was not implemented correctly with the CGI that comes with perl 5.8.

I have tried to do a print STDERR CGI::charset() and it returns iso-8859-1 which seems correct.

But the call to CGI::charset earlier seems to set something that then later breaks attaching.

-- KennethLavrsen - 31 Jul 2010

Debugged further.

In lib/Foswiki/UI/Upload.pm around line 136 we have

        try {
            $tmpFilePath = $query->tmpFileName($fh);
        }

The $tmpFilePath is set to a value like "/usr/tmp/CGItemp54808" when I have commented out CGI::charset( $Foswiki::cfg{Site}{CharSet} ); in Foswiki.pm

Without commenting out the CGI::Charset line the $tmpFilePath is an empty string. So $query->tmpFileName($fh) fails.

Also in lib/Foswiki/Engine/CGI.pm in the sub prepareUploads the $this->{cgi}->tmpFileName($fname) returns nothing if we have called CGI::charset.

It seems that in perl 5.8 there is a problem with calling this. We cannot use that call in that place. Maybe the problem is that we cannot call CGI this early. I tried to add the same line of code just before the $this->{cgi}->tmpFileName($fname) in lib/Foswiki/Engine/CGI.pm and that did not cause harm.

Adding GilmarSantosJr for comments also.

I have spent 5 hours on this bug now. Need sleep.

-- KennethLavrsen - 01 Aug 2010

The call to CGI::charset is correct and very necessary, unfortunately. The call to tmpFileName has been suspect for years.

Without being able to reproduce this, it's hard to comment. One thing to try is moving the charset call after the CGI object has been constructed:

Index: Foswiki.pm
===================================================================
--- Foswiki.pm   (revision 8357)
+++ Foswiki.pm   (working copy)
@@ -380,16 +380,6 @@
         setlocale( &LC_COLLATE, $Foswiki::cfg{Site}{Locale} );
     }
 
-    if ( defined $Foswiki::cfg{Site}{CharSet} ) {
-
-        # Ensure the auto-encoding in CGI uses the correct character set.
-        # CGI defaults to iso-8859-1, and has a special exception for
-        # iso-8859-1 and windows1252 in CGI::escapeHTML which breaks
-        # UTF-8 content. See Item758. Get this wrong, and CGI will
-        # fail to encode certain UTF-8 characters correctly.
-        CGI::charset( $Foswiki::cfg{Site}{CharSet} );
-    }
-
     $macros{CHARSET} = sub {
         $Foswiki::cfg{Site}{CharSet} || CGI::charset();
     };
@@ -1624,6 +1614,16 @@
     $query ||= new Foswiki::Request();
     my $this = bless( { sandbox => 'Foswiki::Sandbox' }, $class );
 
+    if ( defined $Foswiki::cfg{Site}{CharSet} ) {
+
+        # Ensure the auto-encoding in CGI uses the correct character set.
+        # CGI defaults to iso-8859-1, and has a special exception for
+        # iso-8859-1 and windows1252 in CGI::escapeHTML which breaks
+        # UTF-8 content. See Item758. Get this wrong, and CGI will
+        # fail to encode certain UTF-8 characters correctly.
+        CGI::charset( $Foswiki::cfg{Site}{CharSet} );
+    }
+
     $this->{request}  = $query;
     $this->{cgiQuery} = $query;    # for backwards compatibility in contribs
     $this->{response} = new Foswiki::Response();

Though the bug that was fixed by the charset call will have to be checked for carefully.

-- CrawfordCurrie - 01 Aug 2010

My guess was correct. We call CGI::charset too early.

Your patch cures the problem. But what about the lines

     $macros{CHARSET} = sub {
         $Foswiki::cfg{Site}{CharSet} || CGI::charset();
     };

That must need moving also.

-- KennethLavrsen - 01 Aug 2010

According to Crawford it is OK because it is a sub.

I have verified that the fix works. I have also changed my test site to UTF8 and tested that Item758 is still fixed, and it is. I tried both with the actual Chinese characters and with Danish characters

-- KennethLavrsen - 01 Aug 2010

I believe that the "fix" above was subtly masking a deeper problem, which is documented in Item12084

-- CrawfordCurrie - 24 Oct 2013

Commenting is disabled while not logged in