Item14668: DBCachePlugin: $encode($ formfield(...)) breaks on closing brackets within the formfield

I encountered a case with quotes within a formfield value which broke my VarSet construct (see Sandbox.TestSet for an example using a search, but the same applies to dbquery of course).

To help with this I wrote $encode($formfield(Description)) which solved the quoting problem just fine.

Unfortunately then I got another case with a closing bracket ) within the formfield. This broke the $encode().

I see that closing brackets are indeed handled and replaced with a translation token, but this replacement is reversed before executing _expandFormatTokens() which performs the $encode() and other things:

lib/Foswiki/Plugins/DBCachePlugin/Core.pm

sub _expandFormatTokens {
  my $text = shift;

  return '' unless defined $text;

  $text =~ s/\$perce?nt/\%/g;
  $text =~ s/\$nop//g;
  $text =~ s/\$n/\n/g;
  $text =~ s/\$encode\((.*?)\)/_entityEncode($1)/ges;
  $text =~ s/\$trunc\((.*?),\s*(\d+)\)/substr($1,0,$2)/ges;
  $text =~ s/\$lc\((.*?)\)/lc($1)/ge;
  $text =~ s/\$uc\((.*?)\)/uc($1)/ge;
  $text =~ s/\$dollar/\$/g;

  return $text;
}

So my first thought is to reverse the replacement after _expandFormatTokens() is called.

Another solution would be to create $encformfield() which does an extra _entityEncode() after getting the field. But what about $trunc(), $lc() and $uc() then?

This is my fix for now:

--- lib/Foswiki/Plugins/DBCachePlugin/Core-12.0.pm   2018-10-01 13:13:38.000000000 +0200
+++ lib/Foswiki/Plugins/DBCachePlugin/Core.pm   2018-10-12 15:20:34.959321000 +0200
@@ -337,7 +337,6 @@
       $line =~ s/\$rss\((.*?)\)/_rss($1, $web, $thisTopic)/ges;
       $line =~ s/\$translate\((.*?)\)/_translate($1, $theWeb, $theTopic)/ges;
 
-      $line =~ s/${TranslationToken}/)/g;
       push @result, $line;
 
       $Foswiki::Plugins::DBCachePlugin::addDependency->($web, $topicName);
@@ -350,6 +349,7 @@
 
   $text = _expandVariables($text, $thisWeb, $thisTopic, count => ($hits ? $hits->count : 0), web => $thisWeb);
   $text = _expandFormatTokens($text);
+  $text =~ s/${TranslationToken}/)/g;
 
   $this->fixInclude($thisWeb, $text) if $theRemote;
 

-- StefanH - 23 Mar 2018

Unfortunately $encode() breaks macro expansion. For example if there is some colored markup within the formfield, too many characters are encoded and the markup code stays as it is.

Because of this I created another variable $saveformfield() which only replaces doublequotes (for now):

--- lib/Foswiki/Plugins/DBCachePlugin/Core-12.0.pm   2018-10-12 15:28:00.780743000 +0200
+++ lib/Foswiki/Plugins/DBCachePlugin/Core.pm   2018-10-12 15:33:16.588132000 +0200
@@ -316,6 +316,12 @@
         $temp =~ s#\)#${TranslationToken}#g;
         $temp =~ s#\r?\n#$theNewline#gs if defined $theNewline;
         $temp/geo;
+      $line =~ s/\$saveformfield\((.*?)\)/
+        my $temp = $theDB->getFormField($topicName, $1);
+        $temp =~ s#\)#${TranslationToken}#g;
+        $temp =~ s#\r?\n#$theNewline#gs if defined $theNewline;
+        $temp =~ s|(["\|])|'&#'.ord($1).';'|ge;
+        $temp/geo;
       $line =~ s/\$expand\((.*?)\)/
         my $temp = $1;
         $temp = $theDB->expandPath($topicObj, $temp);

-- StefanH - 26 Mar 2018

I updated the patches to work with the latest DBCachePlugin 11.00.

-- StefanH - 21 Aug 2018

I added the pipe to be replaced as well to not brake wiki tables. I am still not really happy with this solution, maybe there is another approach to solve this?

-- StefanH - 10 Sep 2018

Updated patches for DBCachePlugin 12.00.

-- StefanH - 12 Oct 2018

Commenting is disabled while not logged in