Item12011: WysiwygPlugin fails to handle parenthesis in some parts of converted text (regex quoting error)

I have a error with editing page in foswiki (1.1.5) (I go to raw-editing (buttot wiki text in editor) from wysywyg-editing mode):

Unmatched ( in regex; marked by <-- HERE in m/\.%N%%ICON{connections}%  ==I change text== ( <-- HERE == $/

This error I have in Wysiwyg-editing mode on save result too.

I look log-file and read, that:

| 2012-07-24T12:07:47Z warning | Unmatched ( in regex; marked by <-- HERE in m/\
.^V%N%^W^V%ICON{connections}%^W  ==I change text== ( <-- HERE == $/ at /usr/local/ww
w/foswiki/lib/Foswiki/Plugins/WysiwygPlugin/HTML2TML/Node.pm line 1456.
 at /usr/local/www/foswiki/lib/Foswiki/Plugins/WysiwygPlugin/HTML2TML/Node.pm line 1456

I edit Node.pm and change string in Node.pm:

--- /tmp/Node.pm        2012-07-24 16:12:43.245258843 +0400
+++ /usr/local/www/foswiki/lib/Foswiki/Plugins/WysiwygPlugin/HTML2TML/Node.pm  2012-07-24 16:07:30.094270100 +0400
@@ -1453,7 +1453,7 @@
             $cleantext =~ s/^$this->{context}->{web}\.//;

             # if the clean text is the known topic we can ignore it
-            if ( ( $cleantext eq $href || $href =~ /\.\Q$cleantext\E$/ )
+            if ( ( $cleantext eq $href || $href =~ /\.$cleantext$/ )
                 && !$forceTML )
             {
                 return ( 0,

and all work.

-- RomanKondakov - 24 Jul 2012

Hi Roman, thank you very much for your patch. It looks good, but I'm having trouble reproducing the problem.

Can you please provide the exact text which triggers the fault?

Can you please also tell us which version of WysiwygPlugin you have installed? You can see at System.InstalledPlugins

On our current Release01x01 branch of Foswiki, I used the following text and successfully transitioned to & from WYSIWYG:

%N%
%ICON{connections}%
  ==I change text== (

I think this was accidentally fixed as a part of Item1396, but I have added a unit test so we can avoid this again in future.

If we can reproduce this error reliably we should raise this to urgent.

-- PaulHarvey - 25 Jul 2012 - 01:47

WysiwygPlugin (1.1.12, $Rev: 14939 (2012-05-31) $): Translator framework for WYSIWYG editors

I change text, because it is russian in KOI8-R codepage. I do not know, that you see in you installation.

It is from error.log

| 2012-07-24T12:07:47Z warning | Unmatched ( in regex; marked by <-- HERE in m/\.^V%N%^W^V%ICON{connections}%^W &#1055;&#1086;&#1076;&#1082;&#1083;&#1102;&#1095;&#1077;&#1085;&#1080;&#1077; &#1082; !WiFi &#1060;&#1072;&#1082;&#1091;&#1083;&#1100;&#1090;&#1077;&#1090;&#1072; ( <-- HERE &#1074; &#1090;&#1086;&#1084; &#1095;&#1080;&#1089;&#1083;&#1077; &#1076;&#1083;&#1103; &#1074;&#1099;&#1093;&#1086;&#1076;&#1072; &#1085;&#1072; &#1074;&#1099;&#1089;&#1086;&#1082;&#1086;&#1087;&#1088;&#1086;&#1080;&#1079;&#1074;&#1086;&#1076;&#1080;&#1090;&#1077;&#1083;&#1100;&#1085;&#1099;&#1077; &#1089;&#1080;&#1089;&#1090;&#1077;&#1084;&#1099; &#1060;&#1072;&#1082;&#1091;&#1083;&#1100;&#1090;&#1077;&#1090;&#1072;$/ at /usr/local/www/foswiki/lib/Foswiki/Plugins/WysiwygPlugin/HTML2TML/Node.pm line 1456.
 at /usr/local/www/foswiki/lib/Foswiki/Plugins/WysiwygPlugin/HTML2TML/Node.pm line 1456
        Foswiki::Plugins::WysiwygPlugin::HTML2TML::Node::_handleA('Foswiki::Plugins::WysiwygPlugin::HTML2TML::Node=HASH(0x10fdaa8)', 20) called at /usr/local/www/foswiki/lib/Foswiki/Plugins/WysiwygPlugin/HTML2TML/Node.pm line 591
        Foswiki::Plugins::WysiwygPlugin::HTML2TML::Node::generate('Foswiki::Plugins::WysiwygPlugin::HTML2TML::Node=HASH(0x10fdaa8)', 20) called at /usr/local/www/foswiki/lib/Foswiki/Plugins/WysiwygPlugin/HTML2TML/Node.pm line 922

-- RomanKondakov - 28 Jul 2012

lib/LocalSite.cfg:

$Foswiki::cfg{UseLocale} = 1;
$Foswiki::cfg{Site}{Locale} = 'ru_RU.KOI8-R';
$Foswiki::cfg{Site}{CharSet} = 'koi8-r';

-- RomanKondakov - 28 Jul 2012

Sorry, for copy-paste error.log it is translates in HTML-codes. In log I see KOI8-R symbols, not a HTML-codes.

I wait UTF-8 foswiki too.

-- RomanKondakov - 28 Jul 2012

We've been unable to recreate this. Deferring to 1.2. If locale is involved, there are several issues with locale in 1.1 that are being looked at for 1.2.

---

Okay. I'll put myself on the WaitingFor, as a note to try this again with KOI8-R

-- PaulHarvey - 03 Nov 2012

I am sorry, Paul!

That I am must doing now?

-- RomanKondakov - 23 Feb 2015

Roman

We need to review this carefully. At least it appears to me that something has gotten lost somewhere.

A change has been made to the Foswiki code to test for a possible regex error caused by editing Russian text via Wysiwyg which when you either save that topic or move to raw edit (both of which convert HTML to TML), then an error message appears in the text.

The test does not appear to me to be testing Russian, so it's not surprising it doesn't fail.

You supplied a patch and that has not been applied to FW and tested (after all the test as checked in doesn't fail).

Could you attach a text file with example Russian text of the topic before any editing. Please also describe or show what edits you actually need to make to the text to create this error.

Finally, we are working on FW 1.2 and there have been changes that get rid of a number of difficulties with using different locales.

Thanks

-- JulianLevens - 24 Feb 2015

Dropping myself from this task. Removing the \Q \E from the regex is probably not the right thing to do, it will cause the regex to treat the text as containing regex meta characters. Also deferring to 1.2.1. Hopefully some of the work in 1.2 will have bypassed this issue.

-- GeorgeClark - 21 Mar 2015

This needs validation / recreation on Foswiki 2.1.x. With the new Unicode core, hopefully issues like these are resolved. If you can create a TestItem12011 topic on Foswiki.org, that would be a great help.

-- GeorgeClark - 03 Dec 2016

Commenting is disabled while not logged in