Item8032: The means to put WYSIWYG on form fields

My client just asked me whether TWiki/Foswiki can edit it's Text Fields using Wysiwyg.

What's easy

• Tagging the fields so that TinyMCE gets invoked on them

Challenges

Issue	Proposed solution
* Lavr: Wiki application depend on content in form fields. Some would break with html tags in the field (this can easily happen with TinyMCE, e.g. Just take a table that saves well as TML. Now pull a little on the side in Wysiwyg mode and save. Result - an HTML table which is hard to get back in TML). But a formatted search for formfield content can easily get into trouble	form defintions would benefit from a "never html" flag

Status

Outstanding issues

1. Would the community want to have plain textareas as well as wysiwyg text areas?
   1. Lavr: Martin. I would say yes. A possible way to implement it would be to add a new attribute to form definitions (like we have H for hidden and M for mandatory) to enable Wysiwyg editing on textarea fields.
   2. The reason people may want both is that small text areas do not work well with Wysiwyg. Or you may need more control of what people type in of code they have no direct control over
2. Should I package this as a single plugin, or updates to three plugins? Is it okay to overwrite single plugins?

Wysiwyg forms works, but has a serious bug that needs to be solved before the code can be used in production:

1. Excess BR's appear in the HTML content, e.g. after the <li>'s
2. The browser reorganises these to appear before the <ol>
3. When round-tripped, the BRs stay in the wrong place and become missing from the end of the lines for which they are needed.

This breaks all linebreak sensitive code, especially tables.

If you transition Wysiwyg -> TML a single field, and leave the rest in Wysiwyg, when you save the page the TML field is corrupted.

1. Does the field in TML form retain its HTML SECRET_ID?

Test Plan

• Do HTML links to pages inside the wiki get canonicalised?
• Have we a list of conditions in which WYSIWYG was thought to break, and are we sure we have addressed them all?

Code changes

I've attached the code changes:

Archive:  WysiwygFormFields.zip
  Length     Date   Time    Name
 --------    ----   ----    ----
        0  01-05-09 18:56   WysiwygFormFields/
     9887  01-05-09 17:53   WysiwygFormFields/FieldDefinition.pm
    29648  01-05-09 18:53   WysiwygFormFields/WysiwygPlugin.pm.pre-diff
    13885  01-05-09 17:52   WysiwygFormFields/twiki_tiny.js
     5393  01-05-09 17:28   WysiwygFormFields/WysiwygPlugin.pm.16836-17359
    29648  01-05-09 17:52   WysiwygFormFields/WysiwygPlugin.pm.orig
    30155  01-05-09 18:54   WysiwygFormFields/WysiwygPlugin.pm
 --------                   -------
   118616                   7 files

• FieldDefinition.pm replaces lib/TWiki/Form/FieldDefinition.pm (4.2.3)
• WysiwygPlugin.pm replaces lib/TWiki/Plugins/WysiwygPlugin.pm (17359)
• twiki_tiny.js replaces pub/TWiki/TinyMCEPlugin/twiki_tiny.js (rev unknown).

NB. the js file strictly speaking is the/twiki_tiny_src.js file, but the installer would put the abridged version to /twiki_tiny.js

You may need to enable the plugin for fields since http://trac.foswiki.org/changeset/1475. To do this, add the following to TWikiPreferences:

   * Set TINYMCEPLUGIN_INIT = mode:"textareas",
   editor_selector : "twikiTextarea|twikiEditFormTextAreaField", 
   ... (+ rest of the settings from TWiki.TinyMCEPlugin)

Background

I asked on irc http://irclogs.foswiki.org/bin/irclogger_log/foswiki?date=2008-12-22,Mon&sel=562#l558 ; my conclusion is that it is not possible.

It appears that people may have hacked this by getting TinyMCE to "trigger on all forms with style defined in editor_selector" (set the "style of your textarea to foswikiWysiwygEdit") however, "newlines and encoding will screw up".

Apparently, "using WYSIWYG for meta will break it". As far as EugenMayer knows CrawfordCurrie has his reasons not to take WYSIWYG editing for meta fields, among which, according to KennethLavrsen that "Wysiwyg will even put some garbage text in the fields if you let it work form fields." For instance, "contents are not cleaned up after editing, links are 'absolute' even if they are internal". The code for this is in TranslateHTML2TML.

Eugen said it's the same problem for DBConnectorPlugin. and that he will "talk to Crawford to implement a different 'transform' way, so that every content edited by WYSIWYG is converted properly"

-- MartinCleaver - 22 Dec 2008

foswikiWysiwygEdit was added by me a few days ago because we got the problem with Wysiwyg attacking form fields for probably the 5th time.

The Wysiwyg does not support form fields. That is a known fact.

It can support editing textareas inside a topic area if you construct it right. But no matter what style class we use - it does not change the fact that textareas in formfields are single line entities with breaks. You cannot make a TML table in a form field for the same reason.

Maybe a more universal long term solution would be to allow meta to be be multiline so you can do the same in a textarea field as you can do inside a topic.

Having Wysiwyg to translate HTML to TML in two different ways depending on form field or not sounds like a nightmare.

• KennethLavrsen - 22 Dec 2008

Eugen has kindly offered a likely patch to accomplish this. Abridged:

if ($text =~ s/^(%META:[A-Z]+{.*?}%r?n)//s) {
$meta = $meta.$1;
}
then down, the call to the transform function by
$_[0] = TranslateHTML2TML( $text, $_[1], $_[2], $meta);

-- MartinCleaver - 23 Dec 2008

Looking at http://irclogs.foswiki.org/bin/irclogger_log/foswiki?date=2008-12-22,Mon&sel=828#l824; on investigation Eugen's patch looks like it would suffer from whatever problems the TranslateHTML2TML "don't process meta" code would have added.

I've done some further analysis, following from Eugen's reference to per http://trac.foswiki.org/browser/trunk/WysiwygPlugin/lib/Foswiki/Plugins/WysiwygPlugin.pm#L201 which

1. Takes the page content from the HTML editor, surrounded by TWiki META directives
2. Identifies META data at the top of the content
3. Identifies META data at the bottom of the content (this includes META:FIELD definitions)
4. Takes what's remaining, the content, and, transforms HTML in the content to its TML equivalent.
5. Recomposes the output to be TWiki equivalent

The upshot is that this process, which ignores META, does not translate HTML in META:FIELDs into their TML equivalents.

In sub TranslateHTML2TML {
   198      # SMELL: really, really bad smell; bloody core should NOT pass text
   199       # with embedded meta to plugins! It is VERY BAD DESIGN!!!
   200       my $top = '';
   201       if ($text =~ s/^(%META:[A-Z]+{.*?}%\r?\n)//s) {
   202           $top = $1;
   203       }
   204       my $bottom = '';
   205       $text =~ s/^(%META:[A-Z]+{.*?}%\r?\n)/$bottom = "$1$bottom";''/gem;
   206   
   207       my $opts = {
   208           web => $web,
   209           topic => $topic,
   210           convertImage => \&_convertImage,
   211           rewriteURL => \&postConvertURL,
   212           very_clean => 1, # aggressively polish saved HTML
   213       };
   214   
                   ######## CONVERT ONLY THE TEXT
   215       $text = $html2tml->convert( $text, $opts );
   216   
   217       $text =~ s/\s+$/\n/s;
   218   
                    ######## PUT THE TOP and BOTTOM back into the stream 
   219       return $top.$text.$bottom;

Notwithstanding that Textareas are different to topic texts (see below), I believe there are three parts to a solution.

Details below.

I would appreciate ideas of whether what I have suggested will work. It now looks a bit more involved so I want to make sure I can figure out what effort is needed.

1 Meta form fields need to be edited with the WYSIWYG Javascript editor

EugenMayer: Actually if you put the lines into your WYSIWYG, all you need now is activate TinyMCE for forms. This is a really easy task. In TWikiPreferences * Set TINYMCEPLUGIN_INIT =

editor_selector : "(twikiTextarea|twikiEditFormTextAreaField)",

1b TML newlines would need storing as x0dx0a for fields because Textareas are different to topic texts

Where Kenneth mentions above that "textareas in formfields are single line entities with breaks", breaks means <br> whereas in topic content <br> would be stored as a newline.

In TWiki 4.1 the sequence 0xd0xa in a field value would be sent to the renderer as a newline, making TML in formfields render properly. In TWiki 4.2.x onward something changed with the effect that the embedded sequence:

   * first
   * second
   * third

shows not as:

• first
• second
• third

but rather as:

 
   • first
   * second
   * third

This has been fixed in the past, Item5489 talks through it.

2. The form fields would need to be not ignored by TranslateHTML2TML's "don't process meta"*

Instead of ignoring a HTML2TML conversion of fields, push them through one by one .

Eugen's patch goes in at line 206 of TranslateHTML2TML:

if ( $meta =~ s/<!--$SECRET_ID-->//go) {
   $meta =~ s/(^%META:FIELD{[^{]*.?value=")(.*?)("})/$1.uri_escape($html2tml->convert( uri_unescape($2), $opts )).$3/gme;
}

It looks like, although not all metadata should be pushed through conversion, some META:FIELD data should be. I am assuming this would take care of:

• "contents are not cleaned up after editing, links are 'absolute' even if they are internal"

I am further unclear on:

• whether this transform would be computationally expensive

3. TML in form fields would need to be rendered properly on display, AND render of x0dx0a as newlines

Not rendering fields as TML is deliberate, and implemented by:

338     $value = Foswiki::Render::protectFormFieldValue( $value, $attrs );

Showing TML in form fields would be quite simple to verify.

3b render TML x0dx0a as newlines

There must be code in the 4.1 to 4.2 check in that we can disable.

3c editing of TML in fields needs rendering as HTML for the edit cycle

The WYSIWYG plugin is "responsible for translating TML to HTML before an edit starts and translating the resultant HTML back into TML."

TML stored in form fields needs to get rendered.

4 TinyMCEPlugin needs to be modified to support simultaneous TML<->HTML roundtrips

It turns out that TinyMCEPlugin runs a bunch of assumptions. The most important of which is that there will only ever be one tinymce textarea on the page. TinyMCE It uses a REST interface to translate stored TML into HTML that tinymce can use, but the twiki_tiny.js code that calls this is not re-entrant. In the case where you want multiple wysiwyg form fields on the page all fields would need transforming from TML to HTML, and you'd likely want to transform them all at the same time during page load. Therefore the TinyMCEPlugin would need now to be reentrant to deal with requests for multiple fields during page load.

Notes

• Eugen's solution is to alter the parameter signature of HTML2TML, adding ,$meta as the last parameter to the call TranslateHTML2TML in (what I assume is) beforeSaveHandler

144   sub beforeSaveHandler {
145       #my( $text, $topic, $web ) = @_;
146       my $query = Foswiki::Func::getCgiQuery();
147       return unless $query;
148   
149       return unless defined( $query->param( 'wysiwyg_edit' ));
150   
15.       $_[0] = TranslateHTML2TML( $_[0], $_[1], $_[2] );
15.   }

Adding a meta parameter thusly:

150
151   # extracting meta
152   my $meta = '';
153   if ($text =~ s/^(%META:[A-Z]+{.*?}%r?n)//s) {
154       $meta = $meta.$1;
155   }
156   $_[0] = TranslateHTML2TML( $_[0], $_[1], $_[2], $meta );
15.   }

This does not address the other caller of TranslateHTML2TML, afterEditHandler

The routine has a confused notion of $text which is not just the TML of the page but the entire content: top meta, text and bottom meta.

---

Editing raw shows the following in the form fields topic text.

<!--WYSIWYG content - do not remove this comment, and never use this identical text in your topics-->

Editing WYSIWYG shows the content rendered WYSIWYG.

---

At this point I realised that I needed TML content embedded in the fields to render properly. As per Item5489

---

NB. Sizing of fields shows up wrong, especially the tiny textarea field which was specified as

---

It would be simpler to store html in the topic fields, but this would create inconsistencies with the rest of Foswiki.

---

Test cases:

1. Table round tripping 1 In TML edit mode, enter into a field the text

| A | B |
| C | D |

1. Look at the field in rendered mode. It should show as a TML table (alternates blue and white)
2. Edit the field in WYSIWYG mode. It should show as a HTML table. It does not. It shows as TML and all on one line. Save the topic.
3. Look at the field in rendered mode. It now shows as a broken TML table, all on one line in rendered mode.

This is because 1) newlines are not encoded and are therefore lost.

---

(Eugen's comment deleted, as per IRC conv).

Although Eugen did his changes for 16836 of WysiwygPlugin, according to http://develop.twiki.org/trac/changeset/17714/twiki/trunk/WysiwygPlugin/lib/TWiki/Plugins/WysiwygPlugin.pm?old=16836&old_path=twiki%2Ftrunk%2FWysiwygPlugin%2Flib%2FTWiki%2FPlugins%2FWysiwygPlugin.pm beforeSaveHandler and TranslateHTML2TML are identical.

Eugen kindly gave me his changes in full:

Eugen's 4.1.2 with mods

sub beforeSaveHandler {
    #my( $text, $topic, $web ) = @_;
    my $query = TWiki::Func::getCgiQuery();
   
    return unless $query;#

    # if we also want to convert form fields, we have to convert $text anyway
    # because the afterEditHandler has only converted the main content, not meta fields
    # we put this check into tra

    if ( defined( $query->param( 'wysiwyg_edit' )) || $_[0] =~ /<!--$SECRET_ID-->/) {
    }
    my $text = $_[0];

    # extracting meta
    my $meta = '';
    if ($text =~ s/^(%META:[A-Z]+{.*?}%\r?\n)//s) {
        $meta = $meta.$1;
    }
    $text =~ s/^(%META:[A-Z]+{.*?}%\r?\n)/$meta = "$1$meta";''/gem;
    # text is now without meta information
    $_[0] = TranslateHTML2TML( $text, $_[1], $_[2], $meta);
}

sub TranslateHTML2TML {
    my( $text, $topic, $web , $meta) = @_; 
    
    unless( $html2tml ) {
        require TWiki::Plugins::WysiwygPlugin::HTML2TML;
        $html2tml = new TWiki::Plugins::WysiwygPlugin::HTML2TML();
    }

    # SMELL: really, really bad smell; bloody core should NOT pass text
    # with embedded meta to plugins! It is VERY BAD DESIGN!!!
    # TODO the extraction of meta to top and bottom should not be needed anymore, as its done externally
    # afterEditHandler anyway only passes main content without meta information, and afterSaveHandler NOW filters meta
    # himself and passes it as $meta. So this lines should be deleted
    my $top = '';
    if ($text =~ s/^(%META:[A-Z]+{.*?}%\r?\n)//s) {
        $top = $1;
    }
  
  my $bottom = '';
   $text =~ s/^(%META:[A-Z]+{.*?}%\r?\n)/$bottom = "$1$bottom";''/gem;
    
    my $opts = {
        web => $web,
        topic => $topic,
        convertImage => \&_convertImage,
        rewriteURL => \&postConvertURL,
        very_clean => 1, # aggressively polish saved HTML
    };

    #convert main content if secret id is present, so we have HTML code
    if ( $text =~ s/<!--$SECRET_ID-->//go) {
       $text = $html2tml->convert( $text, $opts );
    }
    
    # notice wen need to url_decode the form field value before converting, as it is url_decoded ( what differs to the main content )
    # and therefore the converter expect someting url_decoded
    # in the end we need to url_encode the value, as it has to be save url_endocded
    # traslate form fields which have been edited with WYSIWYG and are therefore html code
    # SMELL: maybe better regexp
    #unless (open _LOGGING,"> /var/www/gds/twiki/data/html2tml.log") {
    unless (open _LOGGING,"> /dev/null") {
        die "kann Ausgabedatei nicht öffnen";
    }
    
    print _LOGGING "---------META before-----\n\r";
    print _LOGGING "$meta\n\r";
    
    if ( $meta =~ s/<!--$SECRET_ID-->//go) {
       #if($meta =~ s/(^%META:FIELD{[^{]*.?value=")(<!--$SECRET_ID-->)(.*?)("})/$1.uri_escape($html2tml->convert( uri_unescape($3), $opts )).$4/gme) {
      # SMELL: iam not sure this is needed. It seems like secret_id should be alway in front. twiki_tiny_js is settings it to the front,
      # also the TML2HTML handler
      #$meta =~ s/(^%META:FIELD{[^{]*.?value=")(.*?)(<!--$SECRET_ID-->)("})/$1.uri_escape($html2tml->convert( uri_unescape($2), $opts )).$4/gme;
      $meta =~ s/(^%META:FIELD{[^{]*.?value=")(.*?)("})/$1.uri_escape($html2tml->convert( uri_unescape($2), $opts )).$3/gme;
      print _LOGGING "\n\r---------1-----\n\r";
      print _LOGGING "$1\n\r";
      print _LOGGING "---------2 converted----\n\r";
      print _LOGGING $html2tml->convert( uri_unescape($2), $opts )."\n\r";
      print _LOGGING "---------META after-----\n\r";
      print _LOGGING "$meta\n\r";
       #}
       close _LOGGING; 
    }
    $text =~ s/\s+$/\n/s;
    
    # Attention: $SECRET_ID is NEVER saved, as we never save HTML code. After the translation, we remove it.
    # It just indicates, wheather the current content is HTML or not
    return $text.$meta;
}

---

Eugen and I spoke by phone this afternoon. We now understand why TML is being shown in the WYSIWYG formfields: setUpContent (called from Preferences -> * Set TINYMCEPLUGIN_INIT =    setupcontent_callback : TWikiTiny.setUpContent): the browser loads with TML in all textareas and is responsible for making callbacks for each textarea that needs to show as HTML.

The current setupcontent_callback is shown next, and only initialises for the topic textarea.

    // Set up content for the initial edit
    setUpContent : function(editor_id, body, doc) {
        // If we haven't done it before, then transform from TML
        // to HTML. We need this test so that pressing the 'back'
        // button from a failed save doesn't banjax the old content.
        if (TWikiTiny.initialisedFromServer) return;
        var editor = tinyMCE.getInstanceById(editor_id);
        TWikiTiny.switchToWYSIWYG(editor);
        TWikiTiny.initialisedFromServer = true;
    },

http://wiki.moxiecode.com/index.php/TinyMCE:Configuration/setupcontent_callback has documentation. It actually states that setUpContent is deprecated, and replaced with a set of event bindings.

It appears that http://wiki.moxiecode.com/index.php/TinyMCE:API/tinymce.Editor/onInit is the one we would need.

---

TWiki stores it's content in wiki markup. When you edit in tinymce the TWiki TinyMCEPlugin invokes a translation process to turn the HTML that TinyMCE outputs into twikimarkup (TML).

It turns out that TinyMCEPlugin runs a bunch of assumptions. The most important of which is that there will only ever be one tinymce textarea on the page. TinyMCE It uses a REST interface to translate stored TML into HTML that tinymce can use, but the twiki_tiny.js code that calls this is not re-entrant. In the case where you want multiple wysiwyg form fields on the page all fields would need transforming from TML to HTML, and you'd likely want to transform them all at the same time during page load. Therefore the TinyMCEPlugin would need now to be reentrant to deal with requests for multiple fields during page load.

twiki_tiny.js uses a single class-level global variable for its EDITOR and REQUEST objects. In single request form this works fine. Where multiple requests happen the last request made wins, previous values are overwritten by later calls, even if that previous call has not finished.

Eugen points out that twiki_tiny.js makes life difficult for itself by hand-coding the REST/AJAX requests. It does not use a framework such as JQuery. I'll assume this was because TWiki did not standardise on a JS library at the point that TWiki tiny was written.

Going forward, we have two approaches we can take: 1 Adapt the existing code to hold a local variable in place of the global class-level ones. (The class makes use of anonymous functions for it's callbacks, so the context driving parameters would have to be constructed as closures). 2 Rewrite in something like JQuery.

Rewriting in jQuery would significantly reduce the amount of code needed and simplify. In turn it would likely make the code more robust. TWiki_tiny.js is 300 lines of code, of which perhaps 125 lines would disappear.

• MartinCleaver - 04 Jan 2009

Code candidate, uses closures instead of class level globals and is therefore re-entrant:

twiki/pub/TWiki/TinyMCEPlugin# cat twiki_tiny.js
/*
  Copyright (C) 2007 Crawford Currie http://wikiring.com and Arthur Clemens
  Copyright (C) 2009 Martin Cleaver http://www.blendedperspectives.com
  All Rights Reserved.

  This program is free software; you can redistribute it and/or
  modify it under the terms of the GNU General Public License
  as published by the Free Software Foundation; either version 2
  of the License, or (at your option) any later version. For
  more details read LICENSE in the root of the TWiki distribution.

  This program is distributed in the hope that it will be useful,
  but WITHOUT ANY WARRANTY; without even the implied warranty of
  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

  As per the GPL, removal of this notice is prohibited.
*/
/* 
2009-01-04: Martin Cleaver: made thread safe as part of Foswiki:Items8032

*/

/*
Notes:
   * This class would benefit from use of jQuery instead of raw HTTP calls. MC
*/

// The TWikiTiny class object
var TWikiTiny = {

    twikiVars : null,
    metaTags : null,

    tml2html : new Array(), // callbacks, attached in plugins
    html2tml : new Array(), // callbacks, attached in plugins

    // Get a TWiki variable from the set passed
    getTWikiVar : function (name) {
        if (TWikiTiny.twikiVars == null) {
            var sets = tinyMCE.getParam("twiki_vars", "");
            TWikiTiny.twikiVars = eval(sets);
        }
        return TWikiTiny.twikiVars[name];
    },

    expandVariables : function(url) {
        for (var i in TWikiTiny.twikiVars) {
            url = url.replace('%' + i + '%', TWikiTiny.twikiVars[i], 'g');
        }
        return url;
    },

    enableSave: function(enabled) {
        var status = enabled ? null : "disabled";
        var elm = document.getElementById("save");
        if (elm) {
            elm.disabled = status;
        }
        elm = document.getElementById("preview");
        if (elm) {
            elm.style.display = 'none'; // Item5263: broken preview
            elm.disabled = status;
        }
    },


    reqObject: function(method, text) {
        // Work out the REST URL+PATH from the handler
        var url = TWikiTiny.getTWikiVar("SCRIPTURL");
        var suffix = TWikiTiny.getTWikiVar("SCRIPTSUFFIX");
        if (suffix == null) suffix = '';

        url += "/rest" + suffix + "/WysiwygPlugin/" + method;
        var path = TWikiTiny.getTWikiVar("WEB") + '.' + TWikiTiny.getTWikiVar("TOPIC");

   // Set up the object
   var req = null;
        if (tinyMCE.isIE) {
            // branch for IE/Windows ActiveX version
            req = new ActiveXObject("Microsoft.XMLHTTP");
        } else {
            // branch for native XMLHttpRequest object
            req = new XMLHttpRequest();
        }

        req.open("POST", url, true);
        req.setRequestHeader(
            "Content-type", "application/x-www-form-urlencoded");
        var params = "nocache=" + encodeURIComponent((new Date()).getTime())
        + "&topic=" + encodeURIComponent(path)
        + "&text=" + encodeURIComponent(text);
    
        req.setRequestHeader(
            "Content-length", params.length);
        req.setRequestHeader("Connection", "close");
   return {req: req, params: params};
    },

    transform2: function(closure, method, inputMarkup, onReadyToSend, onReply, onFail) {

   // TODO: ASSERT closure is not null

//   alert("Transforming "+closure.editor_id + " with method "+method);

        closure.request = new Object();
   var reqAndParams =  TWikiTiny.reqObject(method, inputMarkup);

   closure.request.req = reqAndParams.req;
   var params = reqAndParams.params;

        closure.request.req.onreadystatechange = function() {
            // Callback for XMLHttpRequest
            // only if request.req shows "complete"
            if (closure.request.req.readyState == 4) {
                // only if "OK"
                if (closure.request.req.status == 200) {
   //   alert("Reply for "+closure.editor_id);
                    onReply();
                } else {
                    onFail();
                }
            }
        };
        onReadyToSend();
        closure.request.req.send(params);
    },


    initialisedFromServer : new Array(),

    // Set up content for the initial edit
    setUpContent : function(editor_id, body, doc) {
        // If we haven't done it before, then transform from TML
        // to HTML. We need this test so that pressing the 'back'
        // button from a failed save doesn't banjax the old content.
        if (TWikiTiny.initialisedFromServer[editor_id]) {return} 
        var editor = tinyMCE.getInstanceById(editor_id);
        TWikiTiny.switchToWYSIWYG(editor);
        TWikiTiny.initialisedFromServer[editor_id] = true;
    },

    // Convert HTML content to textarea. Called from the WYSIWYG->raw switch
    switchToRaw : function (editor) {
        // As shown by OliverKrueger in Item5138, trivial text may include
        // UTF-8 chars. These need to be encoded to entities before we can
        // pass the string back to the server. This is done in triggerSave,
        // but note that it requires cleanup:true to work.
        editor.triggerSave(false, true);
        var text = editor.oldTargetElement.value;

   var closure = new Object;  // closure needed to persist the editor and request context for 
   closure.editor = editor;
   closure.editor_id = editor.editorId;
   closure.request = null;


        // Evaluate post-processors
        for (var i = 0; i < TWikiTiny.html2tml.length; i++) {
            var cb = TWikiTiny.html2tml[i];
            text = cb.apply(editor, [ editor, text ]);
        }
        TWikiTiny.transform2(
            closure, "html2tml", text,
            function () {
                // onReadyToSend
                TWikiTiny.enableSave(false);
                var te = closure.editor.oldTargetElement;
                te.value = "Please wait... retrieving page from server";
            },
            function () {
                var te = closure.editor.oldTargetElement;
                var text = closure.request.req.responseText;
                te.value = text;
                TWikiTiny.enableSave(true);
            },
            function () {
                var te = closure.editor.oldTargetElement;
                te.value = "There was a problem retrieving the page: "
                    + closure.request.req.statusText;
                //TWikiTiny.enableSave(true); leave it disabled
            });

   TWikiTiny.conversionButton("show", editor);

        // SMELL: what if there is already an onchange handler?
        editor.oldTargetElement.onchange = function() {
            var editor = tinyMCE.getEditoranceById(eid);
            editor.isNotDirty = false;
            return true;
        }
    },

    // Convert textarea content to HTML. This is invoked from the content
    // setup handler, and also from the raw->WYSIWYG switch
    switchToWYSIWYG : function (editor) {
   var closure = new Object;  // needed for the closure
   closure.editor = editor;
   closure.editor_id = IFRAME_ID;
   closure.request = null;
        // Kill the change handler to avoid excess fires
        editor.oldTargetElement.onchange = null;
        // Need to tinyMCE.execCommand("mceToggleEditor", null, editor_id);
        TWikiTiny.transform2(
            closure, "tml2html", editor.oldTargetElement.value,
            function () {
                // onReadyToSend
                TWikiTiny.enableSave(false);

                tinyMCE.setInnerHTML(
                    closure.editor.getBody(),
                    "<span class='twikiAlert'>Please wait... retrieving page from server.</span>");
            },
            function () {
                // Handle the reply
                var text = closure.request.req.responseText;

                // Evaluate any registered pre-processors
                for (var i = 0; i < TWikiTiny.tml2html.length; i++) {
                    var cb = TWikiTiny.tml2html[i];
                    text = cb.apply(editor, [ editor, text ]);
                }
                tinyMCE.setInnerHTML(closure.editor.getBody(), text);
                closure.editor.isNotDirty = true;
                TWikiTiny.enableSave(true);
            },
            function () {
                // Handle a failure
                tinyMCE.setInnerHTML(
                    closure.editor.getBody(),
                    "<div class='twikiAlert'>"
                    + "There was a problem retrieving the page: "
                    + closure.request.req.statusText + "</div>");
                //TWikiTiny.enableSave(true); leave save disabled
            });

   TWikiTiny.conversionButton("hide", editor);
    },

   conversionButton: function(showOrHide, editor) {
        // Show or Hide the button for the switch back to WYSIWYG mode
        var eid = editor.editorId;
        var id = eid + "_2WYSIWYG";
        var el = document.getElementById(id);

   if (showOrHide == "hide") {
           // Hide the conversion button, if it exists
           if (el) {   
               // exists, hide it
               el.style.display = "none";
           }
        } {
           if (el) {
               // exists, unhide it
               el.style.display = "block";
           } else {
               // does not exist, create it
               el = document.createElement('INPUT');
               el.id = id;
               el.type = "button";
               el.value = "WYSIWYG";
               el.className = "twikiButton";
               el.onclick = function () {
                   tinyMCE.execCommand("mceToggleEditor", null, editor.editorId);
                   return false;
               }
      // SMELL: does this need placing at XX1
               // Need to insert after to avoid knackering 'back'
               var pel = editor.oldTargetElement.parentNode;
               pel.insertBefore(el, editor.oldTargetElement);
           }
   }
   // XX1 SMELL ref.
    },

    // Callback on save. Make sure the WYSIWYG flag ID is there.
    saveCallback : function(editor_id, html, body) {
        // Evaluate any registered post-processors
        var editor = tinyMCE.getInstanceById(editor_id);
        for (var i = 0; i < TWikiTiny.html2tml.length; i++) {
            var cb = TWikiTiny.html2tml[i];
            html = cb.apply(editor, [ editor, html ]);
        }
        var secret_id = tinyMCE.getParam('twiki_secret_id');
        if (secret_id != null && html.indexOf(
                '<!--' + secret_id + '-->') == -1) {
            // Something ate the ID. Probably IE. Add it back.
            html = '<!--' + secret_id + '-->' + html;
        }
        return html;
    },

    // Called 
    // Called on URL insertion, but not on image sources. Expand TWiki
    // variables in the url.
    convertLink : function(url, node, onSave){
        if(onSave == null)
            onSave = false;
        var orig = url;
        var pubUrl = TWikiTiny.getTWikiVar("PUBURL");
        var vsu = TWikiTiny.getTWikiVar("VIEWSCRIPTURL");
        url = TWikiTiny.expandVariables(url);
        if (onSave) {
            if ((url.indexOf(pubUrl + '/') != 0) &&
                (url.indexOf(vsu + '/') == 0)) {
                url = url.substr(vsu.length + 1);
                url = url.replace(/\/+/g, '.');
                if (url.indexOf(TWikiTiny.getTWikiVar('WEB') + '.') == 0) {
                    url = url.substr(TWikiTiny.getTWikiVar('WEB').length + 1);
                }
            }
        } else {
            if (url.indexOf('/') == -1) {
                // if it's a wikiword, make a suitable link
                var match = /^((?:\w+\.)*)(\w+)$/.exec(url);
                if (match != null) {
                    var web = match[1];
                    var topic = match[2];
                    if (web == null || web.length == 0) {
                        web = TWikiTiny.getTWikiVar("WEB");
                    }
                    web = web.replace(/\.+/g, '/');
                    web = web.replace(/\/+$/, '');
                    url = vsu + '/' + web + '/' + topic;
                }
            }
        }
        return url;
    },

    // Called from Insert Image, when the image is inserted. The resultant
    // URL is only used when displaying the image in the picture dialog. It
    // is thrown away (reverts to the typed address) when the image is
    // actually inserted, at which time convertLink is called.
    convertPubURL : function(url){
        var orig = url;
        var base = TWikiTiny.getTWikiVar("PUBURL") + '/'
        + TWikiTiny.getTWikiVar("WEB") + '/'
        + TWikiTiny.getTWikiVar("TOPIC") + '/';
        url = TWikiTiny.expandVariables(url);
        if (url.indexOf('/') == -1) {
            url = base + url;
        }
        return url;
    },

    getMetaTag : function(inKey) {
        if (TWikiTiny.metaTags == null || TWikiTiny.metaTags.length == 0) {
            // Do this the brute-force way because of the Firefox problem
            // seen sporadically on Bugs where the DOM appears complete, but
            // the META tags are not all found by getElementsByTagName
            var head = document.getElementsByTagName("META");
            head = head[0].parentNode.childNodes;
            TWikiTiny.metaTags = new Array();
            for (var i = 0; i < head.length; i++) {
                if (head[i].tagName != null &&
                    head[i].tagName.toUpperCase() == 'META') {
                    TWikiTiny.metaTags[head[i].name] = head[i].content;
                }
            }
        }
        return TWikiTiny.metaTags[inKey]; 
    },
    
    install : function() {
        // find the TINYMCEPLUGIN_INIT META
        var tmce_init = TWikiTiny.getMetaTag('TINYMCEPLUGIN_INIT');
        if (tmce_init != null) {
            eval("tinyMCE.init({" + unescape(tmce_init) + "});");
            return;
        }
        alert("Unable to install TinyMCE; <META name='TINYMCEPLUGIN_INIT' is missing"); 
    }
};

---

Strange behaviour

This is a list of items:
   1 One
   1 Two
   1 Three
   1 Four

And a table:
| A B | B |
| C | C |

Is translated as:

This is a list of items: <ol><br /><li> One</li> <li> Two</li> <li> Three</li> <li> Four</li></ol> <br /><p>&nbsp;</p><br />And a table:<br /><br /><br /><table border="1" cellspacing="0" cellpadding="0" class="twikiTable" id="table1"><br />   <tbody><br />      <tr class="twikiTableOdd twikiTableRowdataBgSorted0 twikiTableRowdataBg0"><br />   <td class="twikiTableCol0 twikiFirstCol" valign="top" bgcolor="#ffffff"> A B </td><br />         <td class="twikiTableCol1 twikiLastCol" valign="top" bgcolor="#ffffff"> B </td><br />      </tr><br />      <tr class="twikiTableEven twikiTableRowdataBgSorted1 twikiTableRowdataBg1"><br />      <td class="twikiTableCol0 twikiFirstCol twikiLast" valign="top" bgcolor="#edf4f9"> C </td><br />         <td class="twikiTableCol1 twikiLastCol twikiLast" valign="top" bgcolor="#edf4f9"> C </td><br />      </tr><br />   </tbody></table>  

i.e.

This is a list of items:

1. One
2. Two
3. Three
4. Four

 

And a table:

A B	B
C	C

Firefox 3.0.5 on a mac renders this as:

This is a list of items: <ol><br /><li> One<br /></li> <li> Two<br /></li> <li> Three<br /></li> <li> Four<br /></li></ol> <br /><p /><br />And a table:<br /><nop><br /><nop><br /><table cellspacing="0" id="table1" cellpadding="0" class="twikiTable" rules="rows" border="1"><br />   <tbody><br />      <tr class="twikiTableOdd twikiTableRowdataBgSorted0 twikiTableRowdataBg0"><br />   <td bgcolor="#ffffff" valign="top" class="twikiTableCol0 twikiFirstCol"> A B </td><br />         <td bgcolor="#ffffff" valign="top" class="twikiTableCol1 twikiLastCol"> B </td><br />      </tr><br />      <tr class="twikiTableEven twikiTableRowdataBgSorted1 twikiTableRowdataBg1"><br />      <td bgcolor="#edf4f9" valign="top" class="twikiTableCol0 twikiFirstCol twikiLast"> C </td><br />         <td bgcolor="#edf4f9" valign="top" class="twikiTableCol1 twikiLastCol twikiLast"> C </td><br />      </tr><br />   </tbody></table>

i.e. This is a list of items:

1. One
   
2. Two
   
3. Three
   
4. Four
   

And a table:

A B	B
C	C

---

Suggest this is incorporated into the work of the WysiwygTaskTeam - you would be welcome as a member, Martin!

-- CrawfordCurrie - 21 Jan 2009

Martin: Is the attachment here still your latest work? Before I embark too far on TinyMCEPluginShouldBeMoreModular I should evaluate what you've done here so I can be sure that I don't get in the way.

-- PaulHarvey - 21 Oct 2009

Sorry, I wasn't looking at Foswiki for a few months.

Yes, this is the latest version.

-- MartinCleaver - 03 Feb 2010

Hi Martin,

Thanks for the update. I believe there are two major parts for WYSIWYG on formfields:

• Addressing the technical issues detailed very thoroughly in Tasks.Item8032.
  • Newline encoding of formfields messes up rendering of TML lists/tables.
  • Other encoding issues.
  • Javascript isn't reentrant.
  • WysiwygPlugin doesn't translate formfields.
• Fundamental mechanisms for WYSIWYG are lacking
  • Not modular enough. Need to more easily accommodate special tml2html/html2tml requirements of multiple editors.
  • Not enough control of strictness/permitted content in the html2tml translation. Should be easy to specify, Eg. never allow HTML tables.
  • Javascript (non-TMCE specific ajax parts) should belong to WysiwygPlugin.
  • Javascript needs a re-write in jQuery.

    • $('textarea.foswikiEdit').foswikiwysiwyg({editor : 'TinyMCEPlugin', ...})

  • Need more communication with translator from client. More HTTP headers? Translator needs to know:
    • Editor used
    • TML policy
    • These two things may be changed by the user while editing
  • Need end-to-end testing via Selenium.

Hopefully the technical issues can be resolved in time for 1.1, but they may not be. I would be very happy if we can WYSIWYG on formfields shipped with 1.1.

The secondary issues - re-arranging Wysiwyg API - is beyond 1.1 IMHO.

-- PaulHarvey - 20 Feb 2010

This would be really awesome for new tasks on foswiki.org.

-- JayenAshar - 27 Oct 2011

Congratulations SvenDowideit, for fixing this 3.5 year old feature request (setting to WaitingFor SvenDowideit)

-- PaulHarvey - 18 Jun 2012 - 07:24

ok, feature req accepted by 14dayrule: WysiwygFormFields

-- SvenDowideit - 20 Jun 2012

just to remind me - I still need to extract this from the plugin i made, and put it into WysiwygPlugin - and in the process abstract the triggering of a wysiwyg edit from the implementation (tinyMCE, cke, whatever)

-- SvenDowideit - 25 Jun 2012

Deferring this feature to 2.0. There haven't been checkins on this feature for 2 years, though it doesn't seem to be finished. Most work is done on a non-crore plugin WysiwygFormfieldPlugin. Best would be to continue this feature to mature in its scope before adding it to the core.

-- MichaelDaum - 02 Jun 2014

This is not, and never has been, a function of WysiwygPlugin. It's not even an issue of TinyMCEPlugin. It's a core issue, (or maybe JQueryPlugin at a stretch)

Changed to core.

-- Main.CrawfordCurrie - 28 Mar 2017 - 12:50

Commenting is disabled while not logged in