Feature Proposal: Why wouldn't Foswiki support Creole markup ?

Motivation

Today, there are many more competing wiki engines than previously. Creole is an attempt to establish a common way of easy markup between wikis. In addition, many wikis explored the capability of syndication. Hence support for Creole in Foswiki should be a move in the "right direction"

Note that there has been a Feature Freezing announced on 6th april, but I could not find a clear list of what's in or not for the next release. Probably this is not an issue since this would be a plugin.

I moved the "marketing" stuff here to the WikiMatrixTalk page.Some more to move below.

Description and Documentation

Extend the markup capabilities of FosWiki in order to support the Wiki Lingua Franca : Creole.

Examples

Taken from the MediaWiki entry:

Emphasized text:

//emphasized// (e.g., italics)

**strongly emphasized** (e.g., bold)

Lists:

* Bullet list
* Second item
** Sub item

# Numbered list
# Second item
## Sub item

Links:

Link to [[wikipage]]
[[link_address|link text]]

Headings: (closing equals signs are optional)

= Extra-large heading
== Large heading
=== Medium heading
==== Small heading

Linebreaks:

Force\\linebreak

Horizontal Line:

----

Images:

{{Image.jpg|title}}

Tables:

|= |= table |= header |



| 
 a 
 | 
 table 
 | 
 row 
 |
| 
 b 
 | 
 table 
 | 
 row 
 |

No markup:

{{{
This text will //not// be **formatted**.
}}}

Impact

WhatDoesItAffect:

%WHATDOESITAFFECT%

edit

Implementation

-- Contributors: ChristopheVermeulen - 28 Apr 2010

Discussion

A couple of comments (refactored, non-Creole stuff moved) :

1. WikiCreole is a lowest-common-denominator that works well, i guess, for people who move between wiki implementations using their individual markups. While support for Creole would be a nice-to-have, it's not something people are clamouring for (AFAIK).
2. We had a Wikipedia article, but some time ago a number of Wikipedians voted to delete it saying that Foswiki "wasn't notable enough". For it to become notable enough for re-inclusion, there have to be independently verifiable sources that mention it e.g. press reports, articles, papers etc. Until those sources are available, the page will simply get deleted again.

-- CrawfordCurrie - 28 Apr 2010

Creole support shouldn't be too hard to add as a plugin as far as it is concerned with wiki markup (not applications including templates and macro parsing). Note, that there are plugins to import data from other wiki engines like mediawiki to foswiki as well as add mediawiki table support.

-- MichaelDaum - 28 Apr 2010

It's worth noting that while rendering creole via plugin would not be hard, making it work with the WYSIWYG editor would be another can of worms.

-- PaulHarvey - 28 Apr 2010

Glad to see so many quick responses. Indeed a good proof that the community is very active. My comment on support is that for the newcomer, everything that is visible (on the Home Web) is the article on the "feature freeze on 6th April", but nowhere a list of those features. It is only when you delve into the Development web that you see there are so many tasks, discussions and code changes, as well as the release plan. I hope you don't take offense if I (try to) include a link from there ? Then I'll refactor this discussion, as it has little to do with the Creole topic... Regarding the "notability" of FosWiki, this is probably again a marketing view, but don't you want as a community to have many users that make use of your work ? I think this is worth some effort.

A final note to MichaelDaum, ReasonsToUseFoswiki and ReasonsNotToUseTWiki are probably two good examples of topics that do not serve FosWiki very well. OK, the statistics show there are much more software commits in FosWiki. But this reinforces the idea that the FosWiki community just want the "old Twiki" users to migrate to punish Peter from his coup, more than trying to care for the FosWiki users' wishes. Maybe (I hope I don't get flamed for saying this Peter understands it better when it compares Twiki to MediaWiki...

So, back to the subject : assume that I'd like to make a plugin for Creole, where should I start ? (My PERL is a bit rusty, dates from when I contributed a bit to Twiki ... 12 years ago)

-- ChristopheVermeulen - 30 Apr 2010

Lack of a marketing focus has always been a problem, so your remarks, far from attracting flames, are very welcome. Peter always was very good at painting a rosy picture of TWiki, and we miss having someone with those skills. I don't liked "TWiki baiting", which is why I have never indulged in it myself.

As I understand it, notability isn't something we can control. It has to be someone from outside the community who takes note; or so the deletion discussion on Wikipedia led me to believe.

Christophe, anything you can do to help make our marketing more attractive to new users, please just do it. This is a democratic community, and the worst that can happen is that someone disagrees and changes it again. If they just make noises and complain without actually doing something better, then just ignore them.

-- CrawfordCurrie - 30 Apr 2010

A plugin that interprets Creole markup when a page is rendered may increase the learning curve for your wiki, rather than decrease it.

I suspect that your wiki's users are not homogenous (and even if they are now, then I suspect they will not always be homogenous), and so some will prefer the WYSIWYG editor, some will prefer editing Foswiki's TML (perhaps with assistance from NatEditPlugin) and others will prefer editing Creole Markup. If some write content in Creole and others write content in TML (either directly or via the WYSIWYG editor), then all users may end up having to learn both sets of markup. This is especially likely when users collaborate (taking into account that the WYSIWYG editor is unlikely to support Creole markup).

I would like to suggest another way to support Creole markup: a "Creole markup editor". A "Creole markup editor" would be a plain <textarea>, just like Foswiki's "raw edit". Foswiki-style TML markup would be translated to Creole markup when the user clicks the "edit creole markup" button, and the Creole markup would be translated back to Foswiki's TML when the user clicks "save".

This approach has several advantages:

• The underlying wiki only has one markup language: TML. This makes the wiki "future compatible". The wiki may be upgraded, or the Creole markup editor plugin could be disabled, or plugins with hidden assumptions might be installed, all without breaking any content.
• Different users may collaborate on the same content with different editors (raw, WYSIWYG and creole), and in principle this would not increase the learning curve.
• This technology could also form the basis for a "Creole import/export" facility.

I suspect it would be harder to implement a "Creole editor" correctly than a plugin that simply renders Creole markup. A "Creole editor" would have the potential to break existing topics, just like the WYSIWYG editor unfortunately sometimes does, and so the editor should disable itself (and revert to Foswiki's TML editor) if converting the topic to Creole and back changes the topic content.

The conversions may have to be rather complex. New content with Creole markup (typed by the user, i.e. not translated from TML) might combine elements of Foswiki and Creole markup. For example, //one_ two _three// should be rendered as one_ two _three but a simple translation of Creole markup's // to TML's _ would produce one two three. To avoid that effect, the plugin could convert the inner 's to _, i.e. producing _one two three, or else use <literal>.

A "Creole editor" might be more work (and more difficult work) than a "Creole renderer", but I think the "Creole editor" is better, from an architectural perspective.

If the point of supporting Creole markup is simply to lower the bar for new users who already know Creole markup and want to use a "text editor" rather than a "WYSIWYG editor", then consider NatEditPlugin - that plugin might just provide sufficient help and support for those users.

-- MichaelTempest - 30 Apr 2010

Conversion may at best only be complex. Most probably it is impossible as two input languages don't propperly map onto our store format, which is TML. Unfortunately, TML is not powerful enough to cover all requirements as a store format.

That's why our current WYSIWYG editor breaks things so often (besides tinymce being plain broken) and why any other translation back and forth will most probably be a lossy process.

Instead, I'd suggest to add a flag to a topic's meta data to define what kind of content it holds: xml, html, tml, creol, ... and then use the matching renderer and editor.

On the above marketing related comments. I fully subscribe to what you say, Christophe. Well, it is not easy to come to a louder more positive marketing message. Still, things aren't hootchie-cootchie as the other project keeps mocking around with trandemarks and copyright again right now. We will try harder to ignore them, but our history is no easy burdon.

-- MichaelDaum - 01 May 2010

Thanks for the support.

First about the Creole stuff. While I may be(come) fit (again) to create a Creole plugin, since I tweaked the JosWiki engine I used then quite intensively, the making of a Creole Markup Editor is definitely beyond my reach. To date, I don't know at all how the plugin structure works, I've forgotten 80% of my Perl, and I have no clue on which instance of Foswiki I could possibly test that plugin.

That said, I do not agree on the aspects of learning curve and more complexity if Creole was supported : The Wysiwyg editor right now does not really support CamelCase, nor Web.Topic links in a very consistent way anyway, or so I feel. So why would you expect its support of Creole to be so important.

-- ChristopheVermeulen - 04 May 2010

There is a vast amount of work in WysiwygPlugin already to convert TML into HTML and back again, so that WYSIYWG editors will play with Foswiki topics.

Otherwise it'd be pure HTML in topics.

Links are a pain point, but the fact it's possible to use a HTML editor on a TML topic and (potentially) not have a single HTML tag end up in the edited topic - that is an achievement.

The magic I'm talking about converts *bolded*, _italicised_ text into <strong>bolded</strong>, <em>italicised</em> text, and back again, for example.

-- PaulHarvey - 05 May 2010

I think I understand a bit more now. Sorry, but my knowledge of (T)wiki is more than 10 years old, actually I made most changes in a JosWiki with just CGI.pm and the set of view.cgi etc. So the actual structure with objects packages, etc. goes beyond my understanding until now.

So to recap, there is a TopicMarkupLanguage derived from the historical JosWiki (with the three spaces and --- headers), a convertor between that and HTML (back and forth for the TinyMCE editor that needs HTML input), so using Creole input in it would work the first time, but at the second edition it may be either backconverted to TML, or wrongly translated to HTML, etc. I think this is not such a big issue for a start though. My idea was to be more user-friendly in the sense that alternative markups woud be rendered, and not stay as [|] or # characters.

Therefore, I'd consider first making a markup plugin that simply translates the Creole stuff above to HTML at rendering , and not consider at all the Wysiwyg issues. Getting that working is already a challenge...

-- ChristopheVermeulen - 05 May 2010