Item9973: TinyMCE attach dialogue lists no attachments for topics named with international characters

Actually, there are two problems that are probably connected to each other. All appear in topics with Cyrillic names, no matter what locale and encoding (of course, Cyrillic) I choose.

1. When I attach an image to a topic and then try to edit it using WYSIWYG editor, the link to an image breaks. The raw text (inside the IMG tag) after saving with WYSIWYG looks like this:

%PUBURLPATH%/Sandbox/\x{00bf}\x{00e0}\x{00de}\x{00d1}\x{00dd}\x{00d0}\x{00ef}\x{00c2}\x{00d5}\x{00dc}\x{00d0}

The numbers look like Unicode codes but with wrong format, so the browser could not recognize them.

2. When I try to attach file inside the WYSIWYG editor using Manage Attachments dialog (Upload Attachment tab), I've got "Access denied" error message. It appears to me that the plugin tries to get access to a folder in file system using wrong folder name, probably using invalid characters. When I do the same with topics, which names do not contain international characters, everything works just fine.

I would be grateful for any input and help with this problem, because it really prevents using the full power of Foswiki in non-English communities.

-- YuryGolev - 08 Nov 2010

Verified.

In the Wysiwyg editor attach dialog you cannot even attach a file with a simple umlaut character. Mr A-Z has struck again.

-- KennethLavrsen - 09 Nov 2010

I have the same problem using Greek language, and the solution is to use UTF-8 encoding and make TinyMCE NOT TO USE entity encoding (entity_encoding = raw;). This used to be a preference setting inside tinymce.pm, but i don't know where to put it in FOSWIKI version 1.1.2

-- StefanosKouzof - 17 Nov 2010

Sadly, even UTF-8 does not help. If I use Cyrillic characters in the topic name, all is pretty the same. Manage Attachments dialog in TinyMCE cannot see attachments, which were uploaded in a normal way. Also, if I attach files inside TinyMCE, they get attached to another (non-existing) topic.

-- YuryGolev - 17 Nov 2010

It's TinyMCEPlugin at fault. If you manually craft a request to the WysiwygPlugin handler, it does the right thing.

For some reason the topic name is inappropriately encoded or escaped.. anyway, mangled by the TinyMCEPlugin Javascript. Changing affected component to TinyMCEPlugin.

-- PaulHarvey - 18 Nov 2010

The TINYMCEPLUGIN_INIT which builds a JSON-looking thing gets URL encoded. Why, I'm not sure. I guess so it could be stuffed into a <meta tag.

I'm ripping out the <meta tag madness and replacing with inline <script which will add the JSON to directly. This way we avoid unnecessary double handling/encoding issues.

I'm making the assumption that our topic and attachment names will be strings that contain only valid characters set for our {SiteEncoding}, and that Foswiki will deliver HTML pages to the browser with a matching Content-Encoding header. Then the JSON string should be free to contain whatever entities exist on the foswiki side.

-- PaulHarvey - 18 Nov 2010

I have committed a fix, but I am ignorant ascii/english user Please try the new version at Extensions/Testing.TinyMCEPlugin. I have tested with UTF-8 which seems to work now.

I have added a new preference setting, TINYMCEPLUGIN_ENTITY_ENCODING. Now defaults to 'numeric', which think should be good enough for everybody. But for those people who need 'raw', you can set it much easier now.

Should also close Tasks.Item8085

-- PaulHarvey - 22 Nov 2010

Thank you very much, Paul! Your efforts are very much appreciated. You have done a great job. I've just tested the fixed plugin. Manage Attachments dialog works just fine. Amazing! But it still cannot see uploaded attachments, if I set SiteEncoding to CP1251. The good new: with UTF-8 site encoding it seems like working fine.

-- YuryGolev - 22 Nov 2010

Working with images is still a headache. I try the CP1251 encoding. After the second editing TinyMCE does not see them, just shows small boxes. Worse thing is that you better not to edit images, otherwise they disappear completely. But I am happy anyway! At last we have WYSIWYG, which (if used with caution) can be used to edit international topics with images. Thank you again, Paul! I are doing a great job. God bless you!

-- YuryGolev - 22 Nov 2010

Okay, in hindsight, it's obvious that my fix will only work for UTF8. I will have to revert and fix the escaping issue properly.

-- PaulHarvey - 22 Nov 2010

Actually, I forgot to remove some of the old unescape code in js. I will make a new fix.

-- PaulHarvey - 22 Nov 2010

WysiwygPlugin also seems to have a problem with non-ASCII topic names. It does not revert a path within IMG tag back to Foswiki variables, such as ATTACHURLPATH. The result -- non-ASCII characters in URLs, and you need to set up browser to understand such paths. Is there a cure for this?

-- YuryGolev - 24 Nov 2010

Yury, is this after my changes, or before, both? Please revert to the official released TinyMCEPlugin at Extensions.TinyMCEPlugin.

I'll try to see if I can see what's up with URLs

-- PaulHarvey - 24 Nov 2010

No, Paul, it was even before your change.

-- YuryGolev - 24 Nov 2010

Hello Yury,

I finally figured out a very simple fix in WysiwygPlugin for listing attachments on topics named with international characters from cp1251 or utf8 encoding.

Please test Extensions/Testing.WysiwygPlugin

-- PaulHarvey - 26 Nov 2010

Hello again Yury,

I have uploaded yet another version of Extensions/Testing.WysiwygPlugin which incorporates distro:502bbb79121c - I hope this fixes the failure to convert URLs of attachments on topics named with cp1251 characters to %ATTACHURL% macros.

Added a cp1251 test, but we probably need some utf8 ones.

-- PaulHarvey - 27 Nov 2010

Hello Paul!

Thank you for your work. A great job! Manage Attachments dialog works perfectly. But there are still some problems.

1. TinyMCE does not show images, only boxes with broken links. HTML source looks like this:

 /foswiki/pub/Sandbox/ПерваяСсылка/Smiley-neutral.gif 

But when you save topic, browser shows them. These codes are good for browser, but are not valid URL, I suppose.

2. If you copy an image (I mean a box representing an image) and then paste it somewhere in the same topic, this pasted image breaks completely. Even the browser does not show it.

-- YuryGolev - 27 Nov 2010

Oops, confirmed. URLs need to be escaped with URI escaping pattern, but the rest of the topic text needs to stay as it is (I think)... hmm, this could be sticky.

-- PaulHarvey - 27 Nov 2010

It would be nice also to take into accout URLs, which are in square brackets.

-- YuryGolev - 27 Nov 2010

Hi Yury, thank you for your testing and prompt replies. Could you please tell me if the current trunk behaviour is better or worse than 1.1.2. I don't have time this month to make further changes, but maybe after 1.1.3 is released. What do you think? Should we release 1.1.3 as-is? I've actually already merged the code over...

-- PaulHarvey - 06 Jan 2011

Currently the plugin's behavior is better. So, I think we can release 1.1.3 and continue to improve it.

-- YuryGolev - 06 Jan 2011

Thanks. Continued in Item10230

-- PaulHarvey - 06 Jan 2011

Commenting is disabled while not logged in