Release Meeting: 05 Sep 2016

(08:33:44 AM) JulianLevens entered the room.
(09:01:07 AM) gac410: Hi everyone
(09:02:00 AM) JulianLevens: hi
(09:02:02 AM) gac410: Everyone ready for the release meeting?
(09:02:25 AM) JulianLevens: Well I think my fingers are
(09:02:55 AM) vrurg: Hi
(09:03:31 AM) gac410: Hi Vadim,   Hi Julian 
(09:04:35 AM) JulianLevens: Hi Vadim
(09:04:41 AM) gac410: First up on the agenda -   New tasks / Release blockers
(09:06:33 AM) gac410: We've had a few tasks recently that were more support requests than bugs.  https://foswiki.org/Tasks/Item14126   and https://foswiki.org/Tasks/Item14142
(09:07:34 AM) gac410: We found a bug in caching and strikeone - https://foswiki.org/Tasks/Item14150    that's probably ready for release. 
(09:08:00 AM) gac410: (The strikeone keys are cached and not regenerated,  even in dirty areas)
(09:09:06 AM) gac410: And there's an upstream bug in TinyMCE that causes a client loop on certain malformed html (ie TML) in the source.
(09:09:44 AM) gac410: MichaelDaum found that one  but no fixes. :(    He also fixed the caching issue.  
(09:14:07 AM) JulianLevens: I cannot comment on strikeone - its a mystery to me
(09:14:21 AM) gac410: MichaelDaum:   Are you there?    Did you ever report the TMCE loop upstream?
(09:14:51 AM) gac410: JulianLevens:  yeah.  Michael and I pretty much proved that one out.  He figured out the fix,  I had a recreation.
(09:15:25 AM) JulianLevens: The other 'Support Requests' become bugs in docu, don't they
(09:16:03 AM) gac410: Not sure.   Some seem to be just side effects of old plugins, or misunderstandings.
(09:17:25 AM) MichaelDaum: Hi there. Sorry for being late.
(09:17:38 AM) gac410: No prob....   At least you're here  ;)
(09:18:37 AM) MichaelDaum: nope. I did not have any feedback on the upstream bug. will try again.
(09:19:09 AM) MichaelDaum: the more I think about it the less convinced I am that tinymce is a good strategy for wysiwyg long term
(09:19:48 AM) gac410: The whole concept of TML -> HTML -> TML is a bit shaky at best
(09:20:23 AM) MichaelDaum: jast, did you ever use ckedit in a TML -> HTML -> TML loop ... using Wysiwyg in Qwiki?
(09:20:30 AM) JulianLevens: agreed
(09:20:35 AM) MichaelDaum: ^WysiwygPlugin^
(09:20:59 AM) gac410: ping jast
(09:21:31 AM) MichaelDaum: I am also fighting with understanding the internal api of wysiwyg to do simple stuff like inserting a h1 n the like. docu looks good at first sight but then falls short of such basics.
(09:21:41 AM) gac410: (Jast might be another lurker like Lavr   He's present here all the time from what I can tell.)
(09:21:50 AM) MichaelDaum: ^internal api of tinymce that is
(09:22:43 AM) JulianLevens: Isn't there an alternative that could handle something like TML
(09:23:31 AM) MichaelDaum: I've got codemirror working as a hightlighting code editor
(09:24:17 AM) JulianLevens: Wasn't there a sister project for text editing
(09:24:21 AM) MichaelDaum: it highlights tml typography and macros similar to chili ... while you edit it, as well as code folding as headings ... nice but not a wysiwyg of course
(09:24:23 AM) CDot [~crawford@foswiki/developer/cdot] entered the room.
(09:24:35 AM) CDot: WysiwygPlugin was designed to be entirely independent of TinyMCE
(09:24:45 AM) CDot: there may have been some hacks to massage the HTML so it works better for TMCE - gac410 did some work there
(09:24:56 AM) CDot: but it should be pretty much stand-alone
(09:25:36 AM) gac410: I think jast did have a CKEditor version  ... but IIRC the integration was pretty complex because of the differences in how the editors generate html.
(09:26:58 AM) MichaelDaum: some feedback from him would be gr8
(09:27:04 AM) ***gac410 doesn't remember doing much work on Wysiwyg/TMCE.   Other than recalling nightmares of white space loss / growth.
(09:28:09 AM) JulianLevens: http://prosemirror.net/
(09:28:11 AM) MichaelDaum: anyway: this is a rabbit hole that I an digging in and out with others
(09:28:25 AM) CDot: gac410: yeah, some of which you fixed by massaging the HTML. It's not a big deal, and your changes should work for any editor.
(09:28:46 AM) CDot: Is TMCE no longer supported, then?
(09:29:05 AM) gac410: ?   TMCE project is still active.  We're just years behind.
(09:29:08 AM) CDot: IIRC jast et al did a CK integration
(09:29:11 AM) MichaelDaum: yes, prosemirror is on my list as well ... done by the codemirror guy. yet not out of alpha as far as I get it: internals change quite some still
(09:29:45 AM) MichaelDaum: Qwiki does not convert HTML to TML
(09:30:22 AM) JulianLevens: Ok, I look forward to it maturing, I'm hopeful it's a better foundation than WysiwygPlugin et al
(09:30:28 AM) CDot: I did do half the work for a TMCE upgrade to latest, late last year. There was some work to do on the initialisation sequence, and several of the dialogs needed redesigning, IIRC.
(09:30:44 AM) CDot: but it wasn't a huge amount of work.
(09:31:08 AM) ***uebera|| would love to have a browser based interface like this --> http://i.amz.mshcdn.com/nDPE72XCAerC3yflQiVfjn_u7XE=/fit-in/1200x9600/http%3A%2F%2Fmashable.com%2Fwp-content%2Fuploads%2F2013%2F06%2FMouapp.jpg (well, one can dream, right?)
(09:31:10 AM) gac410: The whole concept of editing in TML in HTML and the resulting loss of rich markup after save is really confusing for users.  Especially loss of crafted table markup.
(09:32:03 AM) MichaelDaum: I think the only way out of that trap is to reduce editor features accessible to the user 
(09:32:18 AM) gac410: right.
(09:33:15 AM) gac410: uebera||:   yes,  that's and interesting concept.   dual WYSIWYG both on the html and markdown side.   The other editing concept we miss is the "live collaborative edting" where each change you made is seen by others.
(09:33:34 AM) gac410: I don't recall the examples now.  but that would also be a cool feature.
(09:35:11 AM) CDot: cutting down TMCE features is something I looked at, way back when. I thought it was all a bit of a minefield.
(09:35:16 AM) MichaelDaum: uebera||, for that we would need a proper javascript parser for foswiki's marup
(09:35:19 AM) MichaelDaum: markup
(09:35:32 AM) CDot: I did however make most features turn on/off from a Foswiki preference.
(09:36:46 AM) MichaelDaum: I approached it a bit differently in my experimental branch of natedit: have a common shell including the toolbar and buttons; these then instrument the editor engine underneath.
(09:37:10 AM) gac410: MichaelDaum:    Your hotfix for the slideshow xss ... is that fix acceptable to apply?    Maybe we ought to consider a 2.1.3 soonish.    Can you check it in?
(09:37:19 AM) MichaelDaum: yes
(09:38:23 AM) gac410: Also,   There are a bunch of JQueryPlugin fixes which you checked into master only... but marked release in 2.1.3  ...    So I'm not planning on picking them up in 2.1.3 patch release.  I'll fix the tasks to released in 2.2.0
(09:39:30 AM) MichaelDaum: there is more to come to JQueryPlugin, this time checked in to the right branch :/
(09:39:47 AM) gac410: Did you cherry  pick the ones only on master?   
(09:40:47 AM) MichaelDaum: no, i did not cherry pick any of those checkins from master to 2.1.x
(09:41:20 AM) MichaelDaum: I still have too many changes in my checkout area to be able to easily switch branches
(09:41:40 AM) gac410: Should they be copied over?    This is the hardest part of RM... trying to keep this stuff straight.   
(09:42:03 AM) MichaelDaum: I
(09:42:09 AM) MichaelDaum: 'll figure out how to do that
(09:43:27 AM) ***gac410 uses "git stash" a lot ... to save changes when switching between branches.     And tries to isolate all local changes it alternate webs.  I use an alternate "Usersweb" and "Litterbox"  instead of Main and Sandbox.
(09:44:01 AM) MichaelDaum: for now I created a new checkout area which I keep clean
(09:44:31 AM) gac410: And have a LocalDataContrib   which pseudo-installs Usersweb, Litterbox,  LocalSite.cfg,  etc.
(09:44:43 AM) MichaelDaum: git seriously lacks a "do you really wanna nuke any other changes" kind of dialog ... and burned me a coupe of times seriously 
(09:45:35 AM) gac410: y.   indeed.     git clean is pretty safe. ... until you add the "-fd" which really does wipe you out without question.
(09:46:12 AM) ***gac410 has been burned and can't remember what he lost.   Just feeling of deja-vu at times.   I could swear I wrote somethign to  ...
(09:46:18 AM) MichaelDaum: this is Linus shining through these cracks. I hear him saying "if you don't grok git, you deserve it"
(09:46:28 AM) gac410: :) 
(09:47:24 AM) gac410: Anyway...   No other urgent tasks.   And from what I can tell there has been NO movement by anyone on features for Release 2.2 ...
(09:47:51 AM) gac410: So September "feature freeze"    I assume we need to push that to December?   
(09:48:24 AM) ***MichaelDaum moves the calendar entry to 1 dec
(09:49:23 AM) gac410: Maybe there is another topic to discuss.     How can we engage some new developers ...   Our plugins library is in serious bit-rot.
(09:49:43 AM) gac410: Might be a question for the board 
(09:50:26 AM) gac410: And vrurg ...   did you have anything you want to bring up?
(09:51:11 AM) vrurg: Not really. The OO branch is in stabilization stage. 
(09:51:41 AM) vrurg: New plugins is in development. When done I'll present it for community consideration.
(09:52:15 AM) gac410: vrurg:   have you given any thought to any help with compatibility?    I tried to test ClamAVScan on your branch.   And was blocked by API changes ... could not throw any oops events.
(09:52:48 AM) vrurg: You mean the exception? 
(09:52:59 AM) MichaelDaum: there are two things we can do with regards to bit-rot: (1) work on those plugins that each of us has taken responsibility of (2) manage the rotten bits accordingly ... maybe make them uninstallable in some way
(09:54:11 AM) gac410: maybe add a banner %UNMAINTAINED%   or %NEEDSWORK%   to the extension topics.
(09:54:37 AM) MichaelDaum: how about moving them out of Extensions into another web?
(09:54:45 AM) gac410: The worst part is we don't even know which ones are in trouble.   We really need at least a concerted  test-every-extension effort.
(09:56:09 AM) vrurg: gac410: Ask me later on the main channel, perhaps. Tha't more to routine questioning.
(09:56:18 AM) gac410: vrurg: okay
(09:56:23 AM) JulianLevens: I'll commit to report on Extensions for the next meeting. I've code that scan every ext on f.o and on github
(09:56:34 AM) gac410: scan for ???
(09:56:47 AM) JulianLevens: Lots of different things
(09:57:03 AM) JulianLevens: Some are consistency issues on f.o.
(09:57:14 AM) gac410: So... maybe...  another radical approach.   Would it be helpful to actually release extensions and install directly from github ... for future releases.
(09:57:20 AM) JulianLevens: Some are differences between github and f.o
(09:57:56 AM) JulianLevens: github might work
(09:58:17 AM) vrurg: BTW, the problem of having different extension versions for different releases is unresolved yet.
(09:59:02 AM) JulianLevens: My main concern with using github is that we stop eating our own cookie (aka dogfood)
(09:59:26 AM) gac410: right vrurug, that's sort of where I was going with github hosted releases.  JulianLevens  true.
(10:00:26 AM) vrurg: gac410: that part popped up while I was typing. ;)
(10:00:48 AM) gac410: We've discussed extension versioning on many occasions over the years.  But no solutions.    discarded proposals litter our roadside.   
(10:01:19 AM) JulianLevens: Note that in scanning I can *test* for known coding incompatibles or errors
(10:01:33 AM) gac410: It's a really complex problem especially if we maintain backwards compat to older installed releases.
(10:02:03 AM) JulianLevens: Ideally automated running of tests, but that assume an ext has any tests
(10:02:26 AM) MichaelDaum: ... and that these tests are in any way meaningful
(10:02:39 AM) gac410: JulianLevens:   Hard part is detecting need for unicode work.   no idea how to detect lurking issues.    Any direct file I/O would be one sign.
(10:03:35 AM) gac410: It sounds ugly,  but I suspect the only way would be to divide up the 300+ possible extensions across the developers and ask that each one be installed and tested .. with some unicode topics too.
(10:04:11 AM) gac410: And ... also review Tasks web for significant reports that should block availablility of the extension.
(10:04:13 AM) JulianLevens: I came up with ideas to handle versioning of extensions, while still giving Ext authors some choices
(10:04:16 AM) uebera||: Using different Perls/"main "distributions?
(10:04:18 AM) vrurg: Compatibility maintanance cannot last forever. I think eventually extensions must be separated by major versions. 
(10:04:34 AM) vrurg: I mean major fw versions.
(10:05:06 AM) JulianLevens: yes, my idea is to allow the author to decide when to make the break
(10:05:17 AM) JulianLevens: among other things
(10:05:44 AM) gac410: I'd say it should be sufficient to test with most recent perl   5.22 or 5.24,   and if the perl code is broken,  that puts the extension on the list of "To be fixed or archived"
(10:06:05 AM) JulianLevens: Could I suggest that I report back on extensions and we can discuss next time
(10:06:15 AM) gac410: Just for our own project "reputation"   I think we want extensions to "just work"   rather than crashing/burning.
(10:06:22 AM) uebera||: JulianLevens: +1
(10:06:34 AM) JulianLevens: I'll try to get a topic on f.o. before the next meeting
(10:06:45 AM) gac410: 350 extensions  sounds good ... until we find out that many don't work.
(10:07:22 AM) JulianLevens: IIRC there were 580 ish on github, but only 350 ish on f.o
(10:07:23 AM) gac410: JulianLevens:   Sounds good.    Note that there is a task with a list of all extensions Jomo tested for utf8 / unicode compatibility
(10:07:48 AM) JulianLevens: Thank, I'll look around in general for related topics
(10:11:07 AM) gac410: Anyway... we're about +1:09 on meeting time.    Anything else to discuss?      Next meeting   19 September.    Things to ponder.   1)  How do we entice in new developers.   2) How do we fix the extension version challenge.
(10:11:13 AM) uebera||: Regarding the WYSIWYG editor/JS parser topic--did someone have a look at https://www.mediawiki.org/wiki/Extension:VisualEditor (essentially powered by https://www.mediawiki.org/wiki/Parsoid, https://github.com/wikimedia/parsoid)? If the latter could be adapted... [just found it]
(10:13:14 AM) MichaelDaum: there is a markdown parser part of codemirror as well which might be easier to borrow from while creating a js-tml parser
(10:13:55 AM) CDot: Waste of time, I fear. The parsing problems in TML stem from macros. The markdown is trivial.
(10:14:44 AM) JulianLevens: yes, we allow some contorted macro constructs
(10:14:46 AM) CDot: BTW if you want a JS tml parser I have one somewhere. Effectively a direct translation of the perl one, hacked up in an hour or so.
(10:15:09 AM) MichaelDaum: ... part of configure.js ... but highly incomplete
(10:15:30 AM) CDot: aha... quite likely
(10:15:43 AM) CDot: oh yes, that was derived from it
(10:15:56 AM) ***CDot remembers now
(10:15:57 AM) uebera||: Can't we simply ignore macros for the live preview?
(10:16:06 AM) CDot: ignore %INCLUDE?
(10:16:22 AM) CDot: you could, but.....
(10:16:55 AM) gac410: I think editing contents of includes would make for very confusing behavior.   We'd have a multi-topic-edit situation   which might have very undesirable results
(10:17:30 AM) gac410: You could include them,  but as "locked" (view only)    during the edit.
(10:19:24 AM) JulianLevens: Isn't the issue with macros is not processing them but parsing them - or at least the extreme uses of them
(10:19:43 AM) gac410: I think another huge editing/viewing issue is the META settings vs. inline settings.   One recent task reporting a broken web and security issues,  was because a plugin was editing the hidden meta settings,  and user was going nuts editing * set  ACLs   and getting nowhere.  
(10:20:36 AM) JulianLevens: I've created a pref in the past to capture then end of macro Set thing = "Blah" }%
(10:21:06 AM) JulianLevens: It's hard then to separate out %MACRO%thing%
(10:26:50 AM) uebera||: I guess step one would be to identify a suitable parser which features a tuneable grammar. For starters, all macros (even %INCLUDE) could be rendered verbatim.
(10:27:28 AM) uebera||: Might not be ready on the 1st of December, though... ;)
(10:33:46 AM) gac410: That's what wysiwyg editor does today -  all macros are verbatim.
(10:35:08 AM) gac410: Okay all ... thank you all very much.   We're at   +1:30   on the meeting.   Lets wrap this up.
(10:37:18 AM) MichaelDaum: thanks 
(10:37:43 AM) uebera||: cu
(10:37:56 AM) gac410: Next meeting   19 September.    Things to ponder.   1)  How do we entice in new developers.   2) How do we fix the extension version challenge.