Release Meeting: 22 Feb 2016

Conversation with #foswiki-release at Mon 22 Feb 2016 07:52:12 AM EST on gac410@chat.freenode.net (irc)
(08:00:30 AM) MichaelDaum: hi there
(08:00:45 AM) jomo  entered the room.
(08:00:50 AM) gac410: Howdy everyone Good morning. ... I'll be on a rather tight schedule today - Dentist appt in 2 hours :(
(08:00:56 AM) jomo: hi all :)
(08:01:08 AM) gac410: Hi jomo
(08:02:26 AM) gac410: Agenda is at http://foswiki.org/Development/ReleaseMeeting02X02_20160222
(08:04:12 AM) jomo: maybe an loud shout in the #foswiki - to join here... ;)
(08:04:24 AM) gac410: There are a few new urgent tasks, but nothing jumps out as being particularly difficult.
(08:04:32 AM) gac410: Did the shout just before you joined
(08:04:48 AM) jomo: :)
(08:06:04 AM) gac410: There are a number of accepted proposals waiting for 2.2 to open up. I assume everyone is aware that 2.1 has been branched, so we are good for merge of feature branches into master
(08:07:11 AM) gac410: Anyone want to discuss any of the tasks, or can we move to Feature proposals.
(08:10:25 AM) gac410: http://foswiki.org/Development/ContinueCanonicalSCRIPTURLDev There are two new concerns that impact the implementation.
(08:12:44 AM) gac410: rest is one exeption - needing rest/Subject/Verb but jsonrpc is another, but needs jsonrpc/NameSpace. I originally proposed adding subject=, verb=, and following that example, namespace= But those are one-off solutions What about the next script
(08:13:57 AM) MichaelDaum: the only concerns I have about the current impl are in http://foswiki.org/Tasks/Item13795
(08:14:10 AM) gac410: I'm thinking now, the default operand should be the PATH, not just the script. SCRIPTURL{"rest/Plugin/verb" ... }% or "jsonrpc/NameSpace"
(08:14:17 AM) MichaelDaum: /bin/view/Foo/Bar?topic=Foo.Bar simply doesn't cut it
(08:15:25 AM) gac410: +1. I agree. That will definitely go. I'm thinking that "if a full path is provided", then web, topic go in the query string, otherwise they are used to create the path and NOT duplicated
(08:15:29 AM) MichaelDaum: Foswiki::Func::getScriptUrl(Path) is quite fine actually. though the TML equivalent is none.
(08:16:29 AM) MichaelDaum: the _default param should be the script name, as is the case in Foswiki::Func::getScriptUrl in perl and foswiki.getScriptUrl() in javascript
(08:17:17 AM) gac410: How do we insert the "stuff" needed for rest and json. They are not web/topic, and inventing new parameters asks for collisions
(08:17:30 AM) MichaelDaum: what I was recently thinking is that we should fix the way pathinfo is processed in REST and JSONRPC: default webName to Main, default topicName to WebHome
(08:18:09 AM) MichaelDaum: the way pathinfo is captured in these cases is always wrong and brings the core and plugins in a bad state
(08:18:59 AM) MichaelDaum: what do you mean by "insert stuff"
(08:19:04 AM) MichaelDaum: url params?
(08:20:06 AM) gac410: rest is "rest/<Subject>/<verb>" jsonrpc is "jsonrpc/NameSpace" neither of which can be inserted using the SCRIPTURL macro. Have to use Concatenation today.
(08:20:45 AM) gac410: They both conflict with the core design "script/<Web>/<Topic>"
(08:21:30 AM) jomo: the SCRIPURL is nice because it is universal call to any script. In the future PSGI we _probably_ will not have "scripts", just different mount-points or just different "routes" in the core ... (don't know) So thinking about the forward compatibility - maybe we will need different macros - aka: %RESTURL{"subj" arg1=".." arg2="..."} and/or %JSONRPC{....}%... i don't know - just asking...
(08:22:07 AM) MichaelDaum: yes that makes sense
(08:22:10 AM) gac410: I only can come up with 2 solutions. 1) Have _default parameter = the PATH, not the SCRIPT, or 2) Add path=,. or subject=/verb=/namespace= parameters
(08:23:04 AM) gac410: Okay. There's a 3rd idea. RESTURL and JSONRPC instead of SCRIPTURL. (and the *PATH variants)
(08:23:06 AM) MichaelDaum: as soon as an impl isn't straight forward anymore for SCRIPTURL, it might be a good idea to use different macros as jomo proposed
(08:23:20 AM) andreli  entered the room.
(08:23:43 AM) ***jomo can't judge the impact of this...
(08:24:18 AM) gac410: Well I didn't think SCRIPTURL{"rest/CommentPlugin/comment" arg1= arg2= ...}% was all that bad
(08:24:27 AM) MichaelDaum: interestingly foswiki.getScriptUrl("jsonrpc", "some-name-space") is just fine returnign https://.../bin/jsonrpc/some-name-space
(08:25:00 AM) gac410: Can our MACRO processor use multiple default strings?
(08:25:02 AM) MichaelDaum: difference is that we only have one unnamed _default param for TML macros
(08:25:10 AM) MichaelDaum: I am not sure
(08:25:43 AM) MichaelDaum: SCRIPTURL{"jsonrpc" namespace="foobar"}% isn't bad either
(08:26:00 AM) gac410: So that's a 4th solution. Extend SCRIPTURL to allow _default to be an array.
(08:26:08 AM) MichaelDaum: as is SCRIPTURL{"rest" verb="foo" subject"bar"}%
(08:26:33 AM) MichaelDaum: if action = rest then look for verb and subject params
(08:26:35 AM) gac410: I was just concerned at least for rest, that subject= and verb= *could* collide with other query parameters.
(08:26:42 AM) MichaelDaum: if action =jsonrpc then look for a namespace param
(08:26:50 AM) gac410: That's my original MichaelDaum
(08:26:57 AM) MichaelDaum: and I like that most
(08:27:29 AM) gac410: And if script=rest then web=topic= are query string. etc.
(08:27:57 AM) gac410: So that brought up. How does a *new* script register that it needs a non-standard path.
(08:28:05 AM) MichaelDaum: while I said a default topic url param should be appended to rest at first, I now think that we should fix that in the core
(08:29:11 AM) gac410: I think rest will default to Main.WebHome ... Maybe some implementations check it anyway, but I think we always have a default
(08:29:40 AM) MichaelDaum: that would be brand new info for me
(08:30:11 AM) gac410: I'll capture all this and add it to the things that may need fixing on the proposal.
(08:30:14 AM) MichaelDaum: without a topic url param to rest webName defaults to subject and topicName to verb ... which crashes a couple of things
(08:31:18 AM) gac410: Okay MichaelDaum So that might be a core bug in Foswiki.pm where it parses the path. I did a lot of work on that in 2.0 due to the bugs in HomePagePlugin.
(08:31:52 AM) MichaelDaum: back to your question: how do new scripts specify their url format
(08:32:10 AM) MichaelDaum: difficult one
(08:32:31 AM) gac410: So one more issue with SCRIPTURL Anchors. I wanted to allow an argument of # #='somefragment' but that doesn't pass the Foswiki::Attrs parser.
(08:32:47 AM) vrurg: Hi everybody!
(08:32:55 AM) MichaelDaum: hi vrurg
(08:34:01 AM) gac410: For fragment / anchor. Extend Foswiki::Attrs to allow a parameter of #? or use "fragment=" which might collide with existing query params or ???
(08:35:27 AM) MichaelDaum: extend Attrs
(08:35:49 AM) MichaelDaum: or use anchor="..."
(08:37:10 AM) vrurg: As to the scripts – it's a question I wanted to ask in the future. Can't we get rid of them and put it all into a single one? They're all the same and only change the action parameter.
(08:37:18 AM) gac410: Hm back to registering new macros. Should this all be moved into the "Template" space somehow?
(08:37:48 AM) gac410: vrurg: It depends on you auth mechanism. TemplateAuth ... sure. ApacheAuth ... Not so easy.
(08:38:14 AM) gac410: We already collapse them for fcgi
(08:38:33 AM) gac410: At least under the covers.
(08:38:34 AM) vrurg: And I'm sorry for not making the desing proposal topic since the last release meeting. Got into some troubles, cannot tune myselft for writing it.
(08:38:53 AM) MichaelDaum: no problem
(08:38:59 AM) gac410: no prob. Things around here happen on their own schedule :D
(08:42:21 AM) vrurg: I keep it in mind all the time anyway. And just a brief report: I'm probably more than half the way down to finish the conversion stage. From the tests perspective it's ManageDotPmTests, but the core code undergone way more changes.
(08:43:32 AM) vrurg: Next stage would be to get rid of the monstrous BEGIN blocks. So, still gonna be plenty of time to discuss the plans.
(08:44:04 AM) gac410: vrurg: In the apache config, you need visibility to the action to be able to detect and authorize scripts. FilesMatch, to force auth, which must change to LocationMatch when scripts are combined
(08:45:15 AM) gac410: So the scripts could all be combined, but I assume that would mean that any existing sites would have to reconfigure apache to use a different match. I'm unsure about nginx/lighttpd
(08:46:31 AM) gac410: Okay. Next proposal ... http://foswiki.org/Development/ReduceImpactOfCGIDotPMinFoswiki has stalled, and my initial implementation sucked, and needed some re-design.
(08:46:42 AM) vrurg: gac410: Sure they will. The actual need for authnetication I would prefer to have as a part of the core code without the need to delegate this to third-party software.
(08:46:56 AM) vrurg: But that's something of lesser priority anyway.
(08:47:18 AM) gac410: vrurg, that will probably be a non-starter. To many enterprises use the web server to implement SSO
(08:48:03 AM) gac410: IT dictates "this is how thou shalt establish identity" and web app takes it from apache. That was how it was where I worked.
(08:48:53 AM) vrurg: I think jomo would be of some help with it later. Ok, forget the script/auth, that can wait.
(08:48:53 AM) gac410: We had to write a custom LoginManager / Mapper to get the identity from apache using a it provided ticketing module.
(08:50:26 AM) jast: you could extend/alter the engine to push stuff that requires auth through a redirect, unless a valid auth'd session exists
(08:50:27 AM) jomo: vrurg: :) good faith - but you know - my perl is terrible... here is already an brainstorming about the AUTH needs. In my mind - the whole auth problem is reduced to some PSGI middleware...
(08:50:45 AM) jast: that would be compatible with apache-based auth, too
(08:50:49 AM) gac410: So I'm with you in spirit, vrurg. Handling it all internally would be nice But I don't think you can get away with that in enterprises.
(08:50:54 AM) jomo: mean: auth = authentication (not authorisation)
(08:50:54 AM) jast: in fact it's how CasLoginContrib works
(08:51:03 AM) jast: err, well, that one is still based on the normal script names
(08:51:19 AM) jast: but the existing PoC-ish implementation of google auth in UnifiedAuthContrib does work like that
(08:51:53 AM) vrurg: Ppl, stop takling on auth, please! That's just don't worth it now, really. Only a matter of convinience and 'proper way to do it', nothin else. CGI.pm is really what matters.
(08:51:57 AM) jast: it doesn't use apache at this point, but the principle is the same
(08:52:57 AM) gac410: Okay. Back to "ReduceImpactOfCGIDotPMinFoswiki ... That was a proposal to replace all the CGI::markup() calls with the template engine.
(08:53:01 AM) vrurg: Back to CGI. I have recently patched Request.pm as a part of Moo-fication.
(08:53:11 AM) MichaelDaum: Reducing the impact of CGI.pm also means: recode %PLUGININFO and morph it into something useful
(08:53:29 AM) gac410: WTF is PLUGININGFO ?
(08:53:43 AM) gac410: :)
(08:53:50 AM) MichaelDaum: sorry PLUGINDESCRIPTION
(08:53:53 AM) jast: isn't a template call for each tag a little too much overhead?
(08:54:23 AM) MichaelDaum: there is too much HTML formatting in it using CGI.pm
(08:54:24 AM) MichaelDaum: it should use "the fantastic 4": header, format, separator, footer
(08:54:34 AM) gac410: It's PLUGINDESCRIPTION is not mentioned in the proposal
(08:54:46 AM) MichaelDaum: it is why I created the proposal initially
(08:55:51 AM) MichaelDaum: and then things evolved
(08:56:22 AM) gac410: Somehow that got lost. *nothing* in that proposal mentions PLUGINDESCRIPTION ... I'm not sure what that even is. Still grepping. :(
(08:56:42 AM) MichaelDaum: see System.InstalledPlugins
(08:57:09 AM) jomo: me lost too - for me the reduceCGI means: kill such constructions as: $search .= CGI::div( { class => 'foswikiTopRow' }, ...
(08:57:14 AM) MichaelDaum: there is no way to make any other - more interesting use - of PLUGINDESCRIPTION because it has hard-coded html templating using CGI.pm
(08:57:23 AM) MichaelDaum: jomo, right
(08:57:38 AM) jast: from the proposal, I favour CDot's approach
(08:57:49 AM) MichaelDaum: this is considered an anti-pattern
(08:57:51 AM) jast: for places where the markup structure needs to be hardcoded
(08:58:03 AM) jast: in other places I guess the least bad thing we can do is header/footer/format/separator
(08:58:32 AM) gac410: jast: we don't want to hardcode classes, etc. in perl. That's a really bad approach
(08:59:04 AM) jast: the template approach has similar drawbacks
(08:59:15 AM) MichaelDaum: a simple string "<div class='foswikiTopRow'>" is just fine. we _only_ need a more intelligent way of hardcoding html when form elements are involved. or stuff is encoded into attr position.
(08:59:22 AM) jast: instead of being hardcoded there, it's now hardcoded in templates. you can override it, but not on e.g. a per-macro basis
(09:00:06 AM) jomo: as usually - i'm for using CPAN.... here is https://metacpan.org/pod/HTML::Tiny - which is really great replacement for the all CGI::some_html generation... :(
(09:00:08 AM) gac410: By moving into the templates, it becomes part of the Skin, instead of the core code.
(09:00:26 AM) jast: if I want to render one instance of PLUGINDESCRIPTIONS in a table and another in a bullet list, the templates are no help whatsoever
(09:00:37 AM) jomo: but _MOST_ of the code should be moved into templates anyway.
(09:01:20 AM) MichaelDaum: there wont be much people overriding any templates for PLUGINDESCRIPTION
(09:01:25 AM) MichaelDaum: overkill
(09:01:40 AM) jast: but then what do we win by moving it into templates?
(09:01:46 AM) gac410: How we generate the installed plugin descriptions has NOTHING to do with the plugindescriptions macro IMHO. That could be rewritten anytime.
(09:01:52 AM) jast: err, disregard, misread :)
(09:02:43 AM) MichaelDaum: gac410, right. and System.InstalledPlugins should simply use the fantastic4 params to template it directly within the topic itself.
(09:02:55 AM) jast: anyway, my point is, templates are the wrong compromise IMO
(09:03:03 AM) MichaelDaum: +1
(09:03:04 AM) jast: I don't see any significant advantage over hardcoding as it is now
(09:03:49 AM) jomo: jast: advantege - skinnable things... hardcoded arent skinnable..
(09:03:57 AM) jast: except with templates we have to add additional ugly to the templates so we can allow the code to pass in class names
(09:04:02 AM) vrurg: jast: templates are easier to change, one template could be used by different pieces of code and provide consistent interface all across the site.
(09:04:12 AM) jast: okay, but you can only skin the textarea in one single way if there's a single template for textarea tags
(09:04:25 AM) jast: you don't always *want* a consistent interface
(09:04:33 AM) jast: typically there's a need for various "flavours" of an element
(09:04:59 AM) jast: so you do have visual consistency, but not on a "this tag always looks the same no matter where I use it" level
(09:05:00 AM) ***gac410 doesn't remember now why the template approach was recommended / proposed. I know I implemented the trial branch in the heat of the encoding fire, but I don't remember why.
(09:05:01 AM) jomo: jast: therefore here could be search-form template and jump-field template... as blocks...
(09:05:29 AM) vrurg: jast: Consistent means common design. Templates could handle conditionals inside to have different output for different cases. But still it will be consistent output in one common style defined by a skin.
(09:05:42 AM) MichaelDaum: hardcoding html in perl is fine as long as you can override it using header/format/separator/footer
(09:05:48 AM) MichaelDaum: params to the related TML macro
(09:06:59 AM) jast: what jomo says is an option
(09:07:09 AM) vrurg: Most reliable way would be an intermidiate rendering language which would be a basic subset of TML rendered by a skin into the final html. But this way would slow down things.
(09:07:53 AM) MichaelDaum: this starts to become a bike shed discussion
(09:08:07 AM) gac410: I propose that we table this for now. MichaelDaum, jast, Jomo, vrurg, please rip apart http://foswiki.org/Development/ReduceImpactOfCGIDotPMinFoswiki and suggest a better implementation.
(09:08:07 AM) gac410: If we can settle on a good design, I'm still willing to work on implementation.
(09:08:27 AM) jast: I'm not sure what the specific pain points are
(09:09:06 AM) MichaelDaum: you spot CGI.pm being used to create HTML? what's the best way to remove it? record it to the ReduceImpact feature proposal.
(09:09:10 AM) gac410: The major pain point is to eliminate all calls to CGI:: to render HTML.
(09:09:13 AM) jast: maybe that should be worked out in the proposal before trying to look for an ideal solution
(09:09:30 AM) jomo: gac410: the template approach is because the possibility change the code for some different skins. Imagine someone want implement for example the Foundation5 CSS framework. It needs to use some specific nested HTML constroctions and CSS classes - hard to change whith hardcoded html. easy with templates...
(09:09:30 AM) jast: okay, the ideal solution to that is what CDot suggests
(09:09:39 AM) jast: it's easy and straightforward to implement
(09:09:52 AM) jast: but since people seem to advocate other solutions, that tells me that's not the only/major pain point
(09:09:54 AM) gac410: But leaves us unable to easily skin stuff
(09:10:48 AM) jast: jomo's approach will allow us to do *anything*, but also leave us in template hell
(09:11:13 AM) gac410: Right. Some external deprecations is somewhat forcing a change. So is there a better more flexible solution, given our current implementation is not long term viable.
(09:11:25 AM) jomo: two step approach. 1.) use CDot's solution aka Render::table({.....}) etc... 2.) step-by-step move all HTML to templates. 1.) is fast 2.) is final.. ;)
(09:11:43 AM) gac410: So a bikeshed discussion forced by an external decision
(09:12:09 AM) jast: CDot's ::html would be a quick fix for the externally induced problem and it'd leave us no worse off than before
(09:12:25 AM) gac410: Rather than CDo't solution seems like HTML::Tiny would be a good option as well.
(09:12:32 AM) gac410: Why write our own?
(09:12:47 AM) jast: it's quite literally fifteen lines of code
(09:12:50 AM) vrurg: gac410: Right.
(09:13:01 AM) jast: I don't mind dependencies but some people do AFAIK
(09:13:18 AM) gac410: Right jast, But if we have to touch every place CGI:: is called, why not do it once, in a more forward looking solution.
(09:13:31 AM) jast: because it's a LOT of work
(09:13:34 AM) vrurg: jast: It always looks like 'just 15 lines'. Then you end up with a new egine as monstrous as the one intended to be replaced... ;)
(09:13:45 AM) jast: doing it incrementally is more realistic IMO
(09:14:18 AM) jast: vrurg: there's nothing wrong with more code
(09:14:28 AM) jast: it starts being wrong when the control flow stops being understandable
(09:14:52 AM) jast: if you have an appropriate separation of concerns, a huge code base can be handled just fine
(09:14:55 AM) vrurg: Except supporting that code later. What's the point of putting effort into something done by somebody else?
(09:15:01 AM) gac410: Anyway, I don't think we can solve this here. I have about 15 minutes left before I have to get ready to leave. Proposal needs some rework / discussion.
(09:15:14 AM) ***jomo not really care about the "middle" steps, until at the end everything would be skinnable... :) :)
(09:15:21 AM) jast: as I said, I don't have a strong opinion on roll-our-own vs CPAN
(09:15:30 AM) jast: just covering the other side of the argument
(09:15:48 AM) gac410: If we end up duplicating HTML::Tiny then I'd rather just use CPAN. With distribution checks on availability as usual.
(09:16:05 AM) vrurg: jast: Lavr would cover that side in the best possible way. ;)
(09:16:34 AM) gac410: Lavr ... are you around?
(09:16:34 AM) jast: okay. for now, are we going to try and go through the other proposals?
(09:17:24 AM) gac410: Lavr had 4 accepted proposals. ... just wondering if he will work on them for 2.2 No sense discussing if he is not here.
(09:17:40 AM) gac410: JulianLevens has one proposal which MichaelTempest blocked.
(09:17:55 AM) gac410: http://foswiki.org/Development/StoresShouldBePassedConfigHash
(09:18:25 AM) gac410: But Julian is not here either, and MichaelTempest seems to have left the project. So we either withdraw his objection, or take this to a community vote.
(09:18:38 AM) jast: MichaelTempest seems to have blocked based on concerns about getting the release out in time
(09:18:41 AM) jast: that's no longer relevant
(09:19:13 AM) gac410: Okay. In that case, it's just up to Julian if he wants to go ahead and for 2.2 or 3.0 .
(09:19:20 AM) jast: another 14 day period, maybe?
(09:19:31 AM) gac410: Makes sense. Store has changed a LOT.
(09:20:24 AM) gac410: There are a LOT of other feature proposals languishing in http://foswiki.org/Development/FeatureProposals
(09:20:27 AM) jast: that's all I can comment for now, not terribly familiar with the details of the proposal
(09:21:10 AM) gac410: I've been gradually parking proposals that have lost developers. It would be helpful if everyone could pick a few to review and either be willing to adopt, or park as needing a developer.
(09:21:32 AM) gac410: we really need to get the feature proposals back under control.
(09:21:41 AM) jast: makes sense
(09:22:39 AM) gac410: Right now we've got 7 proposals accepted for a 2.2 release. See http://foswiki.org/Development/ReleasePlan
(09:23:15 AM) gac410: And 3 that need more work. Michael - the preventing excess reads in store. Is that still viable or can we discard that one for now?
(09:23:56 AM) MichaelDaum: I've got fixes for RcsStoreContrib
(09:24:02 AM) MichaelDaum: what we lack is a benchmark
(09:24:23 AM) MichaelDaum: so please leave it open
(09:24:26 AM) MichaelDaum: dont discard
(09:24:35 AM) gac410: Okay
(09:24:52 AM) jast: I believe at this point we're only discarding old proposals from developers who are no longer active
(09:24:58 AM) jast: or, rather, "parking"
(09:25:38 AM) gac410: Right. But in that case I did ask if we can discard it :D
(09:25:46 AM) jast: oh. my bad.
(09:26:14 AM) jast: I'm going to park some proposals
(09:26:46 AM) gac410: But in any event, nothing gets deleted. Either parked, rejected or set do Discarded. But they can always have new life by anyone changing status :)
(09:27:58 AM) gac410: Anyone have any new business for the Release meeting? We've covered the stuff that I put on the agenda. vrurg: Anything?
(09:28:13 AM) jast: nothing from me
(09:28:45 AM) jomo: nothing more for release meeting from me ;)
(09:29:10 AM) vrurg: nothing yet.
(09:29:25 AM) vrurg: Hope to have for the next one. :)
(09:30:01 AM) gac410: Okay everyone Thanks! next meeting March 7th.
(09:32:24 AM) MichaelDaum: thanks
(09:32:32 AM) vrurg: thanks
(09:32:39 AM) andreli: bye
(09:32:50 AM) gac410: I'll get the meeting notes updated when I get back. And I'll rework the SCRIPTURL proposal. Everyone, please argue about the HTML generation proposal in the meantime :D
(09:38:23 AM) andreli left the room (quit: Remote host closed the connection).