Release Meeting: 03 Oct 2016

(09:00:10 AM) gac410: Hi everyone ..  welcome to the release meeting.   
(09:00:22 AM) jomo entered the room.
(09:00:40 AM) gac410: Hi jomo
(09:00:48 AM) jomo: hi
(09:00:58 AM) jomo: any news?
(09:01:06 AM) gac410: It's been quiet
(09:01:28 AM) vrurg: Hi!
(09:01:47 AM) gac410: I remember sometime in the past week wanting to ask you something jomo, .... and I have no idea what it was.   :(
(09:02:10 AM) jomo: sure nothing important then :)
(09:02:59 AM) jomo: also just write anything to the main chat - me time-to-time (not so often) checking the irc-logs...
(09:03:49 AM) gac410: Can't find it in the logs,  so must have just thought it to myself.   Where's jomo when you need him   ;)
(09:04:12 AM) jomo: lol ;)
(09:05:27 AM) gac410: Anyway ... no new blockers have come up in the past weeks.  So from a tasks perspective it's been quiet.    
(09:06:52 AM) gac410: jomo you had an old task about possible cache poisoning ... https://foswiki.org/Tasks/Item13797     I think that can be No Action.   I could not recreate it 
(09:07:25 AM) gac410: Ah... it's marked Security.   can't discuss it here :(
(09:07:38 AM) jomo: in the last release-meeting we agreed - just set it to NoAction (for now- maybe sometime will return to it) :)
(09:07:50 AM) gac410: okay will do.
(09:09:05 AM) jomo: any task opened by me what nneed some attention from my side?
(09:09:07 AM) gac410: vrurg ... anything to cover with  your work?
(09:09:52 AM) gac410: jomo,  I don't think so.      I've been trying to scrub the 450+ "New"  tasks that have been languishing in task purgatory.     
(09:10:24 AM) jomo: omg...
(09:11:59 AM) gac410: We really need to make sure tasks are either "Confirmed"   or "No Actioned"   and have proper categories set so that they can be found.   It's a rather hopeless task though :(
(09:12:03 AM) jomo: anyway - i saw in some other open-source project (couldn't remember which one) - every closed task must end with an unit-test case. In such way the task can't be introduced again, because here is an unit-test. But, it is probably an tremelous work...
(09:12:42 AM) gac410: y ... Really every "opened"  task should get a unit test,  so you can tell when you've fixed it   :D
(09:12:58 AM) jomo: ah - yeah :)
(09:13:43 AM) vrurg: Sorry, I was away.
(09:14:18 AM) vrurg: I had quite tough two weeks, hardly managed to make an example extension module with docs.
(09:14:21 AM) jomo: what is a status of task which are written, so theyre opened, but the current foswiki can't solve them?
(09:14:58 AM) gac410: hm   Needs Developer?    or "Confirmed"    
(09:15:20 AM) vrurg: The docs are ugly but that's my English and no reason to polish them before the thing is accepted.
(09:15:23 AM) jomo: i have one of such task, just trying to fid it...
(09:16:32 AM) gac410: vrurg:    I think I like what I was seeing in the empty extension ...    Instead of present or mssing   sub someCallback{}       you create a ref    someCallback => { ... }   right?
(09:17:07 AM) gac410: So if I want an after save hander,    I define afterSave => { code to run after save }
(09:19:08 AM) vrurg: No, it's gonna be somewhat different. Callback names are just strings because they can change in time. So, for an extension it would be 'callbackHandler afterSave => sub {...};'
(09:19:41 AM) gac410: vrurg,    I was the one who vounteered to write an simple plugin ...   but I've been very busy myself ...   so never got to catch up with you.
(09:21:12 AM) vrurg:  It wouldn't be possible for me to help you either. Urgent and big task at work and some personal issues... 
(09:22:43 AM) gac410: no problem   I understand.    
(09:23:38 AM) vrurg: gac410: Regarding the callbacks again – I think your syntax could be implemented too. But it won't work if different modules would request registering same name callbacks.
(09:24:05 AM) gac410: No problem.     I was just trying to recall what I read in your docs.
(09:24:43 AM) gac410: Generally I've been quickly scanning patches - didn't go looking at the actual docs.   So following diff's can be diff-icult
(09:26:06 AM) vrurg: I'm just thinking. Trying to weigh pros and contras. I do allow short callback names in case they're registered only once. Pure convinience thing but has it's weak points.
(09:26:43 AM) gac410: vrurg:    I assume you'll need BuildContrib to support the lib/Extension/SomeExtension directory ...   and presence of a Config.spec,  etc.   
(09:27:01 AM) gac410: Or do you envision a different packaging / installation procss
(09:28:12 AM) gac410: Ah... VERSION.    we pretty much moved away from "triplet" version   and "use version;  ..." code.     It was getting too complex,  and simple decimal  1.1 versions work fine.
(09:28:14 AM) vrurg: I'll need somebody's help on this. Frankly saying, I'm a bit tired and may decide to take a break.
(09:28:45 AM) gac410: I don't blame you on that one.    Hero development can only go just so far.    
(09:30:02 AM) vrurg: I like version though. It's universal and flexible. But as to API versioning I have another thing in mind. Usually when API changes it means some feature came in or out.
(09:30:28 AM) gac410: We don't have enough of a majority here for big decisions for sure.   But are we reaching the point where you'd like a decision on,  1) Moving forward with OO extensions, and 2) Accepting  OO branch for the next major release. 
(09:31:17 AM) gac410: vrurg:   I agree.  I was all for the version  ... and pushed it hard.   Until we discoverd that (stop me if you've heard this before ...)    RedHat didn't support it.
(09:31:36 AM) vrurg: So, I'm thinking of declaring these features as keywords. Then an extension could simple 'request' them. This way an old extension with simple requirements may survive some major API changes simple because it requests 'someVeryBasic' feature.
(09:32:12 AM) gac410: The issue with extensions was for older sites on older OS that wanting new plugins.    tripped over version....   But of course your code doesn't run into that because it needs a much newer perl in the first place.
(09:32:35 AM) vrurg: What? No version in RedHat???
(09:32:47 AM) gac410: vrurg:    Y,  keyword features vs. API versions.    We actually explored that for a bit.   using context
(09:32:50 AM) vrurg: Are, for older versions. 
(09:33:33 AM) gac410: vrurg:    y.   for RH stuck on perl 5.8.8    version was not updated either.  The modern version appeared in 5.10.1   iirc.   Royal pain.   But not an issue for you!
(09:33:57 AM) vrurg: Context is a good concept. I like it. But it's runtime thing. While extensions are being loaded before anything else (even LSC) and cannot really get use of it.
(09:34:16 AM) JulianLevens entered the room.
(09:35:04 AM) gac410: https://foswiki.org/Development/AddFeatureLevelContext
(09:35:42 AM) gac410: The challenge was that the JavaScript and Skin side of things can't determine if a feature exists.     Context allows that.
(09:35:58 AM) vrurg: So, back to the decisions to make – I think it is time to move on. Extensions if accepted could be declared beta-feature and polished for 2-3 minor version before getting to production-ready stage. 
(09:36:46 AM) gac410: So the new extensino model sits along side the old API.   doesn't replace it.   We don't need to convert extensions.
(09:36:50 AM) jomo: just for the record :D :D :D - could someone explain for further readers what the "context" really means? With simple english? :D
(09:37:39 AM) gac410: It was just a way of declaring a state for the environment.       So "context"  was a series of  words that either exist or do not exist in the environment
(09:37:54 AM) vrurg: Pushing the OO branch to the major version would also need some work because there're issues I wasn't able to address. I'll have to get through my notes and see what's left behind. 
(09:38:43 AM) gac410: set context "static"    will tell everyone that there can be no dynamic web interaction  (like a printed or pdf output)... So if context static,  don't render edit buttons.
(09:39:12 AM) jomo: ok - that would be enough clear :)
(09:39:35 AM) jomo: not would - it is enough clear :)
(09:40:28 AM) gac410: There is a whole big list of contexts.      https://foswiki.org/System/IfStatements#Context_identifiers
(09:40:31 AM) jomo: someone would call it "flag".... :)
(09:40:35 AM) gac410: yes.  
(09:41:19 AM) jomo: but yes, the "context" sould more enterprise-ish :)
(09:41:30 AM) jomo: s/sould/sounds/
(09:41:31 AM) vrurg: gac410: Actually, feature set context as I see it is only needed to decide wether to load or disable an extension. Could be statically defined – it's anyway not runtime changable. But could be imported into the context later to make it dynamically available.
(09:41:56 AM) gac410: vrurg.   The two feature contexts that have been already defined:
(09:42:07 AM) gac410: SUPPORTS_PARA_INDENT            render supports the paragraph indent syntax
(09:42:07 AM) gac410: SUPPORTS_PREF_SET_URLS            Preferences can be set in the URL 
(09:42:32 AM) gac410: CDot added them iirc
(09:42:34 AM) vrurg: Say, there is a static feature someFeature for API – it later becomes 'API_someFeature' context key.
(09:42:57 AM) gac410: yeah that would work 
(09:44:03 AM) gac410: vrurg:   So we have the topic: https://foswiki.org/Development/WhippingOOFoswikiIntoShape   ....   Can that serve as your list of  stuff that needs to be done?
(09:44:35 AM) vrurg: It will. I have it always opened in my browser.
(09:44:52 AM) gac410: Ah ha...   Good.  
(09:45:53 AM) vrurg: I'm afraid I forgot about a big problem. And now don't even remember what it was about because it came across my mind while driving and wasn't noted.
(09:45:55 AM) gac410: The project is small enought that I don't think we can afford to have more than one major release in development.   Or even minor.    So maybe we should even consider scrapping 2.2,   ... there has been NO progress on features.
(09:46:59 AM) gac410: I've got about 10 minutes left.   I have a hard stop at 1400Z ...  have to go meet someone.    
(09:48:06 AM) vrurg: aha, it's anyway close to be around 1hr long meeting.
(09:50:01 AM) gac410: I do recall one concern is that setlib, and loss of very early dependency checking.   Maybe we need to ship just a plain old perl cgi script that  does a check and print of the minimal dependencies.
(09:50:54 AM) vrurg: Actually it could be a part of the starting script.
(09:51:19 AM) vrurg: I could even guess there is a Plack middleware which could take care of this.
(09:51:37 AM) gac410: hosted sites.   No starting script.     "Ftp" in a directory and run a web request.    That's it.   no shell,     cpanel for dependencies.  
(09:52:33 AM) vrurg: But it reminded me about one feature I wanted to add. The initial config could be in .ini format for user readability (what I always dislike about setclib).
(09:53:27 AM) vrurg: By 'starting script' I meant action scripts.
(09:53:34 AM) gac410: well  setlib  is NEVER edited.     but y,  LocalLib.cfg  could be some other format.   .ini file?      Aint no windoze here.
(09:53:51 AM) jomo: vrug: as i told already - the plack middlewares are designed to be executed in EVERY request. What is the point to have a middleware for configuration-things? Such things should be loaded before (in the app-inot phase) and not at every request.... IMHO...
(09:53:59 AM) vrurg: .ini is easy to read, could be commented and users know it well.
(09:54:11 AM) gac410: I don't think I
(09:54:30 AM) gac410: I don't remember the last time I ever saw an ini file.      
(09:54:48 AM) vrurg: It is not to replace LSC – it's just a bootstrap thing for very early init.
(09:54:48 AM) gac410: probably 6+ years ago
(09:55:08 AM) jomo: gac410: the current LSC format is just terrible - and it isn't portable, and also impossible to read it with external (non perl) tools...
(09:55:26 AM) gac410: nobody should read it.    That's what configure is for.
(09:55:46 AM) jomo: using portable serialisation (like: INI, or yaml, or even XML :D) only could help...
(09:56:03 AM) jomo: no the config must be readable
(09:56:20 AM) vrurg: Let's discuss it later, when we have more time. My only point here is to let any extension register any simple initial data it may need upon startup. 
(09:57:17 AM) vrurg: jomo: If we get my new extensions involved there could be different configuration implementations.
(09:57:22 AM) gac410: Guys.   there is enough to do already.    PLEASE don't go re-inventing the config file.     there is no reason to go reading it.      It's internals behind the scene.     So who is going to rewrite configure and all the checkers and some gazillion lines of code.
(09:57:31 AM) vrurg: So, leave it alone. It's the basic thing nobody would chage.
(09:58:04 AM) jomo: gac410: nobody says anything about rewite of the checkers... ;(
(09:58:42 AM) jomo: here are TWO things - in memory represenation (now an hash) - and the STORED format (aka serialisation)
(09:59:46 AM) gac410: okay.   yes   you could change the serializer.    but again..    Why...     That is a lot of work,  and would effect EVERY extension,  as the .spec file is also in configure file format.
(09:59:49 AM) jomo: you can get the same "in-memory" hash even from yaml..
(09:59:53 AM) vrurg: gac410: Would you get time for this there is a sample extension for Foswiki::Config (in OO, of course) where %Foswiki::cfg is a tied hash. This would keep all the configuration framwork afloat irrelevantly of the backend being used.
(10:00:08 AM) jomo: no, extensions NEVER reads the LSC
(10:00:21 AM) jomo: extensions uses the $Foswiki::cfg
(10:00:38 AM) JulianLevens: Guys, I'm already doing work in this area
(10:00:39 AM) gac410: jomo,    extensions *ship*   a Config.spec file   which uses the same format as Foswiki.spec and LocalSite.cfg.
(10:00:55 AM) vrurg: Guys, there is no point replacing LocalSite.cfg with LocalSite.xml. But what is the point is replacing it with a DB implementation. This is a step towards clusterized foswiki.
(10:00:55 AM) jomo: yes, the *spec file.
(10:01:02 AM) jomo: nut not the LSC itself.
(10:01:08 AM) jomo: ^^but
(10:01:11 AM) JulianLevens: Albeit I started from a different point\
(10:01:45 AM) vrurg: JulianLevens: that's interesting. Start from this point, please. :)
(10:02:09 AM) JulianLevens: I'm with gac410 in that it can wait for another time
(10:02:22 AM) gac410:  I'm out of time.    You guys can hash it out further but I need to run.     I'll leave my session open so I can get the logs later.
(10:02:30 AM) JulianLevens: My starting point was automated creation of a foswiki-server
(10:02:32 AM) vrurg: JulianLevens: Do you have a proposal? Or plan to write one?
(10:02:38 AM) jomo: JulianLevens: ++++++++++++
(10:03:05 AM) jomo: Thats is inline how the "real" plack should work...
(10:03:09 AM) vrurg: gac410: Thanks! CU later!
(10:03:14 AM) jomo: vrurg: CU later
(10:03:16 AM) gac410: Anyway ... thanks all for attending.    Please keep the discussion going.     CU all later.
(10:03:23 AM) jomo: JulianLevens: could you tell more please?
(10:03:27 AM) vrurg: jomo: I'm not leaving yet.
(10:03:32 AM) JulianLevens: https://foswiki.org/Development/FoswikiVagrant
(10:03:43 AM) jomo: ahh. so gac410  :)
(10:04:13 AM) JulianLevens: https://github.com/Jlevens/FoswikiIntegration
(10:05:13 AM) JulianLevens: As part of this I've been thinking of combining the DEPENDENCIES MANIFEST and Config.spec
(10:05:55 AM) JulianLevens: For better dependency management
(10:06:53 AM) vrurg: BTW, good news about the OO branch: It doesn't have major memory leaks. They're lurking somewhere for sure, but running thru few topic and using PerlDoc displays no dynamic leaks, only global variables left behind.
(10:07:06 AM) JulianLevens: I've been playing with JSON not .ini or yaml or XML
(10:07:28 AM) JulianLevens: Not that that totally matters, serializer should be pluggable
(10:08:10 AM) vrurg: JSON is bad for end user as it doesn't support commenting. Unfortunately. :(
(10:08:37 AM) jomo: JulianLevens: again, JSON, yaml, XML or anything "tree-like" is a matter only of the serialisator/deserialisator ... simply anything what is portable is MUCH better as the current perl-code... :(
(10:09:23 AM) JulianLevens: I know, but an easy hack is to allow lines beginning # to be comments and strip them out before parsing the JSON
(10:09:56 AM) jomo: vrurg: don't mix the "config.spec" (which should be commented) with the LSC (in any format) - which should be generated... (e.g. saved by the configure-app - it doesn't need to have comments)
(10:10:46 AM) jomo: when you will fetch the config from the database you will not have comments too.. :)
(10:11:25 AM) vrurg: I'd better avoid hacks. But whatever... Config.spec is for developers – it could remain perl.
(10:11:54 AM) JulianLevens: Yes, but it's currently inflexible
(10:12:37 AM) jomo: but the generated (aka saved by configure-app) LSC should be serialised to ANY common format - my preference is YAML - readable, editable, text-based, simple and PORTABLE...
(10:12:39 AM) JulianLevens: I'm also aware that {SomeConfigValue} = 1; could mean that an 'optional' dependency is not optional anymore
(10:13:22 AM) jomo: i said this in 2012 :( https://foswiki.org/Tasks/Item11836
(10:14:33 AM) vrurg: BTW, what is a big problem with the config is lines where expansion required. '$Foswiki::cfg{SomeDir}/somesubdir' is so inflexible and it currently would kill any attempt to serialize to anything but perl code.
(10:15:01 AM) vrurg: Tied %Foswiki::cfg would resolve the issue but it would still looks terrible.
(10:15:02 AM) JulianLevens: JSON::PP is perl core module
(10:15:09 AM) JulianLevens: YAML aint
(10:15:58 AM) JulianLevens: y, It's a big job and can wait for 4.0
(10:16:11 AM) JulianLevens: Does that gives me a couple of years?
(10:17:06 AM) vrurg: JulianLevens: sure it does. My estimation is ~6 months to get 3.0 ready. Not earlier.
(10:17:06 AM) jomo: we will ship the CPAN-contrib anyway - we need many CPAN modules - so, we really should stop argung "againist cpan modules" - if it isn't core - just ship it as contrib... anyway, the serialised format isn't matter - it could be multiple (JSON also YAML)...
(10:17:52 AM) JulianLevens: Agreed but json is a better default
(10:18:02 AM) ***vrurg joins the format discussion again: DB! DB! DB! ;)
(10:18:13 AM) vrurg: Sorry... 
(10:19:04 AM) jomo: vrurg: when you talk about DB - what this exactly mean?
(10:19:14 AM) jomo: for the config?
(10:19:36 AM) jomo: e.g. you will end with some DB-table, where each row is some config value?
(10:19:40 AM) vrurg: jomo: MySQL, PostgreSQL, or  AnyOtherSQLorNonSQL.
(10:19:45 AM) jomo: OMG :(
(10:19:53 AM) vrurg: The purpose is to have it shared among several servers.
(10:20:08 AM) jomo: try understand the question... to... not only read it ;)
(10:20:39 AM) vrurg: jomo: If you ask about which way to store it – I don't know. It's gonna be up to extension developer.
(10:21:16 AM) vrurg: On my client side I must get just a transparent $app->cfg (in OO branch terms).
(10:22:18 AM) JulianLevens: Anyway guys I have to go
(10:22:37 AM) jomo: cos the configuration is hierarchical (tree-like) thing... using (just one) DB-table isn't very viable. Of course you could simulate the tree in one (or more) tables - but it would be rather hard to read/manage... - im not againist DB - just.... it is MUCH more complicated as change the perl-code to JSON or YAML...
(10:23:30 AM) jomo: much easier is: fetch the whole config as an JSON string from the DB
(10:23:38 AM) jomo: :)
(10:23:48 AM) JulianLevens left the room.
(10:23:58 AM) jomo: JulianLevens: good points - cu
(10:24:06 AM) jomo: hes gone.. :)
(10:24:13 AM) vrurg: jomo: That's now getting beyond usual release meeting format. Just to close the matter: http://search.cpan.org/~bjeps/DBIx-Tree-1.9/Tree.pm as an example. And again: clusterisation.
(10:24:44 AM) vrurg: I'm leaving this room. Getting back to foswiki. CU and thanks everybody!
(10:24:54 AM) jomo: cu - leaving too
(10:25:03 AM) jomo left the room.