Release Meeting: 15 Dec 2014

 
(08:00:36 AM) MichaelDaum_ is now known as MichaelDaum
(08:14:13 AM) CDot entered the room.
(08:15:02 AM) CDot: camera? ugh. What a vile idea.
(08:15:07 AM) gac410: Well quick first question .. Julian made a proposal to change .gitignore strategy.      .gitignore files all become manually managed -  checked into the repo.   Change pseudo-install to maintain a "commented block per extension"  in the .git/info/exclude  file.   
(08:15:55 AM) gac410: I think that it makes sense.     
(08:16:03 AM) gac410: Although I was originally against it.
(08:17:16 AM) gac410: We don't have to solve it now.    Revised proposal for review at your leisure   is  http://foswiki.org/Tasks/Item13135?t=2014-12-14T21:47:22Z
(08:17:37 AM) ***MichaelDaum reading again
(08:18:57 AM) gac410: Release blocker list is down to 10 items  !!   Yeay... GREAT Work everyone.
(08:20:12 AM) CDot: two comments about the way we use git
(08:20:12 AM) gac410: Item12477:  Spurious entries in .changes files    I agreed to fix,  and am procrastinating  :)    
(08:20:41 AM) CDot: fuirst, I really used to like having the changes in the mails, and one mail per checkin; I used the mail search a lot to find changes
(08:20:59 AM) CDot: second, I really miss sequential numbering on checkins
(08:21:19 AM) gac410: Well for the 2nd,  we'd have to go back to subversion.  
(08:21:30 AM) CDot: ure, I can without it
(08:21:38 AM) CDot: the first ios way more iportant
(08:21:52 AM) CDot: sorry about the typing - eating hummus
(08:21:57 AM) gac410: The changes in emails,  we'd probably have to write our email tool.   
(08:22:18 AM) gac410: I don't think there is any way to get github to generate a custom email. 
(08:23:10 AM) CDot: searching histories to find where - and why - a change was made has become a right PITA :-(
(08:23:11 AM) gac410: The easiest connection would probably be to take the "push" rest notification handled by FoswikiOrgPlugin    and have it generate a diff email.
(08:23:40 AM) gac410: Hm     doesn't   git blame help?
(08:23:50 AM) CDot: no
(08:24:34 AM) CDot: I mean yes, it does, sort of, but it's nowhere as nice to use as svn
(08:25:19 AM) gac410: the diff email wasn't svn was it?   that was our own code on f.o   I thought
(08:25:29 AM) CDot: no idea
(08:25:48 AM) ***gac410 needs to go look ... not sure either
(08:27:25 AM) gac410: well lets take that as an action item to pursue.     Michael's time is very limited today.
(08:27:51 AM) ***MichaelDaum finished reading Julian's proposal
(08:28:32 AM) gac410: A lot of his original proposal at the top of the task was changed.   
(08:29:00 AM) MichaelDaum: yea
(08:29:37 AM) MichaelDaum: there are only two things I'd like to keep as it is atm:
(08:30:13 AM) MichaelDaum: (1) freely use distro/.gitignore covering your own local ignore-needs
(08:30:45 AM) MichaelDaum: (2) have YourPlugin/.gitignore curated locally and checked in to git
(08:31:22 AM) gac410: I added a build target ... build.pl gitignore     to help with #2.    
(08:32:09 AM) MichaelDaum: thanks, but I doublt updating a .gitignore for ONE extension is that mutch of a hassle
(08:33:13 AM) MichaelDaum: I won't be much of a fan of one big gitignore covering all of the project. this is much harder to check for misses.
(08:33:15 AM) gac410: well it's intended as a manual aide,  so no big deal either way.  Some of the .gitignore files are pretty big.
(08:33:30 AM) gac410: I agree 
(08:33:34 AM) MichaelDaum: yes. and these big ones are the real problem.
(08:34:10 AM) gac410: It's a manual starting point.   For most extensions it's trivial.   Nobody has to use it.
(08:34:48 AM) MichaelDaum: if distro/.gitignore gets big, well then thats up to the developer. the project wont be responsible for it.
(08:35:22 AM) gac410: The heart of Julians  change is to abandon core/.gitignore  as automatically maintained    and change that to a "block per pseudo-installed extension"   within .git/info/exclude
(08:36:42 AM) gac410: All other .gitignore are manually maintained and checked in. ... except for distro/.gitignore    which is self-ignored.  :)
(08:37:27 AM) MichaelDaum: dunno whats better core/.gitignore or .git/info/exclude ... seems redundant
(08:38:08 AM) gac410: .git/info/exclude can never be checked in.   So it's intended for what we're currently using core/.gitigore
(08:38:45 AM) gac410: core .gitigore ... there are some particular ignore rules for working,   locale/.pot files,  etc.  that we currently miss in our generated ignores.  and would be simple to manually maintain.
(08:39:28 AM) MichaelDaum: ok
(08:40:12 AM) gac410: anyway .. I'm not planning on doing the work on his task for now.     on to blockers /   alpha/beta
(08:40:56 AM) MichaelDaum: yes please
(08:40:56 AM) gac410: Babar did a code review of HtPasswdUser     did not find any hint as to why f.o keeps getting .htpasswd clobbered.    
(08:41:46 AM) gac410: I'm going to add some additional checks,  (ie die rather than write out a small .htpasswd)   and manually install an updated HtPasswdUser on f.o.     I think this one should be a blocker  if we can figure it out.
(08:42:52 AM) gac410: On the http://foswiki.org/Tasks/MinorReleaseBlocker     There are not any that should stop an alpha build I don't think.    It's looking in pretty good shape.  
(08:43:11 AM) gac410: We really need our weblat translation tool running before we start building Beta's though
(08:44:12 AM) CDot: yes, I have added a lot of strings that will need translation
(08:44:35 AM) gac410: Other than Translations being completely missing ... any thoughts on starting an alpha process.  
(08:44:35 AM) CDot: though configure mercifully has none
(08:45:17 AM) CDot: I usually see Michael as our main alpha-testing resource
(08:45:37 AM) CDot: since he seems to find problems in trunk pretty quickly
(08:46:06 AM) gac410: I've asked gmc a couple of times about getting python / wsgi / weblate running on translate.foswiki.org     But real life seems to keep getting in the way.
(08:46:35 AM) CDot: there are two things bugging me, one of which is in the release blockers, the other not
(08:46:45 AM) gac410: I don't know freebsd well enough,  and there are some missing packages  that need special handling with bsd ports & pkgs
(08:46:46 AM) CDot: first is the audit of core dependencies
(08:47:16 AM) CDot: the second is the recording of changes,. And the main impact there is on stores
(08:47:32 AM) CDot: there has to be consistency before we release (and prefly unit tests)
(08:47:33 AM) gac410: I found a few missing dependencies when I started trying to run master extensions on 1.1.9    
(08:48:34 AM) MichaelDaum: we should have a plan for foswiki.org itself
(08:48:36 AM) gac410: you also reverted the HTML5 changes that Michael checked in.    That fixes a broken htmltidy tool which can't handle html5,  but breaks some stuff for newest IE 
(08:48:36 AM) ***CDot is going to audit the change recording once he has fixed MailerContrib i.e. some time this week
(08:48:49 AM) CDot: gac410: way ahead of you.
(08:49:10 AM) MichaelDaum: gac410, I re-reverted this and "fixed" the html validation tests
(08:49:36 AM) gac410: CDot:    okay.    I started some work on .changes,  but tbh,  keep getting lost,  so better to just ignore what I've been poking at, and go for it cdot
(08:49:39 AM) CDot: plan for f.o - I would say only upgrade it when we release
(08:50:04 AM) gac410: hm.    /me thinks it should be our first "Beta" ... 
(08:50:18 AM) MichaelDaum: no make it an alpha please
(08:50:19 AM) gac410: That's what we've done with 1.1.x    
(08:51:57 AM) MichaelDaum: switching f.o to 1.2 means disabling the FW skin overlay. some edit templates might need updating, as well as the menu structure ... which is not part of standard pattern skin.
(08:52:18 AM) gac410: I'd like to add an entry field to our registration form:   "Why do you want f.o access:"    and turn on the 1.2 "Approval" feature   send each registration request to infra list.  
(08:53:03 AM) MichaelDaum: "Why do you want f.o access:" ... report a bug.
(08:53:18 AM) MichaelDaum: .. should be as easy as possible to report bugs
(08:53:54 AM) gac410: Anything.    Just something to stop the damn spammers    We/ve deleted more that 600 accounts using the AntiWikiSpamPlugin   remove user feature.
(08:54:01 AM) CDot: hmmm. We could cook up an email submission form....
(08:54:22 AM) gac410: Anyway,  lets not get off track.  My fault .. 
(08:55:06 AM) ***CDot doubts the volume of email bug reports would swap infra list
(08:56:02 AM) CDot: gac410: do you want to review individual blockers? 
(08:56:08 AM) gac410: Ohhh     email bug report hm.   Definitely not Infra list then.    How about [email protected] 
(08:56:08 AM) gac410: so back to release plan.    2 blockers - DEP and .changes       Then build Alpha ... mostly to 
(08:56:12 AM) CDot: cos I'm not convinced htat's constructive
(08:56:26 AM) gac410: No.    I was not going to go through them task by task. 
(08:56:30 AM) CDot: :-)
(08:56:59 AM) CDot: we have two blockers depending on StephanOsthold
(08:57:12 AM) CDot: seems to me we need to kick Stephan, or kick those reports.
(08:57:15 AM) gac410: Yeah.  Neither of which really need to be a blocker IMO
(08:57:38 AM) gac410: Though commenting out CacheTests  was evil  (sorry Michael)   ... that's not the way to fix unit tests :P
(08:58:02 AM) gac410: :D
(08:58:40 AM) gac410: nginx  issues ... it's mainly the email wizard.    And needs testers to exercise the whole bootstrap process.     AHHH  that reminds me 
(08:59:03 AM) ***CDot has sympathy with that approach. Unit tests really matter when there are a lot of contributors. When you kill the tests, it speeds up dev for the few who remain, but it pisses off an alienates others :-(
(08:59:32 AM) gac410: 1)  Ship FastCgiEngineContrib   as a core extensino.      I think that's a blocker.   It's about time we make Fastcgi / Modperl  a default part of the package.
(09:00:26 AM) gac410: 2) JQTablePlugin ... I'd like to make that a default extension.   But it still has some issues.   worst is broken rowspans again.   HUGE offload for the bots
(09:00:47 AM) CDot: been trying to understand the problems with FCGI for a couple of days now, but really getting nowhere
(09:01:16 AM) ***gac410 doesn't undersand either.  It has been working fine on Foswiki.org for a very long time.
(09:01:26 AM) gac410: or do you mean bootstrap
(09:01:39 AM) CDot: I mean FCGI (as in FastCGIEngineContrib)
(09:01:48 AM) CDot: it has problems with Nginx
(09:02:13 AM) gac410: Yes.   Oh.    Michael was poking at that.    It seemed to work for me but my testing was very limited.
(09:02:32 AM) CDot: y, we've been discussing it.
(09:02:41 AM) MichaelDaum: the remaining issue is reExec on LSC changes.
(09:02:48 AM) CDot: (the contirb, not your testing)
(09:02:55 AM) gac410: yes I understood
(09:03:30 AM) CDot: well, I can't get it to fail, even stomping up and down on LSC 5 times a second
(09:03:30 AM) MichaelDaum: the difference is that with nginx the thing runs in FCGI::ProcManager mode .... which most of us didnt for ages before.
(09:03:54 AM) CDot: ... and that's where I stopped looking at it.
(09:04:38 AM) MichaelDaum: have to run. be back in 1.5h.
(09:04:40 AM) gac410: It seemed to work fine for me too.   I have a deb system with nginx,   I had to poke at the code that assumes LSC exists,  to get past bootstrap.  
(09:05:16 AM) MichaelDaum: problems only show up sporadically and when multiple parallel requests hit the site while LCS is being changed.
(09:05:22 AM) MichaelDaum: then Foswiki locks up
(09:05:52 AM) gac410: Well that's something I'm not going to be able to recreate.  :(   
(09:06:10 AM) MichaelDaum: once I disabled LCS checking things stabilized 
(09:06:33 AM) gac410: Is this a blocker for 1.2?  If we incorporate The EngineContribs  into the release?
(09:07:15 AM) gac410: IMO generally out of box will be much simpler for anyone using fcgi / mod_perl if we'd only bundle the contribs.
(09:07:19 AM) CDot: no. LSC checking is not something a running site should need to do.
(09:08:08 AM) gac410: Well.   tbh,  I use that a lot   After changing any perl module,   just "touch LocalSite.cfg"   and fcgi pulls in the new code.  No need to restart
(09:08:17 AM) gac410: well   a lot is probably relative  :)
(09:09:13 AM) MichaelDaum: I do a service foswiki restart and done.
(09:09:34 AM) gac410: apache vs. nginx
(09:09:46 AM) MichaelDaum: I think best is not to ship FastCGIEngineContrib with 1.2.0 ... 
(09:10:01 AM) MichaelDaum: then the problems can be resolved independently
(09:10:36 AM) MichaelDaum: while it is a critical part of any serious install, it only adds even more things to a core release
(09:10:43 AM) gac410: I'm really unhappy about that :(    Seems to be a very common newbie mistake.    Try to install without the contrib.      
(09:10:55 AM) MichaelDaum: true
(09:11:04 AM) MichaelDaum: but sone newbies may be unable to use it anyway
(09:11:06 AM) gac410: then complain that fw doesn't work.   
(09:11:31 AM) MichaelDaum: have to hurry. back in a while. sorry.
(09:12:12 AM) MichaelDaum is now known as MichaelDaum_
(09:12:14 AM) gac410: We *can* bugfix extensions after fw release.   We've done it all the time on Wysiwyg, TinyMCE   and several others.
(09:13:25 AM) gac410: Anyway,  I guess it's just you and me CDot :)   
(09:14:01 AM) gac410: Any thoughts on pulling in JQTablePlugin as a default extension?   
(09:14:17 AM) CDot: dunno enuf about it
(09:14:41 AM) CDot: but in principle, sounds like a wunnerful idea
(09:14:54 AM) gac410: Hm   I thought that was yours.     It's been running on foswiki.org now for a few weeks. 
(09:15:12 AM) CDot: erm.... don't THINK so
(09:15:54 AM) gac410: Hm   You had the initial commit.    Michaels done a bit of work too.
(09:17:20 AM) gac410: I don't really consider the spurious .changes entries a blocker   It would be *really* nice to fix,  but it's been hanging around as a bug since Sven introduced the feature in one of the 1.1 dot releases.
(09:18:10 AM) CDot: it *is* a release blocker IMHO as it points to a deeper problem that UI found when working on the stores
(09:18:26 AM) CDot: specifically that changes are either not logged, or logged incompletely
(09:18:33 AM) gac410: true.   
(09:18:54 AM) CDot: and I think that really needs to be fixed, as a lot of things depend on the audit trail
(09:19:25 AM) gac410: I have some code   not pushed yet,   I could push to the Item12477 branch    that added all the attachment fields.    and *started* to fix the issue.
(09:19:42 AM) CDot: aye, that would be a start
(09:19:51 AM) gac410: I need to rebase it first,   which would break things for anyone who has pulled.   
(09:19:57 AM) ***CDot has to fix MailerContrib first, but will start to look at it after that is done
(09:20:21 AM) CDot: right now, I have to go and cut in a ceiling
(09:20:22 AM) gac410: Since it's probably you and me    Ill rebase   foswiki/distro Item12744 branch   and push my changes for your purusal 
(09:20:27 AM) gac410: okay have fun.
(09:20:28 AM) CDot: ok, ta
(09:32:42 AM) MichaelDaum_ is now known as MichaelDaum
(09:33:42 AM) MichaelDaum: JQTablePlugin has got some css issues as well. 
(09:33:52 AM) JulianLevens  entered the room.
(09:34:08 AM) MichaelDaum: zebra stripes not being applied afresh when sorting a col
(09:34:37 AM) MichaelDaum: that level of hassle
(09:35:28 AM) gac410: Yeah.   agreed.   Definitely some ugliness    But still at least for f.o,  the bot workload reduction is well worth it I suspect.
(09:35:43 AM) gac410: CDot:  http://stackoverflow.com/questions/4211107/how-to-get-email-diffs-for-github-pushes      Others want this feature too
(09:35:51 AM) gac410: But no easy answer yet.
(09:37:37 AM) gac410: Anyway   distro Item12477 branch is now pushed with my hacking at .changes recording.     I had to force the push, it was badly out of date.   
(09:37:52 AM) MichaelDaum: JQTablePlugin is definitely better than TablePlugin. I'd rather be prepared to see people complain about incompatibilities. 
(09:40:03 AM) gac410: yeah. ... maybe not a good move to make it the default table plugin.   I don't really know for sure. 
(09:40:23 AM) gac410: I opened a feature proposal  http://foswiki.org/Development/JQTablePluginAsDefaultExtension     But it passed most likely due to apathy  :)
(09:42:22 AM) gac410: Michael  regarding the Engine contribs.   Nothing says we can't bugfix them between releases even if they follow release process.    We often do bugfix of default extensions when serious issues arise - Wysiwyg comes to mind.    Others too.
(09:43:10 AM) gac410: We could bundle it in 1.2,  and release note that it has stability issues on Nginx,   with a follow-up contrib release once issue is resolved.  
(09:45:19 AM) gac410: http://foswiki.org/Support/Question1538    another recent example of someone installing and forgetting the FastCGIEngineContrib 
(09:48:37 AM) MichaelDaum: let's better do it properly
(09:48:39 AM) MichaelDaum: two options
(09:49:07 AM) MichaelDaum: (1) don't ship FastCGIEngineContrib as long as problems aren't fixed
(09:49:20 AM) MichaelDaum: (2) ship FastCGIEngineContrib with Nginx problems fixed
(09:50:05 AM) MichaelDaum: Imho we should refrain from (3) ship FastCGIEngineContrib but know that it has got problems 
(09:50:32 AM) MichaelDaum: thats not resonating well with the rest of the world
(09:51:31 AM) JulianLevens: I've not had problems, there again I'm hardly stressing my dev FW, how are you stressing FastCGI
(09:51:54 AM) gac410: I consider nginx a "corner case"   still 
(09:53:08 AM) MichaelDaum: fcgi + nginx is the standard case for my clients. backing off to apache is only a fallback under rare conditions.
(09:53:51 AM) MichaelDaum: this is stable but for them on the Item13010 branch
(09:54:38 AM) MichaelDaum: Item13010 requires some changes to WebDAVContrib as well.
(09:55:23 AM) gac410: But your clients are not the only clients out there.   And apache + fcgi is still probably more prevalent 
(09:56:02 AM) MichaelDaum: as is plain cgi
(09:56:14 AM) MichaelDaum: or mod_perl
(09:57:05 AM) gac410: right.   So by not bundling fcgi / mod_perl,   we have repeated cases of users installing and it doesn't work.  
(09:58:54 AM) MichaelDaum: so you argue for (3) ship FastCGIEngineContrib but know that it has got problems ... plus tell a message along the lines: "Hey we now ship FastCGIEngineContrib. But note it has got issues with Nginx." ... is that a sensible message for our users?
(09:59:04 AM) MichaelDaum: I dont think so
(10:00:12 AM) gac410: We ship lots of stuff with known issues. 
(10:01:06 AM) MichaelDaum: *telco* 
(10:01:45 AM) gac410: Is that better than tellins ALL users,  to use fcgi or mod perl   you need to install twice.    Install with cgi,  git it congfigured,   run extension_installer to get fcgi/mp support,   and then reconfigure apache again?
(10:04:21 AM) JulianLevens: MichaelDaum can you have a test/stress case to demonstrate nginx + fastcgi flakiness
(10:04:27 AM) JulianLevens: s/can/do/
(10:04:32 AM) MichaelDaum is now known as MichaelDaum_
(10:05:43 AM) gac410: per netcraft,   for "active sites"   Apache 50%,  nginx 15%       So we make it difficult for 50% of base,  because of a bug that affects 15% of possible base.
(10:08:03 AM) JulianLevens: My preference is to use nginx +fastcgi and I want to fix FastCGIEngineContrib accordingly, I am sure I can
(10:08:08 AM) JulianLevens: eventually :)
(10:08:57 AM) gac410: And my point is that if it's broken for nginx,  then it's broken for nginx, regardless of whether we bundle or not.     It's NOT broken for apache, and bundling helps those users, and does nothing either way for nginx.
(10:12:02 AM) JulianLevens: I think we need to go for option (3) above: there are somewhat better ways to document the issues:
(10:13:11 AM) gac410: I agree.   And we are ignoring mod_perl as well.    If it's important to nginx,   then make the bug a blocker,   and try our best to get it fixed or at least a good workaround for final release.
(10:15:44 AM) JulianLevens: "Nginx is a relatively recent web-server available on which to run Foswiki and it has been used on some sites successfully. This Contrib is required to work with nginx. However, some of our testing has demonstrated flakiness in some circumstances. We are working hard to address these issues until then we have to advise caution with nginx and FastCGI."
(10:16:18 AM) JulianLevens: When do you expect final release?
(10:17:15 AM) gac410: "However, some of our testing has demonstrated instability following LocalSite.cfg changes.  We recommend restarting the foswiki service under Nginx after configuration changes, and avoid editing the configuration under workload on Nginx"
(10:17:37 AM) JulianLevens: +1
(10:17:59 AM) gac410: Maybe alpha's in the next week or two,   we really need to get the Translation engine fixed and start translation process before we get to release candidates.
(10:43:48 AM) JulianLevens: Do we wrap up somehow, or wait the returning hordes
(10:45:28 AM) gac410: I think we can wrap up ...   CDot and Michael both gone.     did you see Michaels remarks on .gitignore changes?
(10:48:01 AM) gac410: Comments from before you connected:
(10:48:08 AM) gac410: (08:29:37 AM) MichaelDaum: there are only two things I'd like to keep as it is atm:
(10:48:08 AM) gac410: (08:30:13 AM) MichaelDaum: (1) freely use distro/.gitignore covering your own local ignore-needs
(10:48:08 AM) gac410: (08:30:45 AM) MichaelDaum: (2) have YourPlugin/.gitignore curated locally and checked in to git
(10:48:36 AM) gac410: I think that says the move from core/.gitignore to .git/info/exclude for pseudo-install is okay to go.  
(10:55:27 AM) JulianLevens: I don't agree with distro/.gitignore for local ignore needs when there is a distro/.git/info/exclude meant to be used for that. I'll chat with MichaelDaum privately to adjust. For now assume distro/.gitignore is as Michael suggests; there is arguably no strong need for distro/.gitignore to be part of the repo. If we find a strong need to change that then a FeatureProposal will needs writing and people to adjust their wo
(10:55:27 AM) JulianLevens: was hoping to eliminate that possibility now.
(10:56:57 AM) JulianLevens: I'll check these logs when you add them to f.o
(10:56:59 AM) gac410: I don't see any reason that we need distro/.gitignore to ever be checked in.   We can work with next level down. which is consistent across all extensions + core directory.   so really we don't care what anyone does with distro/.gitignore as long as they don't check it in.
(10:57:28 AM) gac410: I was more concerned that someone would object to automating .git/info/exclude    
(10:58:04 AM) JulianLevens: Right now distro/.gitignore will never be checked in and we are safe
(10:58:45 AM) gac410: Next release meeting  ... Monday 12/29.    I think we should skip that due to Christmas week,  and  start afresh on January 12th.
(10:59:14 AM) JulianLevens: Ok by me
(10:59:45 AM) gac410: Along with a kick-off email to foswiki_discuss  foswiki_svn   to try to get more participation.   Hopefully between now and then it's safe to annouce that we will be starting alpha/ builds.
(11:00:01 AM) JulianLevens: Wit respect to the .gitignore tools were you considering a sub-directory with separate sections or ?
(11:00:30 AM) gac410: er...  subdirectory under .git/info ???  
(11:00:42 AM) JulianLevens: Actually, I'll volunteer to uopdate the tools
(11:01:21 AM) gac410: I think just annotated blocks in info/exclude    #====Begin WhateverExtension ===    #===END      
(11:01:55 AM) gac410: And maybe a block   #=== EXTENSIONS     with a list of just the root level directories of any extensions that were added with git clone.
(11:03:05 AM) gac410: Go ahead ... You have the "token" ... assigned as working on the task.
(11:05:05 AM) JulianLevens: Ok, see you soon on #foswiki
(11:05:09 AM) JulianLevens left the room.
(12:30:49 PM) MichaelDaum_ left the room (quit: ).
(01:24:15 PM) CDot left the room (quit: Quit: Leaving.).