Release Meeting: 10 Aug 2015

Details

1. 2.0.2 blocker review

  • 13525 (Closed): Store reads the same file repeatedly from disk May be a duplicate of 13593 (Closed): Foswiki 2.0 is 350% slower doing query SEARCH compared to 1.1.9.
Discussion of the two tasks, Lavr reported that there have been no side effects from the fix to Item13593. MichaelDaum needs to re-profile the performance with the MetaCache patch applied. Downgraded Item13525 to be Normal priority unless profiling shows it's still an issue.
  • 13537 (No Action Required): Cache failures on Foswiki.org
This is being downgraded to Normal, and may end up "No Action". The issue has not occurred again, now that statistics is being run once a day.
  • 13561 (Closed): PERL type configuation settings lose their formatting when LocalSite.cfg is saved. Possibly fixed, needs review by MichaelDaum
  • 13598 (Closed): Error renaming a symlinked web using PlainFile Store. Store issue may need Crawford.
Some discussion. It only impacts sites using Symlinks for web directories. However both Lavr and Jomo run sites with most webs symlinked in a production environment. So this really should be addressed
  • 13560 (Closed): configure does not set initial values when extensions are installed with an external package manager. Install by way of unzip or git has issues - config not correctly populated. Andreli and MichaelDaum are very concerned. Installing from outside of configure seems to be a common approach. Partial fix committed. New wizard to fix up extensions installed external to configure. Fails to change settings when run from web... Javascript issue?
Some discussion. Agreed that a new wizard to merge plugin updates, either via web or shell is acceptable workaround. Wording of "repair" is not really appropriate. Better choice might be "merge". The wizard works in the shell but seems to have exposed an issue in the javascript code, not all updates from the wizard are successfully applied into the DOM. This needs help from someone more knowledgeable in Configure jquery design.
Both Lavr and gac410 have looked into this to figure out why TABLE and EDITTABLE macros are not playing well together. So far, no luck.

Next meeting - - Monday Aug 24, 2015 1300Z — ReleaseMeeting02x01_20150824

IRC Log

(08:17:42 AM) andreli [~andreli@2a01:488:66:1000:b01c:164e:0:1] entered the room.
(08:40:15 AM) andreli left the room (quit: Ping timeout: 250 seconds).
(08:41:38 AM) andreli [~andreli@2a01:488:66:1000:b01c:164e:0:1] entered the room.
(08:50:25 AM) JulianLevens [~JulianLev@82-168-31-129.ip.telfort.nl] entered the room.
(08:58:40 AM) jomo [~jomo@adsl-dyn-18.95-102-221.t-com.sk] entered the room.
(08:59:26 AM) Lavr [~Lavr@foswiki/developer/Lavr] entered the room.
(08:59:55 AM) jomo: already here...
(09:00:15 AM) gac410: Hi everyone.  good day.   I'm unfortunately on a tight schedule today.   
(09:00:51 AM) gac410: Not much different from 2 weeks ago agenda.   Today:  http://foswiki.org/Development/ReleaseMeeting02x01_20150810
(09:00:55 AM) Lavr: Hi all
(09:01:06 AM) gac410: Hi Kenneth
(09:01:09 AM) JulianLevens: HI
(09:02:02 AM) andreli: Hello
(09:02:14 AM) MichaelDaum: Hi
(09:02:39 AM) gac410: Okay.   First on agenda.  Urgent task review.     I think that http://foswiki.org/Tasks/Item13525   and http://foswiki.org/Tasks/Item13593   may end up being the same underlying issue.
(09:03:18 AM) gac410: Lavr,  Item13593 .. .you're testing as you called it, the miracle patch  :D    Any more feedback.
(09:03:48 AM) Lavr: All I tested looked very good. I have seen no negatives.
(09:04:17 AM) MichaelDaum: Item13525: I had no time to profile the new code again. until I find time this bug item seems safe to downgrade its prio.
(09:04:28 AM) MichaelDaum: so that it does not block any 2.0.2
(09:04:34 AM) gac410: MichaelDaum:    And you had profiled and found the original issue of excessive reads from store.       Ah.. okay   Answered before asked :)
(09:05:24 AM) MichaelDaum: those repeated calls to readFile() on the same file seem to be unrelated to %SEARCH ... but who knows
(09:05:58 AM) gac410: I think we can downgrade http://foswiki.org/Tasks/Item13537    ... the cache behavior on foswiki.org   I have not seen issues since we changed statistics to run once/day
(09:06:03 AM) MichaelDaum: uploading nytprof html seems to be a good practice to establish
(09:06:57 AM) gac410: Agreed.   It might be good to have some target topics to profile.    Simple get,   text search, query search, ...    so we have baseline to compare with.
(09:07:50 AM) MichaelDaum: well depends on the issue under consideration: base line rendering, %SEARCH, some other macro, ...
(09:08:09 AM) gac410: right 
(09:09:01 AM) ***MichaelDaum not seen "rollback ineffective with AutoCommit enabled" in the logs
(09:10:07 AM) gac410: We were originally running statistics ever couple of hours.  It was probably way too often.  
(09:10:07 AM) gac410: Another one that we might be able to close is http://foswiki.org/Tasks/Item13561 ...  ugly perl hashes.   I changed the code to format the save. 
(09:10:57 AM) gac410: So I left that waiting for you michael.   If you need the code to preserve what was originally entered, then it's not a fix.   If it's okay to save it in a "pretty" formatted fashion, then probably it's fixed
(09:12:29 AM) gac410: the FormTypes field was readable on next save, after my patch.  
(09:13:10 AM) MichaelDaum: gac410, okay. so how does it format the perl code? does it apply perltidy?
(09:13:36 AM) jast: my guess is Data::Dumper :)
(09:13:41 AM) gac410: No,   I changed an option on Data::Dumper   to not use the shortest form.
(09:13:51 AM) gac410: good guess jast  :)
(09:14:09 AM) jast: I may have stumbled across the commit e-mail
(09:14:55 AM) MichaelDaum: alright. 
(09:15:46 AM) gac410: And staying with configure issues   http://foswiki.org/Tasks/Item13560    is the issue with installing extensions with unzip / git clone
(09:16:54 AM) gac410: I wrote a new wizard.     tools/configure -save -wizard Plugins -method repair       It cross-checks the Foswiki/Plugins and TWiki/Plugins directories with the {Plugins} settings and fills in the missing pieces
(09:17:26 AM) MichaelDaum: yea well this is not a matter of "repairing" LSC
(09:17:29 AM) gac410: At the same time the config.spec gets merged.    Works fine from the command line.  But having a bear of an issue with the javascript / browser DOM
(09:17:55 AM) gac410: okay.  So I'll rename the method to "merge"    :)
(09:18:00 AM) Lavr: It is more an "import" than a "repair"
(09:18:09 AM) MichaelDaum: all defaults in extensions# config.spec shall be applied ... always
(09:18:50 AM) MichaelDaum: nice to have it as a cmdline tool. but the web ui should do so as well. otherwise you end up fighting down pointless error warnings about no-present settings.
(09:19:31 AM) Lavr: configure used to import the specs IF the individual settings were not defined in LSC already
(09:19:45 AM) gac410: I think it's a bug in the javascript, or some magic that I'm missing.   If you "install" an extension using Wizards/Install...    the settings "take"  
(09:19:48 AM) MichaelDaum: ... and we need this behavior back -> regression
(09:20:31 AM) gac410: Lavr,  Configure is no longer monolithic.    IMO to get it back  we revert to monolithic configure.   I can't see how to make it all happen by magic when the configuration is saved in the javascript dom.
(09:20:33 AM) MichaelDaum: this problem also applies to people installing foswiki via apt-get on debian machines
(09:21:12 AM) Lavr: Invoking the import by clicking a button for the plugin inside configure would be OK.
(09:21:13 AM) MichaelDaum: though deb packages could have a post install call to the cmdline tool
(09:21:15 AM) gac410: I added a wizard button to learn / merge extensions.      But the settings don't stick because of something wrong in the javascript space.
(09:21:37 AM) gac410: Yes.   I'm hoping the deb woudl just use the cli tool to merge the settings.
(09:22:55 AM) gac410: The wizard I wrote and checked in.  seems to be 100% fix from shell.   But when run from a button,  it fails to trigger the DOM to detect changes and enable the SAVE button.
(09:23:53 AM) gac410: It *should*...    I c/p the exact same code that the InstallExtension wizard uses to set the fiels.  And it works find for any-old-setting    but seems to ignore the {Module} and {Enabled} settings.  
(09:25:27 AM) gac410: And yet the ExtensionsInstaller *does* set {Module} and {Enabled}   so why one wizard does, and the other doesn't  I'm at a loss.   Debugged the JS and it just doesn't see the change.
(09:26:33 AM) gac410: But I guess the biggest question.   Lavr said a button is fine.   Anyone else?   Button in the UI   maybe with a checker that Warns "New extension detected - click here to merge settings"    
(09:26:41 AM) gac410: would that be an acceptable fix.
(09:27:20 AM) gac410: If so I'll keep working on it.  If not,  then don't want to waste the effort. :)
(09:27:25 AM) Lavr: I could live with it. You always have to enter configure after an unzip anyway so ....
(09:28:06 AM) Lavr: ... when I enable a plugin I may as well "import" configs
(09:28:15 AM) gac410: Also,  if you are sitting there in the right locatino,  after unzip SomeExtension,  follow it with tools/configure -wizard -method merge    
(09:28:56 AM) gac410: And I have to step out for 5 minutes ... contractor is here.    needs decisions  :)   my apologies.
(09:29:17 AM) Lavr: Problem with CLI is that you have to remember. In configure you see the button and know you have to click it.
(09:29:36 AM) Lavr: remember and know. CLI is for developers.
(09:31:02 AM) jomo: Lavr: CLI is also for wiki-admins, who dont't like "clicking" (or actualy don't have installed a browser on the server)... etc.
(09:31:59 AM) Lavr: No worries about a CLI alternative. But it should be possible to activate a plugin from configure.
(09:33:01 AM) Lavr: And the activation should include taking the default settings from Config.spec and putting them in the foswiki config IF THEY ARE NOT ALREADY THERE.
(09:33:34 AM) jomo: xactly (IF THEY ARE NOT ALREADY THERE(!))
(09:33:42 AM) Lavr: Upgrading a plugin should never overwrite the existing values. Just import the defaults so you can apply the default.
(09:35:05 AM) jomo: the only problem is now - when someone upgrades a plugin, and the upgraded plugin deprecated some config-value (e.g. the new version of the plugin, doesn't use some old config-value - it remains as a waste in the LSC?)
(09:35:52 AM) Lavr: Harmless waste I assume
(09:36:27 AM) jomo: yes... but still a waste (what is shown in the UI - afaik)..
(09:36:45 AM) Lavr: Can confuse the user because he sets the value and it does nothing at all
(09:36:56 AM) gac410: If there is no spec,  I don't think it shows in the UI, does it?
(09:37:28 AM) jomo: so, after the resave it disappear from the LSC? - then ok..  :)
(09:37:38 AM) gac410: And when the config is saved,  the values without spec get saved with a comment  # NO SPEC FOR ...
(09:37:49 AM) gac410: no it doesn't disappear.   
(09:38:04 AM) jomo: comment is ok too.. ;)
(09:38:14 AM) gac410: darn,  contractor just yelled "Where's George"    ..
(09:40:15 AM) ***jomo yelling: "Where's George?"
(09:40:21 AM) jomo: :D
(09:41:47 AM) andreli: https://en.wikipedia.org/wiki/Where's_George%3F
(09:42:04 AM) jomo: LOL
(09:44:36 AM) gac410: And I'm back.   Sorry about the interruptions
(09:45:12 AM) gac410: Okay.... I'll keep working on the (*#@$)  wizard 
(09:45:48 AM) gac410: The other blocker is Lavr's   http://foswiki.org/Tasks/Item13570     
(09:46:19 AM) gac410: you'd think that the TABLE and EDITTABLE being on separate lines would be easy to find.   but i'm completely lost down in CDot's new parsers.  
(09:47:19 AM) Lavr: I have also looked. I cannot find it either. I bet the whole issue and all the side effects has its roots from the TABLE/EDITTABLE issue
(09:47:41 AM) gac410: y  I agree.   
(09:49:16 AM) gac410: And last blocker,  another I want to review with CDot   is http://foswiki.org/Tasks/Item13598 ...   Store falls over trying to move symlinked web
(09:49:53 AM) gac410: But I'm not really sure it should be a blocker.   Symlinked webs more a developer thing isn't it?
(09:51:05 AM) jomo: i can live with the error - but i'm using the symlinked webs MOSTLY in production.. (but they isn't get renamed often)...
(09:51:20 AM) Lavr: I symlink all webs because it enables me to have a wikidata directory tree independent of the distribution.
(09:51:30 AM) jomo: xactly
(09:51:31 AM) gac410: Ah... good point.   
(09:51:39 AM) Lavr: That has been very useful when I ran the site on a release candidate
(09:52:12 AM) Lavr: But also I never move webs around. Maybe subwebs but they are in a directory below a symlinked directory
(09:52:47 AM) Lavr: I actually have to manually move directories and create symlinks when a new root web is made. Rare thing because I block all requests for root webs
(09:53:00 AM) gac410: :) 
(09:53:50 AM) Lavr: Surely not a release blocker kind of bug I would say
(09:54:31 AM) gac410: I agree.   ( I leave bugs urgent because there are so many Normals we overlook them )   I'd like to see the EditRow issue addressed for 2.0.2   Anyone recall when CDot gets back 
(09:55:19 AM) jomo: he said 2 weeks - IIRC...
(09:56:15 AM) gac410: So we have a week to go then. ... I'd like to keep on track for a 2.0.2 by beginning of September.  Whatever is fixed make it.
(09:56:15 AM) gac410: Next on the agenda ... feature plans for 2.1.0   and cleanup of the ReleasePlan.    
(09:56:59 AM) gac410: And unfortunately I can't stay.    I have to run to a Dr's appointment  in about 10 minutes.  
(09:57:19 AM) Lavr: And I have a hard stop in 2 mins
(09:57:56 AM) gac410: But it would be good if everyone could look through the backlog of proposals,  and consider what ones should be planned for 2.1   
(09:59:18 AM) gac410: Should we "park" any proposals by developers who have left,   or just remove them from the "committed developer" column?     Or hope that some come back now that 2.0 is released.
(10:00:26 AM) jomo: imho is pointless to have a commited developer - when they doesn't active long time ago...
(10:01:01 AM) jomo: long time - more than 1 year :)
(10:03:22 AM) gac410: I suspect that old "accepted" proposals maybe should have timers reset anyway.  Foswiki has changed so much, that many could be obsolete now.
(10:03:52 AM) jomo: agree
(10:05:17 AM) gac410: well anyway,   I need to run.   I'll leave my session open so that I can get the log.      Thanks everyone.    If someone wants to do any proposal discussions, please go for it. 
(10:10:04 AM) andreli: bye
(10:10:06 AM) andreli left the room (quit: Remote host closed the connection).
(10:10:22 AM) andreli [~andreli@2a01:488:66:1000:b01c:164e:0:1] entered the room.
(10:10:24 AM) andreli left the room (quit: Remote host closed the connection).
(10:11:18 AM) JulianLevens left the room.
(12:11:41 PM) MichaelDaum left the room (quit: Quit: quit).
(05:39:31 PM) jomo left the room.

Topic revision: r2 - 11 Aug 2015, GeorgeClark
The copyright of the content on this website is held by the contributing authors, except where stated elsewhere. See Copyright Statement. Creative Commons License    Legal Imprint    Privacy Policy