Feature Proposal: Ongoing upgrade of minimum perl version

Motivation

We discussed RequirePerl588 a few years ago. We need to review that periodically or agree a schedule if we can.

Description and Documentation

We do not want to maintain support for older perls in general. As time goes by the definition of old perl changes.

We can raise similar feature proposals to re-examine the issue every few years of tie ourselves to a schedule based on the most conservative Enterprise distro we choose.

Examples

Impact

WhatDoesItAffect:

%WHATDOESITAFFECT%

edit

Implementation

• Add "use 5.10.1" to Foswiki.pm from March 2017 (for the next Major FW release)
• configure?
• Document requirement in installation notes

-- Contributors: JulianLevens - 19 Dec 2015

Discussion

We have been here before and initially I was simply going to propose RequirePerl510, but RHEL is the laggard and we need arguably need to wait until RHEL 5.11 goes EOL (end of life) at that time.

Similarly, in November 2020 RHEL 6.7 goes EOL and we can then positively leap to perl 5.16.3 with the next Major FW release at that time.

Indeed, this FP should be been as proposing that we tie our minimum perl version to the oldest perl version in the oldest active RHEL.

We style ourselves as an Enterprise wiki and alas we need to be conservative in the same vein that RHEL operates.

We cannot concern ourselves with every distro but RHEL is the major Enterprise distro (by far) and is therefore the best one to tie ourselves to.

The proposal is therefore not merely a one time change but to establish a FW policy in this area (and document said policy). This policy would always be up for change at any time by anyone raising an FP.

-- JulianLevens - 19 Dec 2015

Another choice would be to leave an older Foswki release as "stable" and continue to backport only "urgent" bugfixes. It adds to our maintenance efforts, needing to double-maintain an older release, but frees us up from waiting for Luddite RHAT. IMHO waiting another 5 years to move to a more modern perl is quite limiting. How many of us will still be around 5 years from now. CPAN packages are just as big an issue, requiring acrobatics to support newer features (Like the CGI move to multi_param(), and for older versions, param().)

Just because an old RHEL is supported for 10 years, there are newer upgrades available, and the enterprise has a choice to upgrade, or remain on old code. If they remain on old Perl, what's wrong with us leaving them on old Foswiki. It's their choice.

I'll go a bit further and suggest that for a policy:

• Each New Major Foswiki Version should support the lowest common Perl for the most recent long-term-support versions of the major distributions.
• Foswiki will provide urgent bug-fix support for the prior release for sites unable to upgrade.
• In other words, the last minor.patch release of Foswiki will be our "LTS" release, and the next major release will step into the next generation.

-- GeorgeClark - 19 Dec 2015

The EOL means:

• we at RedHat promises: this version will be supported by us till...
• and not: We at RedHat forbid you to use anything better - omg

RedHat itself knows the problem , therefore introduced the RedHat Software collections - RHSCL.

For the RHELL 6 and 7 currently available perl is (pretty fresh) 5.20/docker. read here about the perl tag .

So, strongly disagree with this FP and especially with the "policy" part. The policy (i doubt we even need such thing) should contain: The recommended perl version should be the most recent perl which is available and by official RHSCL.

I know nothing about the RHSCL for the RHEL 5.11. If doesn't exists newer perl for the RHEL5.11 we could (as last resort):

• continue to use minimum 5.8.8 till the new OO system (and/or PSGI) comes into life (my codename FoswikiNG) - guessing - approx. in 2017 - e.g. at the EOL (2017-03) for the RHEL 5.11)
• focus the development to FoswikiNG
• the FoswikiNG should be dependent on the RHSCL/docker perl version (and not on the basic system-wide installed perl version)

-- JozefMojzis - 19 Dec 2015

Jozef, I would love to jump us ahead to a much more recent perl.

However, I will play devil's advocate

I understand that RedHat does not forbid you to use anything better. However, won't some employers have a policy of using the base components as much as possible? Yes you can in various ways use different components but doesn't that run some extra risks that some employers will frown upon?

Just because RHSCL provides a supported option to use a later perl, I suspect that many employers may not allow the use of that option.

The reason to establish a policy is to avoid this repeating issue which takes up valuable developer time better spent on new developments. If the policy becomes outdated anyone can raise an FP to ask for it to be amended.

I like George's policy suggestions:

• Each New Major Foswiki Version should support the lowest common Perl for the most recent long-term-support versions of the major distributions.
• Foswiki will provide urgent bug-fix support for the prior release for sites unable to upgrade.
• In other words, the last minor.patch release of Foswiki will be our "LTS" release, and the next major release will step into the next generation.

• Ubuntu 14.04: perl 5.18.2
• Debian 8.0 jessie: perl 5.20.2
• RedHat 7.2: perl 5.16.3
• SuSE Enterprise Linux 12: perl 5.18.2
• CentOS 7-1511: perl 5.16.3
• Gentoo 20140826: perl 5.18.2
• FreeBSD 10.2: perl 5.20.2

Note that assessing what constitutes the latest LTS version is not easy. Some of the above are not 'Enterprise' distributions (e.g. Debian). This still leaves RedHat as the laggard but at least 5.16.3 is a significant step forward.

The question now is do we 'use v5.16.3' from Foswiki 3.0 or even (because we just missed the chance in 2.0) in Foswiki 2.2? Don't the recent Unicode changes somewhat force our hand?

For a policy to it needs to be simple to assess. I would suggest that we use RedHat as our base distro for this, only now the perl in the most recent RedHat Enterprise release.

-- JulianLevens - 20 Dec 2015 - 18:17

Julian, George, maybe i do not understand fully right the proposal and the comments.

Based on the currently known informations e.g. version numbers and known dates (from the distrowatch) - could you please fill the following table?

date from	date to	the perl version allowed to use in Foswiki for CORE development
Jan. 2016	mm.YYYY	5.x.y
mm.YYYY	mm.YYYY	5.x.y
mm.YYYY	mm.YYYY	5.x.y

etc?

Cherry picked some missing features - e.g. currently could not use them because low perl...

• introducted in 5.12
  • Implicit strictures - automatic "use strict"
  • many UNICODE "things" - working \p{prop}
  • working \X constructs in regexes (but see bellow - 5.22)
  • the charnames pragma
  • the parent pragma(!)
  • package NAME VERSION - eliminates the $Package::VERSION strings.
• introduces in 5.14
  • the non-destructive substitution /r regex modifier (!!!)
  • Full functionality for use feature 'unicode_strings' (solves the unicode BUG (!))
  • the /a and the /u regex modifiers
• in 5.16
  • the unicode_eval and evalbytes - enabled by default in use 5.16
• 5.20
  • the use locale finally works with UTF8 locales.(!!!) (unfortunately, still no sort support)
• 5.22
  • the qr/\b{gcb}/ and qr/\b{wb}/ New \b boundaries in regular expressions for the Grapheme Cluster Boundaries and Word boundaries...
  • non capturing regex-modif /n (finally not need to put ?: into every capture group for "do not capture")

Nearly each major version brings the support of the latest UNICODE specification. Currently the latest unicode is 8.0 - introduced in perl 5.23. (5.14-unicode 6.0, 5.16-unicode 6.1, 5.18-unicode 6.2, 5.20-unicode 6.3, 5.22 - unicode-v7).

Also, from the above list are missing the many bugfixes for bugs introduced in lower versions and corrected later.

From the above - (my me) the 5.14 is the absolute minimum and the 5.16 is the really usable minimum.

-- JozefMojzis - 20 Dec 2015

I probably would wait until 3.0 to change a minimum perl version. 5.16.3 sounds really good. I'm sure there will be objections though. As far as using new perl syntax, I'm less inclined to jump onto those changes, if for no reason that they can be obscure and make code harder to understand. So use but with good reason. One really important fix though is the performance of case insensitive regexes, fixed in 5.22 or 5.23 ... can't recall details.

-- GeorgeClark - 21 Dec 2015

Well, I'd propose 'use v5.16.3;' from FW 3.0.

This does suggest to me that we'll need another branch in git. What should that be called: major; release_3 or something else? How would we manage that git flow in addition to our existing git flow, which itself needs documenting.

As George states we would then maintain FW 2.0 with perl 5.8.8 as a minimum version. This raises the question of how long we maintain this release - how do we determine it's end of life? I believe our maintenance of 1.1.x is either EOL or very near to with 1.1.10 hopefully being the last version in that series. Alas, the step from 1.x to 2.x is substantial and some bug fixes may be critical. That potentially leaves us with 3 releases to maintain, I would hope we can work with that if it buys us a much better perl for FW 3.0.

Jozef, your little table can be filled in (with adjustments) as follows:

FW Major	date from	date to	the perl version allowed to use in Foswiki for CORE development
1.x	Now	Now-ish	5.8.8
2.x	Now	July 2020 (5 years from 2.0 release)?	5.8.8
3.x	Now	5 years after release?	5.16.3 — in branch_3
4.x	Now	5 years after release?	5.16.3 or later — depends on when we branch_4, i.e. when 3.0 is actually released check distros as above and see what we can set

When we EOL a release is open to discussion. If we managed a new major release a year and had a 5 year LTS we could end up with at least 4 releases being active and needing maintenance. I suspect in practice we'll be closer to every 2 years and that will hopefully prove to be manageable.

Developers will have to worry less about issues in older perl (esp. wrt Unicode) but with probably one more release to track. I think that's a good trade-off.

-- JulianLevens - 22 Dec 2015 - 13:35

For branching strategies, master serves as the tip of our next Major / Minor. I don't believe that we need to add more branches - at least I hope we don't, as it's pretty confusing as it is.

• Feature branches split out and merge back into master - All shown above the master
  • Branches may be "conceptual" ... something like a "use perl 5.16" would most likely happen directly in master.
  • Features are added only into the master branch
  • The git "merge" command is used to integrate feature branches.
• Release branches split out from master, but never merge back in.
• Development ceases when a branch ends with an █.
• While a branch remains "active", bug fixes have to apply to any "active" branches.
  • Usually by cherry-pick between the branches. We cannot use merge.
    • Merge from master to a release branch would pull in features we don't want.
    • Merge from release back to master would cause issues with the translation files, which are maintained in parallel by weblate.

PROPOSED BRANCH STRATEGY
                                                                ┌──Feature Perl516───┐
       ┌─────────────feature3───────────────────────────┐       │                    │
       │     ┌─feature2───────────────────────────┐     │       │                    │
       │ ┌───────UNICODE────────────┐             │     │       │                    │
       │ │   │                      │             │     │       │                    │
master─┵─┵───┵─┭────────────────────▼──────┭──────▼─────▼──◆──┭─┵────────────────────▼───┭───────────────────>>
               │   ↕         ↕             │                  │                          │
               │                           │                  │                          │  ↕
               └───Release01x01─────█      │   ↕       ↕      │   ↕                      │
                                           │                  │                          └──────Release03x───>>
                                           └───Release02x00──────────█                      
                                                              │   ↕              ↕          ↕ 
                                                              └──Release02x01────────────!───urgent bug─fix─>>
 ◆ Today (22 Dec 2015)
 ┵ Branch to feature
 ▼ Merge in a feature
 ┭ Branch to a Release
 █ Obsolete, all development stops
 ! Stable - routine bugfix stops.
 ↕ Cherry-pick fixes

So if this is our time-line

• Release01x01 branch has ceased development. No more cherry-picking or merges.
• Several features have been already merged into master - UNICODE, feature2 and feature3.
• The ◆ symbol shows where we are.
  • Release02x00 is being maintained in parallel
  • We have not yet branched Release02x01
  • Bug-fixes have to be cherry-picked between master and Release02x00
• Once we branch Release02x01, there is a short period of time where "triple maintenance" happens. master, Release02x00, and Release02x01
• At some point - hopefully not too far out, maintenance stops on Release02x00 (maybe after 2.0.4)
  As far as a major changes, examples being the UNICODE core, or a requirement for Perl 5.16, still branch/merge into master. The implications of our policy, is that an older release will need to remain 'alive' for sites that are unable to adopt the next major release.
• When Release03x00 releases, Release02x01 becomes "stable", getting urgent bug fix only.

The big issue that we currently have not addressed. We have no way to release older versions of extensions that remain compatible with older Foswiki. We either have to back-port critical features (like we did in 1.1.10), or the extensions leave the old releases behind. It's not a git problem, as we could easily branch extensions just like we do with distro. It's the Extensions web and installer.

Does this all seem reasonable?

-- GeorgeClark - 22 Dec 2015

For me, this sound as-so ok - e.g. for major release we could raise the perl version - e.g. the development process for the major release could use the (some) new perl features.

But still: p5p releases new (stable) perl each year, e.g. 5.18/2013, 5.20/2014, 5.22/2015 - so, this year we will get 5.24. Now FW have min. 5.8 (e.g. currently we late 7 (8 soon) stable releases (10,12,14,16,18,20,22, (24)).

The 3.0 will be ready no sooner as 2017/2018 - e.g. when we'll release FW3.0 (with 5.16) the world around (except RHEL) will use 5.26 or 5.28 - so, we will late 5 (or 6) stable releases (18,20,22,24,26, (28)). Slightly better than the current 7(8) - but still very strange.

Bottom line:
Anyway - who knows - maybe in 2018 we will skip FW4.0 and 5.0 too and will release Foswiki 6.0 with perl6 (and perl6-ooo moose-like - with meta-protocol and using p6sgi ) - okay, okay, just kidding...

-- JozefMojzis - 07 Jan 2016

Commenting is disabled while not logged in