Feature Proposal: Add Testing and Archive repositories to Foswiki.org

Motivation

Currently Extensions are released into a single stream, with no ability to hold back major changes, or new significant dependencies. This can result in unexpected behavior when updating a site. We need a short-term fix to allow site admins a better opportunity differentiate between the "bleeding edge" and well tested stable releases of extensions.

Description and Documentation

Add an Extensions/Testing and Extensions/Archived subwebs to foswiki.org.

• This has been modeled in Sandbox/Beta

Extensions/Testing

Test versions, release candidates and experimental Extensions. There should be a time limit on how long extensions remain in this web. For now manually enforced with a "OldExtensions" search page.

Extensions/Archives

Obsolete, unmaintained, or old versions of plugins that need to be kept for backwards compatibility

Examples

Impact

WhatDoesItAffect:

%WHATDOESITAFFECT%

edit

Implementation

(Copied from previous proposal)

Site admin creates the Beta subweb under Extensions.

Copy the infrastructure topics over to the subweb as specified in HowToCreateALocalExtensionRepository

Update general documentation to call attention to the Beta web, and how to point bin/configure at this web.

Update developers documentation for how to upload to the Beta subweb

For notification of beta versions, can include something like the following at the top of the extension toipcs. Probably could be added by buildcontrib when building the upload along with adding the form. It wouldn't be useful to distribute the search with every plugin.

-- Contributors: GeorgeClark - 18 Feb 2010

Discussion

Discussion copied from previous proposal

sounds clear and simple to me. also, i am glad this would force us to test "private" repository setup and iron out any development and documentation issues.

-- WillNorris - 11 Feb 2010

Webs are best used as a sort of sub-wikis like a sub-domain in hosting language. I'd better not over-use webs for structuring content that belongs together as natural as stable and beta releases of the same extension. Instead, we should enhance the BuildContrib to allow to upload to the same extensions topic but attach the beta version with a different naming scheme.

Have a look at Drupal

-- MichaelDaum - 12 Feb 2010

I see a couple of issues with this alternative - the biggest one being that it doesn't address the extension documentation topic. As a site admin, before I download an extension, I'd want to review the Topic page, and that is hidden down in the distribution files. Also, with the attachments solution the bin/configure plugin installer wouldn't have access to the beta versions of the extensions, so we also need to rework configure to recognize that there are beta versions attached to a topic and present them as well. That adds a significant amount of work to bin/configure because now it needs to access the attachments of each topic instead of a simple search the Topics.

The Beta subweb solution addresses both the bin/configure interface and the documentation issue. And works within our existing BuildContrib and bin/configure implementations without needing any major redesign of both functions. The only option I'd like to add would be a change to bin/configure so that it can present both the production and beta versions in the list, instead of only presenting the version in the last repository. As our list of extensions grows it might also be nice to add a search keyword field in front of the [Find Extensions] button so the list can be narrowed.

One more thought is that you could hide the subweb structure from the user somewhat by adding a %SEARCH form to the Extension web topic that lists the beta version of the topic, if any.

This may end up being a case of the perfect being the enemy of the good, but to brainstorm an alternative solution

---

Another use for a Subweb might be an Attic or Obsolete category. Some plugins are abandoned or deprecated, and moving them to an attic would clean up our list of extensions in FindReport and improve the experience for users.

-- GeorgeClark - 12 Feb 2010

Contemplating the ideal of the simplest thing that could work to create a Test Staging area for Extension developers to upload and test that they a didn't upload a zip with missing files b didn't make some other simple mistake c for us as a community to find out if the more complex efforts dreamt up here are even worth doing

• • basically, if only one or two developers make use of it, there's no point expending development time creating a fuller solution

I think Creating a ExtensionsStaging web is the best answer. And here's how I'd do it.

1. create the new web with the FastReport etc in it
2. limit access to that web to users in an ExtensionsStagingGroup
3. allow developers to add an extra entry into their $Foswiki::cfg{ExtensionsRepositories} = 'Foswiki.org=(http://foswiki.org/Extensions/,http://foswiki.org/pub/Extensions/)';= setting in =configure
   • either before, or after the standard Extensions web, depending on the need to test a new Extension with the released Extension set, or to over-ride one.
   • For even more advanced uses, the person that needs it can add filtering URLPARAMs to the Staging area's FastReport.
4. bonus points for removing Staging area Extensions when a newer version is uploaded to the public Extensions area.

This would mean that Michael's small BEGIN monkey patching bug in AddToZonePlugin could have been noticed before it went 'live', while still enabling him to upload it fast.

I am assuming that configure's ExtensionRepositories code has support for authenticated repositories....

If the simple solution is still rejected, I offer a Web on one of my Foswiki installations for the purpose.

-- SvenDowideit - 14 Feb 2010

If only a couple of developers use the concept of a staging or beta or whatever web, I agree that it will be of limited use. I've set up my own staging area and have indeed caught some packaging errors. But what I really wanted was a public space where others could have access to RC / Beta / Testing extensions. If it's limited to developers who can fetch from it, then it becomes even less use.

Actually of more use to the overall community would be a Obsolete/Unmaintained web as well - to put the cruft that seems to grow over time. It became a real problem at T.O, and will eventually get there with us. ToPDFPlugin for example has a banner saying it's unmaintained, and other plugins would be a better choice. It might be a candidate for a Obsolete category.

Point 2 - limit access to ExtensionsStagingGroup to me kills the usefulness. I can test staging on my own server. I want to find a user who can install/test GenPDFAddOn, or give a user who reports a bug and easy place to find/load the fixed version for testing before it goes live.

Point 3 - add an entry to ExtensionsRepositories in bin/configure - yup - but I'd like anyone to be able to do this if they are interested in trying a beta quality extension

Point 4 - good one. Needs to be kept clean or it will become yet another mess.

As much as I wanted the subweb / simple solution, Crawford's points on the ExtensionsGateway pages listing multiple revisions, would have good value. Although cleanup of old versions over time might become an issue.

-- GeorgeClark

in that case, you're up for designing a complete solution - as I pointed out on IRC, if you have a non-limited access to unstable code, then you will need to deal with the very vocal minority that insist on installing them on their production systems, not knowing, accepting or understanding the risks.

and for that, I too think we're too small.

Note that ExtensionsStagingGroup does not mean its limited to people with svn access, its a globally visible commitment showing that you've accepted the risks of using that repository - anyone can intentionally join that group - and hopefully it gets mostly subscribed to only by people that can give useful feedbacks.

Now that I think about it, we wouldn't need to do point 3 - just amend the existing FastReport to include the Staging web - using the Group permission to show / hide the optional bits.

-- SvenDowideit - 14 Feb 2010

i created Sandbox/Beta/ as a playground and prototyping area for the simple, original version of this proposal. there's no need to wait to build some ivory tower when we can solve our immediate needs in "5 mins" (though i like the idea of more extensive support for multiple versions like drupal has).

-- WillNorris - 15 Feb 2010

One issue with uploading an existing extension into the Sandbox/Beta web - the PackageForm is not recovered from the version in the Extensions web. Workaround for now is to copy the Extension topic into Sandbox/Beta before running build.pl upload

-- GeorgeClark - 16 Feb 2010

Today 4th is 14th day. Any last minute concerns before I declare accepted?

-- KennethLavrsen - 04 Mar 2010

We have had a discussion. No concern. The original spec sill stands. Accepted by 14-day rule and I would say also by consensus after a good discussion.

-- KennethLavrsen - 09 Mar 2010

---

how do we deprecate plugins from the debian repository?

http://fosiki.com/Foswiki_debian/pool/main/f/ contains TinyMCEUsabilityUpgradePlugin, which should be moved to the deprecated extensions area. http://fosiki.com/Foswiki_debian/pool/main/f/foswiki-tinymceusabilityupgradeplugin/ (as well as the typoed http://fosiki.com/Foswiki_debian/pool/main/f/foswiki-tinymceusabiltyupgradeplugin/ version) are in the current debian repository.

Tasks.Item8088 describes an old problem with a soon-to-be deprecated plugin. not only do i want to close that bug, i'd like to prevent someone filing a bug again in the future after they've installed the deprecated plugin from the debian repository.

-- WillNorris - 19 Mar 2010