Divide and Conquer

Or how to break Foswiki into (more manageable) pieces by using PSGI.

I found some messages in the IRC. Based on my own past experience, I found that it is better to write an topic as argue directly in the IRC. (because of my bad english). Hopefully the reader will read this topic more times AND and will think deeper about it's content. Sorry, it is an LONG topic. So don't even start reading when you're haven't at least an free half hour. (better more)

Motivation

Me always was astonished, why foswiki.org contains so many articles about some well-known "things". Especially, when the knowledge about such "things" can be obtained by reading some articles on the web which (usually in more details) explain the "thing". Now I understand this.

Even when here is exists the topic QuoVadisPsgiFoswiki (and exists zillion good resources on the web) - here are still some really deep misunderstandings about:

• why the PSGI is even created - so what is it's PURPOSE
• how the PSGI works
• what are the main benefits of the PSGI
• and how the PSGI application should be designed (and why that way)

Reading some recent posts on the IRC, it is clear that in members are still prevalent the following opinion on the PSGI-fication of the Foswiki:

• we want allow to run Foswiki under the PSGI environment
• so we need to develop some another Engine::PSGI (or similar "driver")
• and when this will be done - we have resolved all issues regarding PSGI and "hooray, we could marketing our PSGI Foswiki...

Nothing could be more far from the truth as this. Unfortunately, to understand why the above view is wrong, really need understand how PSGI works. And therefore i created this second episode in the PSGI-saga.

I appeal to readers to open his minds and please, read carefully. Really carefully, do not skip even the "i know this already" parts, because the text-flow builds on the previously defined sections. Also, please, try understand the code fragments and images - they're sure more clear as my broken english.

PSGI in details

The Book of GenesysApps

In the beginning was the CGI and and the CGI was with NCSA. (now known as Apache). The idea behind was so nice that it allowed to create many innocent web-based applications. Each of such application lives naked in the Garden Of Cgi until some application want be alive as long as the Apache, want communicate in more advanced form with the Apache-god or want to do similar evil things.

This was resulted in babylonian chaos where may languages tried to solve the communication and persistence my their own ways. In the perl-nation was created the mod_perl and/or mostly is used the language agnostic FCGI protocol.

The deployment problem

One of the big problems of the old-school web-applications is solve the "deployment problem" - e.g. how to enable/allow run the web-application and allow it communicate with the web-server in many-many different web server environments, like CGI, mod_perl, FCGI (and to be more complicated, here is Apache1, Apache2 ..., and other web-server sw).

To handle the evident discrepancies, the web-app developers started to use a concept of "drivers" or "engines" in their web-applications. And because the majority of the Web-Application is developed using CGI, these drivers only does some translation work from the actual environment to the CGI (to allows continue using CGI-based routines in the web-applications. (like: $query->params and such). This is the Foswiki's case too. For the nitpickers, of course, here are pure mod_perl based apps too, which doesn't uses CGI at all. In this case the subroutine calls allows the data-exchange between the mod_perl and the web-application. I called it under the umbrella term - "communication"..

The method how the "main part of the web-application" and the "engines" communicating is purely application specific. (read: "invented" and developed by each app-developer). Let's call it: *Engine's API*. This is the Foswiki's case too. (even if the Foswiki::Request is still ISA CGI). So, the above image could be redesigned to:

Of course, anyone could draw more different schemes how the web-applications are developed and how they solving the "deployment problem".

Therefore in the image

• i grouped all the "engines + dispatcher" into one gray-box
• and explicitly showing the Engine's API.
• Also note: In the image is changed the "CGI routines" to the "internal routines", because many web-apps doesn't uses CGI-based routines anymore. (for the Foswiki they're still CGI)
• Important: All the above is an integral part of the web-application. (like in the Foswiki).

Despite of the definition of some Engine-API and having "engines" the deployment problem is still hard to solve. Here are too many differences what the application must handles.

Moreover, the application developer could not concentrate his development efforts to the main part of the application, but must care about (and maintenance) the "engines" too. For example: if someone want replace the FCGI and deploy the app in SCGI environment, the web-app developer must develop and support another "engine" for the SCGI.

Need to say: Here is nothing wrong with the above. It provides some level of (internal) abstraction and works OK for many different web-apps for example for the Foswiki too.

The PSGI misthinking

Many developers heard something about the PSGI specification. Like: The PSGI is great, because:

• it replaces the CGI
• could be used with pure-perl web-servers
• it speeds up the development
• lowers the needed maintenance efforts
• it is sexy, cool, modern
• and ... (substitute some other adjectives)
• it is some Perl Superglue for Web frameworks and Web Servers - wow, sounds cool - whatever it means..

So, they decide: I want allow to run my super-duper web-app in the PSGI environment too.

Without any deeper thinking about the real power of the PSGI and learn how the PSGI works, decides:

• i will add some Engine::PSGI to the engines grey-box
• and because I have my own internal API I doesn't need change anything more, so - i will be ready
• (And hopefully i will be sexy too)
• (and also magically will gain some low-maintenance and fast-development ability too - as pure byproduct).

With the picture:

Using this logic, YES the application will be runnable under any PSGI server.

The PSGI's power - Divide and Conquer

Move the ENGINES

Let's repeat one of the above images:

The whole scenario contains TWO parts

1. the web-server - which has DEFINED (already configured) some communication method - like CGI, FCGI, mod_perl (PSGI too?)
2. the web-application - which does the real work (and uses internal "ENGINES" to solving the "deployment problem")

Why adding the PSGI into the "ENGINES" buys nothing?

Because, in the reality here is no such thing as web-server communicating with PSGI

Surprised? Cool. Even if in the common talk are used for example "Starman: PSGI-web-server". Yes, it is PSGI-WEB-SERVER - will explain later. The very common misunderstanding about the PSGI is, thinking about the PSGI in the same level and place as were are the CGI, FCGI, SCGI, mod_perl.

Because the PSGI "sits" elsewhere.

Again, try to explain with image:

Please, please, please compare the two images. The ENGINES

• are the same
• and doing the same things (translates the web-server environment to the "Engine's API" (called as PSGI-specification)

Just the "ENGINES" are:

• moved outside of the web-application
• their implementations are renamed into the Plack::Handler:: namespace ( in the PSGI spec are called as PSGI-Container or PSGI-Server )
• and the previously in-house invented Engine's API is become to well-defined PSGI .

The PSGI specs allows the developer divide the application into one more part - so in the result, we have 3 different parts:

1. the web-server - which has DEFINED (already configured) some communication method - like CGI, FCGI, mod_perl
2. the main web-application - (cleaned - it doesn't contains any specific "ENGINEs")
3. and the "ENGINES" part (which are moved outside of the web-application), called as "PSGI-Container" or "PSGI-Server" (in the Plack::Handler:: namespace)

On the other side, any developer

• could develop his own "engines" aka PSGI-containers
• Such containers could handle any discrepancies in any possible environment.
• Exactly, as in the case of the "built-in" engines.

But, usually they doesn't need develop anything. Because they're already developed. Just check the above picture again:

• on the "left" side of the PSGI-container - is an web-server with already GIVEN communication method
• on the "right" side of the PSGI-container - is the web-application using the exactly defined PSGI specification.

Will repeat myself (the PSGI basics are in the QuoVadisPsgiFoswiki) but for the reference. The PSGI application (BY THE DEFINITION):

• must be an code-reference. (It means something as: my $coderef = sub {.... }, or using any other way to gain an reference to subroutine.
• the PSGI container will call this code-reference. In the moment of the call, the app will get exactly one argument, the environment hashref.
• the called subroutine must return one array ref-reference. This array-reference must points to 3-members array: the status-code, the headers, the body. The latter could be arrayrefs again, (or I/O like object, for handle streaming).

In short - the PSGI spec in one line: my ($status, $headers, $body) = $coderef->($env)

So, now refresh the images from the QuoVadisPsgiFoswiki:

• the invoking model
  
• the connected model
  
• the standalone model
  

As you can see, the web-application is using ONLY PSGI and it is intended to work in any environment. It isn't about some "compatibility". Nobody says: The shoes are compatible with the legs. Because it is their sole purpose - be on the legs. And the leg's size differences are handled in the shoes, and not by the cutting from the legs...

The main PURPOSE, why the PSGI-spec is created - easily SOLVE the deployment compatibility problems. The PSGI app is runnable in ANY environment - because in ANY environment you can manage:

• start the perl-interpreter, (or better: have it persistently running)
• the perl-interpreter runs the needed "modules" from the Plack::* namespace
• which are executes the needed PSGI-container "(engine)" (note: before the web-app is executed)
• and THE given PSGI-container:
  • creates the $env hashref from the underlaying environment + mangles the I/O
  • calls the web-application's coderef (because it is a reference you calling the application indirectly - don't need to know the exact subroutine name)
  • convert (pass) the returned arrayref (or I/O handle) from the application to the "anything" what the web-server needs

When some says: "The STARMAN is PSGI-WEB-SERVER" it is true. Because it contains the the BOTH servers: The Web-server and the PSGI-server (container) too. So it looks like the web-server talks the PSGI, but the perl based web-server just calls the PSGI-container and this container talks the PSGI.

When saying environment, that usually means web-server, but not limited to. For example, allowing run the web-application from the command line, we couldn't talk about the "web-server" - this is just an "environment". The PSGI works again:

• the command-line utility calls the CPAN:Plack::Handler::CLI, and this handler
  • creating the environment to the PSGI-based app
  • calls the app
  • and returns the result (from the obtained arrayref)
• e.g. prints it to the terminal...

As a curiosity, here is already an existing Plack::Handler for the STOMP protocol (which has nothing with the HTTP).

The application based PSGI Engine - revisited

Of course IS POSSIBLE to develop some PSGI ENGINE and adding it into the Foswiki - only for the purpose to add the ability run foswiki for example under the Starman.

As already said: Using this logic, YES the application will be runnable under any PSGI server.

However, in the case of the web-application using CGI-based routines (like Foswiki) the above isn't needed at all! Foswiki runs flawlessly in PSGI environment currently!!! Enough to use the CPAN::CGI::Emulate::PSGI.

Again, one more image:

How it works?

• The PSGI-web-servers (let say Starman) "talks" PSGI (using the embedded PSGI container)
• The CPAN:CGI::Emulate::PSGI translates the PSGI-talk to the CGI protocol and executes the Foswiki (persistently)
• The result: FAST Foswiki which uses CGI only. (The speed is similar (maybe 2-3% slower) as in any other persistent-perl environment - like FCGI (because it uses tempfile).
• NO CHANGES NEEDED AT ALL TO THE FOSWIKI

Me personally uses the above in few my foswiki installation at clients what me manages. I could say: it is better and more stable as FCGI.

Reaction to some messages

I hope that now it is clear, how the PSGI works and what is it's BASIC purpose. So, about some IRC messages:

What jomo proposes means a user will need additional modules. What's worse – he would need them in hosting environments or it won't work. This is where engine come in handy..

The reference implementation of the PSGI - the Plack::*, is pure-perl so

• we not only could add the needed modules into the Foswiki's CPANcontrib
  • so they're will be in the same ./lib directory as the rest of the Foswiki,
  • and as already explained above - the engines does the same things as the Plack::Handler::* modules
  • so, the "Engines" aren't more handy in any way, as any of the PSGI-containers.
• also, please remember: the whole PSGI is also fatpackable, (More about this later).

...whatever jomo dreams of might not be completely true and different PSGI wrappers for CGI, FCGI, mod_perl may happen to be less compatible than we would like them to be. This might make engines necessary to reduce those discrepancies.

If you call the PSGI as an "wrapper", the your Engines are wrappers too. Because the Engines does exactly the same thing as the PSGI-container. The PURPOSE of the PSGI (aka, the reason why it is developed) is: handle the discrepancies in the different deployment environments in way that the developers doesn't need care about the environment, because he needs follow ONLY the very simple PSGI specification.

I want remove any references to request and make engine pure I/O driver which doesn't care about core's architeture.

Don't need invent again one I/O driver. It is already given by the PSGI specification. Please, just follow it.

... Plus, I'm thinking of a test engine which would simulate working environment for unit tests.

Whoa - GREAT! Just you don't need develop it. Here is already an PSGI-application tester. Just look at the CPAN:Plack::Test.

From the docs: Plack::Test is a unified interface to test PSGI applications using HTTP::Request and HTTP::Response objects. It also allows you to run PSGI applications in various ways. The default backend is Plack::Test::MockHTTP, but you may also use any Plack::Handler implementation to run live HTTP requests against a web server.

vrurg: please understand, here is nothing personal against you. You're done an great amount of work in the OO. Just really needed to be clarified some misunderstandings about the PSGI.

More Divide (and Conquer too).

One Container to rule them all - aka the Builder

Another thing, where the PSGI does great job is it's mounting capability.

Any PSGI app, is must be callable from the app.psgi!

You already know (QuoVadisPsgiFoswiki) - the whole app.psgi just returns out wonderful code-reference. E.g. the PSGI foswiki MUST be usable somewhat as the following:

#this is app.psgi file
use Plack::Builder;
use Foswiki::Something;
use Some::OtherApp;
my $foswiki = Foswiki::Something->give_me_the_code_reference;
my $otherapp = Some::OtherApp->to_app;
builder {
    mount '/wiki' => $foswiki;
    mount '/' => $otherapp;
}

I hope that it is clear than the "builder" and "mount" are just an subroutine calls. The above code could be written as:

&builder( sub {
    mount('/wiki', $foswiki);
    mount('/', $otherapp);
});

Side note: The above to_app method name is just an "convention", some authors uses "my $app = MyApp->web_app" or like. Again, by the usual convention, if the web-application has also an command-line interface, the Some::App:: namespace is used for the command line application, and for the web-application is used something other like Some::WebApp or like... But it just an convention, doesn't must be followed...

So,

• from the PSGI-container could be executed more than one PSGI-web-applications (e.g. not only the Foswiki)
• but more importantly, the operation of the mount affects the PATH_INFO and the SCRIPT_NAME.
• However, the well-written PSGI-web-application doesn't need care about it.
• Because, It is done in the CPAN:Plack::App::URLMap, before the $foswiki, or the $otherapp is even called.
• The CPAN:Plack::App::URLMap is an dispatcher app, which just does the right things and the well-written PSGI-web-app doesn't care about the actual mount-point!!!
• So, when the Foswiki is mounted to the path '/wiki'
  • the actual request from the browser will be GET /wiki/bin/some
  • but the Foswiki doesn't need to care about it
  • because the CPAN:Plack::App::URLMap mangles the PATH_INFO to the correct string - e.g. to /bin/some.

It is like a sort of the rewrite, but which is done auto-magically in the PSGI-container, without any configuration. But this means, than the Foswiki MUST NOT care about this, otherwise either 1.) bad (double) PATH_INFO parsing will be done, 2.) or the Foswiki core will be unnecessarily complicated!!! So, we need simplify our code!

Of course, we could re-invent and re-implement the Plack::App::URLMap inside of Foswiki-web-app as a part of the proposed PSGI-Foswiki "ENGINE". But what is the point duplicating an well-written code and break the existing well-defined PSGI logic? (CPAN:Plack::App::URLMap will do his work anyway because the mount calls it.).

And this is just another one small example. Much-much more such examples exists.

The beauty of the code-reference

The code-reference is just one scalar. You can pass it to any other method, or you can call the referenced method. It even could hold reference to anonymous subroutines. They're great and powerful.

In the PSGI world the code-reference method allows break the application even more not only moving the engines outside of the main application. Yes, talking about the middlewares. Please, refresh the "The very important points of views" section in the QuoVadisPsgiFoswiki .

As already explained in the QuoVadisPsgiFoswiki the details, the middlewares acts as dual-faced code-references. And this is one of the main powers of the PSGI spec, because it allows the application layering. MANY, really MANY parts of the web-application could be moved to the middleware level. For example:

• session storage handling
• authentification
• access control
• viewfile-like file-serving (for example using X-SENDFILE headers and i/o like object in the CPAN:Plack::Response)
• many others...

While many such things are already handled in the Foswiki, the way how it is done means: they isn't "compatible" with the rest of the world.

For example, I have one installation, where I need parse the current Foswiki's CGI-Session files by another web-application. Such bastardized hacking could be eliminated by using the CPAN:Plack::Middleware::Session - which could be common for the all applications called from the PSGI container!

The same (and more importantly) is applies to the authentication. We already have the Development.UserAuthMapping2dot0 topic. We need clearly separate 3 things - into alone, more manageable parts:

• the registration
• the authentication
• the user-mapping (aka user -> WikiName)

Instead of developing our auth-mechanism, we could to just use already developed middlewares and focus our development efforts only to really Foswiki-specific mapping part. Moreover, using some Middleware::Auth:: modules allows common authentication for more (or all) web-applications called from the PSGI-container.

Etc..etc.. Such application LAYERING - e.g. when one subroutine calls the "more deeper" subroutine (by the code-refs) is one of the GREATEST thing how to make Foswiki more manageable. In the Foswiki, moving some parts to the middleware level allows more easily maintain and/or change such alone modules, hidden in the simple code-reference.

Of course, nothing breaks the Foswiki developers using pure Foswiki-specific "things" as middlewares, like:

use Plack::Builder;
use Foswiki::WebApp;
my $foswiki = Foswiki::WebApp->to_app;
my $old_foswiki_session_management = $foswiki->get_old_session_management_as_code_reference;
builder {
    mount '/foswiki' => builder {
        enable $old_foswiki_session_management;
        $foswiki;
    },
    mount '/' => sub { .... };
}

So, want the current session handling? Just convert it into the middleware and could continue to use it in the LAYERED PSGI-Foswiki. Using middlewares isn't only about the possibility to use already developed 3rd party middlewares, but also the method of of the layering!

Using the existing 3rd party middlewares could really GREALY reduce out code-base. Less code mean less maintenance. Using the unified PSGI for calling the midddlewares means: less spaghetti code. That's what i call:

• easier and faster development
• easier maintenance

Params, request and etc...

For example, the CPAN:Plack::Request already handles the environment in portable way. It implements 3 different methods for accessing

• 1. The GET parameters alone
• 2. The POST parameters alone
• 3. But also you have a method which contains the merged GET+POST parameters.

It is useful many times, but more importantly: all such methods returns the CPAN:Hash::MultiValue objects (Instead of CGI or like). So, for the request parameter let's call par, you should to use:

my $req = Plack::Request->new($env);
my $hm = $req->parameters; #merged GET+POST
my $par = $hm->{par}
#or just $req->paramaters->{par};

the $hm is CPAN:Hash::MultiValue object (which is special a bit, it is inside-out object).

This means: Foswiki could parse the query parameters by:

1. using the CPAN:Plack::Request object (as a part of the reference PSGI implementation) - as CPAN:Hash::MultiValue objects
2. or the Foswiki will use the current CGI-based query parser on the RAW $env->{...} PSGI variables.

But why for the sake of god WHY we want parse ourself, when the CPAN:Plack::Request already does good job?! Reusing existing routines from the Plack:: namespace (which are already got always loaded in the PSGI env) means REAL REDUCTION of the development efforts.

The CPAN-modules problem again

I couldn't resists and must says some about this:

What jomo proposes means a user will need additional modules. What's worse – he would need them in hosting environments or it won't work.

Once was cleared the ALAPC. (As Little As Possible Cpan). Now again it is used as an "argument". Therefore, again:

I want to use more modules, because using (well tested, reliable and highly rated) 3rd party modules means bring other (highly rated developers) into the project. Indirectly, just by using their well-written modules.

Me using perl many years. I thought, that I "speaking" fluently perl. Util I come to StackOverflow figured out that my knowledge is only dwarfish in compare with some true perl-monks such as: Ikegami, Borodin, Choroba, brian d foy and many many others. Also reading some CPAN modules sources helps understand some advanced facts about the "right-way" (despite the TIMTOWDI, some consistence isn't wrong). Just check the modules developed by: DAGOLDEN, TOBYINK, ETHER, JSWARTZ, SPROUT and again, many-many others. Myself sure isn't such high-rated developer, but maybe I could add some knowledge and/or alternate view to the team's "pool".

So I'm absolutely sure, that using well-selected modules (developed by such high-grade developers) will HELP to the Foswiki. (Please note the "well-selected". Some CPAN modules are crap, but many are great).

Back to the "modules": We need to use such modules - because they:

• shortens our own codebase, which (in exchange) means:
  • less maintenance
  • and more time to make Foswiki more robust and better
  • and instead of the maintenance efforts have more time for developing new features.
• by using their API sets up some margins on how our code should be developed. Using such well-established APIs is good again, because we are forced to follow good practices and also, we could learn some new philosophy how to make things done.

Want example? In the last few years I never used the open, close, opendir, closedir, readdir nor any file-specific perl rouutine. Me just using CPAN:Path:Tiny. It saves for me approx 5-20 lines of typing, decreasing test-coverage needs and it just DWIM. That's what i called as development speedup. BTW the same is applies to CPAN:Moose - by using Traits for attributes.

An well-written PSGI application will use the fast XS-based modules (if they're already installed in the server) but otherwise, it should be fatpackable (using the CPAN:App::FatPacker). This means that all CPAN (and local) dependencies are bundled into ONE big file, so the whole Foswiki code-installation (of course, not the data files nor templates) could be reduced to:

curl http://foswiki.org/fatdistro > foswiki.pl
perl ./foswik.pl
#done, without the need of installing any perl "RPM" package for the given distro :)

Of course, here are constrains - nothing is free

• BEGIN blocks == problems (thank's vrurg great work, they're already gone!!!)
• Dynamic module loading (constructed requires) - needs manual tracing (doable)
• No XS-based modules. (But probably every XS-module has its pure-perl counterpart)

Even more code-reduction

Usually, the web application didn't uses directly the LOW-level Plack::Request and like routines. Also, implementing the

• configuration management
• importing cache handling routines (for example the CPAN:CHI)
• importing the loggers
• etc...

are usually common for many web-applications.

Therefore are developed some FRAMEWORKS on top of the native PSGI. The web-application developers uses such frameworks, for easy and fast web-app-development. Foswiki could also benefit from such frameworks. Just need choose one actively developed PSGI framework. But this mean - boil the ocean and fully redesign the Foswiki. ;).

Will be the result still Foswiki? SURE YES. because it will still handle the

• macro-language
• data-forms
• the wiki markup
• and all other current features.

just with more robust, smaller and cleaner internal structure. And this is, what really counts.

Last citation from the IRC.

(09:49:58 AM) gac410: .... But I cannot imagine that we will just discard the past and start over. If you have any hope of OOFoswiki shipping as a 'next release', then we have to have some compatibilty

(09:50:24 AM) vrurg: gac410: Not without some code changes.

IMHO, isn't bad breaking the backward compatibility, until:

• here are some helper routines for data-conversion (if needed)
• the changes are clearly documented and having some docs how to address the problems
• and most important(!!!) such backward incompatibility changes AREN'T INTRODUCED in every release!!!

End-notes...

We really need think twice about:

• want the Foswiki really the PSGI?
  • pros: really could help reduce our code-base size (good for us), powerful, sexy, modern, and most important: it means EASY deployment.
  • cons: it needs boil the ocean.
    • The TOPIC content level compatibility could be achieved.
    • But not the extensions-based compatibility. They're need to be changed.
• so if yes - and we want PSGI (i hope), it should be well-done by using and accepting the PSGI's "way-of-life". And do not do some half-bakery, or dog-cat solution, because the half-baked PSGI will mean only
  • more work (e.g. we will need manage ONE more engine, manage more code)
  • without the main benefits of the PSGI.
• So, for the Foswiki X.0 we (IMHO) should:
  • divide the Foswiki to layers
  • kick the CGI::
  • kick all engines
  • start using Plack:: inside of the Foswiki
  • optionally: select an good, well-established PSGI framework
  • choose the right collection of the CPAN modules - some of them:
    • CPAN:CHI
    • CPAN:Log::Any
    • CPAN:Type::Tiny
    • CPAN:Path::Tiny
    • CPAN:File::Find::Rule
    • CPAN:Method::Signatures::Simple
    • CPAN:Import::Into
    • CPAN:Config::Any
    • CPAN:CSS::Sass
    • CPAN:Imager (less problematic as Image::Magick)
    • CPAN:Ref::Util
    • CPAN:ListUtil and CPAN:ListMoreUtil
    • and much MORE ...
  • focus on the PSGI (together with the MoreOO - FOOswiki)
  • move some parts of the foswiki to the middleware level (or even better, replace some parts of the foswiki with CPAN-based middlewares)
  • create an good docs about how to rewrite the extensions - and generally clean-up the docs
  • fatpack
  • win

OK, the last 2 bullets = a bit of dream - but...

argh..,

-- JozefMojzis - 09 May 2016

Jozef, I'm still reading this topic – you made it really big. You have the regarding the engines. But you miss another point: we're quite limited on incorporating new packages into the project. This is both about hosting environments and linux distros with strangely formed set of available packages.

So, briefly, the point is that I cannot get rid of the engines until Plack::Handlers are confirmed by the community. Anyway, your graphs are a bit incorrect because for now CGI engine, for example, is simply a collector of information and handler for (aha, aha) 3-element PSGI compliant return array. Yes, it duplicates corresponding Plack handler but it's not much choice we have.

Anyway I'm drifting toward the point where the core becomes full PSGI compliant and if there be engine for PSGI – it shall nothing but a mere transparent compatibility layer.

-- VadimBelman - 10 May 2016

I'm going to go a bit off topic, Jozef, you keep saying that we do not want CPAN That is absolutely not true. You want "good CPAN" while we want "Widely available CPAN". it's NOT that we object to CPAN, we want to use CPAN that is:

• Available in the more popular "enterprise" distributions. Centos, RedHat, Debian, Ubuntu, SuSE ...
• Easily installed by inexperienced admin.
• Resolvable as a dependency ... assuming we ever get the foswiki debian / rpm packages being generated.
• Security maintained outside of Foswiki, so we don't have to take on responsibility for tracking/releasing security patches for bundled CPAN.
• Available on hosted web sites via cpanel or other hosting tools.
• Preferably available in pure perl so that it can be bundled in the CPAN contrib for sites that still cannot install CPAN.
• And if that's not possible, make sure Foswiki is still functional without the module installed.

Saying we don't want CPAN, over and over, does not make it true. One of the big hits against Foswiki is the difficulty of the install. Picking out the very best obscure CPAN that can't be installed as a simple package dependency just won't fly.

So please. We don't object to CPAN, Don't pick the very best, just choose wisely: If you propose a module, review the distributions, and make sure it can be readily installed.

-- GeorgeClark - 10 May 2016

George, trying to write again and even more precisely what i already said above - my view on using CPAN + Foswiki

• Foswiki should
  • use only PP (PurePerl) modules by default
  • and IF (when) the (faster,better XS variants) are installed on the site, should use them (automatically). Otherwise, should use the PP variants only. Like for example does the Moo for the Moo's XS-accessor etc..

Using the CpanContrib you don't need install any CPAN module on the deployment site.

• We could bundle all needed CPAN modules. (omg, there JUST perl - no magic)
• And we probably should bundle the CpanContrib - because what if one day RH changes his mind and removes for example Moo ? (if even RH have package for the Moo currently - IIRC it hasn't).

As I already said too, if the Foswiki will use PP only modules,

• the WHOLE foswiki could be fatpacked even into one single file!
• mean one single file which will contain
  • the all packages (.pm files) from the Foswki:: namespace
  • plus all needed packages from the CPAN
• of course, the fatpacked Foswiki working best on persistent perl environment (like FCGI)

So, i not really get the "available" and "installable" arguments. Because

• if we want have Foswiki easily installable
  • we shouldn't boring the users with the CPAN (nor even with the yum, rpm or anything such)
  • on the hosted env, the user only should upload the Foswiki
  • and RUN IT...
• if we want really reduce our codebase to have an clean, spaghetty free, nice, small codebase /add some other adjectives :)/
  • we should us CPAN (and the EXCELLENT category modules)
  • you know this also (just don't says it) - than here are CPAN modules which solves things much better, faster, and more elegantly as Foswiki's inhouse developed modules...
  • we just need use them.

That's all about the CPAN.

Anyway, the whole point of this topic was elsewhere!

The most important part isn't on the CPAN usage. The major points was:

1. move the "engines" to their right place - outside of the main Foswiki application. if you read the above, you know,
   • we even could continue to use our Foswiki::Engines (if we want)
   • just they SHOULD be moved outside from the main Foswiki web-app.
   • and such "move outside" is easy, when the Foswiki will use the PSGI interface...
2. Start USING the the modules from the CPAN:Plack package inside of the Foswiki (e.g. CPAN:Plack::Request, CPAN:Plack::Builder and such) - because using them:
   • could again help shorten our codebase (please, re-read the above about the "mounting" and about the middlewares)
   • because they're will be already installed (or bundled)
   • because they're included (contained in) in the CPAN:Plack distribution

So, please finally decide : want PSGI or do not want ? because

• if don't want - OK - i could save to myself many hours - not need to write such topics and comments
• but if want the PSGI, we need
  • learn the PSGI !!! (and in deeper details as outlined above)
  • and when got familiar with it, you will find also that we need:
  • start to use the CPAN:Plack package
  • and stop re-inventing inhouse things like the PATH_INFO parsing and such, because the CPAN:Plack contains them already (and their version for example correctly deals with the MOUNTING )
  • and the PSGI way of life really DEEPLY affects the Foswiki web-specific (!!!) internals (aka: the application layering using the PSGI API)

Or do you want re-develop inhouse the whole CPAN:Plack package?!?

Vadim,

ad: the point is that I cannot get rid of the engines until Plack::Handlers are confirmed by the community

Right. But the question isn't about until Plack::Handlers are confirmed by the community, but about will the foswiki going to the PSGI path, or not ?

I simply don't know how to write things to be understand-able. So, trying again using other words.

IF (again IF) the Foswiki want the PSGI, THEN

• you COULD continue to use the current Engines
• just move them OUTSIDE from the main Foswiki web-app
• to the place, where they belong, because
• if you will use the PSGI API
• they belong BEFORE the application (and you havent other choise, because that way is the PSGI defined).
• (and later you will find, than using our own "Engines" is pointless, because the Plack::Handler things does the same job - (and better))

If Foswiki will not follow the PSGI API, then the whole discussion is pointless - and we could stop wasting time with the PSGI tutorials..

-- JozefMojzis - 10 May 2016

I think, everything is said more times in two topics.

Now another view - trying to approach the whole problem from the other side, without arguing how to redesign the internals.

IF we will have ( any sort ) of PSGI, then i will be perlfectrly happy.

Regardles how the Foswiki internals will be done. Because, for me is enough to know the basic 3 things:

1. what routine returns to me the needed code-reference.
2. so i could call it from my app.psgi
3. and it returns for me the needed arrayef.

So, the basic is - when some app.psgi will contain, let say the following few lines ...

use Plack::Builder;

use Foswiki::SomePackage;
my $foswiki = Foswiki::SomePackage->method_name_which_gives_me_the_foswiki_code_reference();

builder {
    mount '/wiki' => builder {
         $foswiki; 
    };
    mount '/' => sub {"Hello wiki"};
}

... and the Foswiki will work ok - then, the foswiki will be great.

If the new Foswiki will get this (together with the Moo + class-hierarchy redesign) - I probably could every my needs hack myself - using overrides, roles and middlewares - so i could modify all the headers, or the body myself, and probably I could use it as a resource for the CPAN:Web::Machine too.

So, Vadim, i quickly checked your latest checkins - looks very promising - thank you.

-- JozefMojzis - 10 May 2016

Commenting is disabled while not logged in