Item8272: Generate compressed JS

For efficiency in SkillsPlugin.

BuildContrib could be smarter in the way it handles different extensions for compression.

-- CrawfordCurrie - 12 Sep 2009

This change makes any extension which uses it uninstallable through pseudo-install. Therefore, pseudo-install needs to be fixed. I guess simply linking / copying the uncompressed version as the compressed one is enough.

GeorgeClark found the incorrest behavior for the latest TwistyPlugin.

-- OlivierRaginel - 14 Feb 2010

Ok, coded a fix. There is still a typo to fix, maybe some docu to redo, and most important...

CDot, do you think it's interesting to make BuildContrib ship gzip'ed versions of CSS, JS, more?

Looking at Michael's JQueryPlugin, he has all 3 versions (uncompressed.js, .js, .js.gz), and I thought having only the uncompressed.js, but putting both .js and .js.gz in the MANIFEST could be enough, and ease the maintenance (especially as this plugin is now core).

Waiting for feedback before closing.

-- OlivierRaginel - 14 Feb 2010

The reason for having all three in the package is:

• .js and .css is used on production sites
• .js.gz .css.gz are served only if you optimized apache to do so but not for IE6
• .uncompressed.js abd .uncompressed.css are used instead of their minified/compressed counterparts if the plugin is in debug mode

-- MichaelDaum - 15 Feb 2010

Sorry Michael, you didn't get my point probably because i wasn't clear enough.

Having all 3 versions in the package is a real advantage, I agree with you. My point was more: do you really need them in Subversion?

Here I was suggesting to only commit the .uncompressed.js, but to put all 3 in the MANIFEST, and BuildContrib would do its magic (it can already do .uncompressed.js => .js).

-- OlivierRaginel - 15 Feb 2010

Babar, true ... will remove the derived files as soon as pseudo-install is able to do the rest of the trickery, as George already pointed out.

-- MichaelDaum - 15 Feb 2010

Ok, so I'll add the .gz trickery to BuildContrib.

• The .js trickery was already there. I've tried removing all the un-necessary stuff from JQueryPlugin, and after building a release, I have the very same filelist. So I'll close this bug and will open a new one to cleanup plugins.

Oups, forgot I had a typo and a small fix in case somebody does not have the minifiers but still wants to do a release (that's bad, but hey...)

-- OlivierRaginel - 15 Feb 2010

We need this magic to work also for js+css files named _src.js as in TinyMCE. In fact despite its name TinyMCE is probably the plugin that most urgently needs .gz versions and yet it doesn't (my oversight).

TinyMCE ships with a great many files, I should really remove the files from /classes out of MANIFEST.

-- PaulHarvey - 16 Feb 2010

This magic ought to work with _src.js. You probably just need to put the .js version and the .js.gz version IN THIS ORDER in the MANIFEST, and BuildContrib will do the rest for you.

-- OlivierRaginel - 16 Feb 2010

This is not closed. I am being flooded by

WARNING: Cannot find source file /var/www/foswiki/core/../TinyMCEPlugin/#/pub/System/TinyMCEPlugin/foswiki.js.gz
WARNING: Cannot find source file /var/www/foswiki/core/../TinyMCEPlugin/#/pub/System/TinyMCEPlugin/foswiki_tiny.js.gz
WARNING: Cannot find source file /var/www/foswiki/core/../TinyMCEPlugin/#/pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/plugins/advhr/editor_plugin.js.gz
WARNING: Cannot find source file /var/www/foswiki/core/../TinyMCEPlugin/#/pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/plugins/advimage/editor_plugin.js.gz
WARNING: Cannot find source file /var/www/foswiki/core/../TinyMCEPlugin/#/pub/System/TinyMCEPlugin/tinymce/jscripts/tiny_mce/plugins/advlink/editor_plugin.js.gz
...

when I do a normal pseudo-install like I have done for years.

A pseudo-install that removes all symlinks first and then a clean pseudo-install. That does not work anymore.

-- KennethLavrsen - 21 Feb 2010

Please first run the compress target for the package. See the docu of BuildContrib on how to do that. This is a procedure you have to keep in mind for all packages that ship css and js ... as they all should ship minified and gzip'ed files.

Yes, pseudo-install should that 4u automatically.

-- MichaelDaum - 21 Feb 2010

It does now (assuming I got it right)

-- CrawfordCurrie - 25 Feb 2010

I have a heavily cooked dev environment I'm thinking of ripping out and starting again, so maybe it's just me, but is anybody else getting their _src* files overwritten with compressed versions? This is happening for me when trying to build trunk/TinyMCEPlugin

Edit: Right, I figured it out. Seems we symlink the missing uncompressed/minified files. Then afterwards when you try to build, buildcontrib overwrites the target of the symlinks, which is the uncompressed version.

Is this what we're going to do? BTW: My apache config (which rewrites .js.gz as .js) serves up the content of the link, rather than the target of the link. So tiny_mce.js is served up as /./@LongLink�������... yuck.

I think this approach is broken.

-- PaulHarvey - 26 Feb 2010

I saw the same kind of thing as Paul did - but it is a tarball, not a link. I checked this by zcat'ing the .gz to another file and running file on it: file reports column_left.css: POSIX tar archive (GNU)

I changed pseudoinstall to use Compress::Zlib (like BuildContrib) or gzip

-- MichaelTempest - 14 Mar 2010

I have no idea why this is waiting for me. Crawford made it use tar, because it was in perl default I guess. But Compress::Zlib is also since 5.9.3, so soonish it will be OK, and for most developers it should be, as they should most likely be running 5.10 now.

-- OlivierRaginel - 24 Mar 2010

Made the compressor use the YUI compressor if it's available and only fall back to the CPAN minifier modules if it has to (e.g. there is no java on the platform).

Also made it check the file dates on derived objects to accelerate the build.

Also made it handle the case where you minify (or zip) when the target is a symbolic link.

Issues with pseudo-install should not be dealt with in this report!

-- CrawfordCurrie - 25 Mar 2010

Re-opening as Crawford only did that for the -link option, but not for the -copy option, and now we're having a lot of trouble on windows with pseudo-install. I've fixed the code, just need to commit.

-- OlivierRaginel - 15 Jun 2010