git

See ProjectOverview for an overview of the development process, and how we do revision control. This document is all about the nitty-gritty details of how we use git to implement that process.

We maintain our code using git. Be aware that this is not a stable release version! You are recommended to use the official releases (see Downloads for the latest release).

This page is primarily meant for developers that want to track latest changes and contribute code back to our git repository.

Quick overview:

• Project address: http://github.com/foswiki
• Commit messages are sent to the DeveloperEventsMailingList.
• Want to contribute? Fork the repository, make your changes, and submit a pull request.
• Need help? Get live support our IRC support channel.

git URLs

Use	URL
primary github account	https://github.com/foswiki/
the official Foswiki distribution	https://github.com/foswiki/distro

What is git?

git is a free distributed revision control system. It records all the source code, and tracks changes to that source code. Our old Subversion (svn) repository was a centralized server.

It can be downloaded from http://git-scm.com. Various GUI clients for Windows and other platforms are available from http://git-scm.com/download/ There is also a book about git usage online at http://git-scm.com/book/

How does Foswiki use git?

The Foswiki project uses http://github.com/foswiki/ to store and track changes to the Foswiki source code, and the source code of many extensions. You don't have to use git to contribute to Foswiki, but you'd have to have a pretty strong reason not to. With the decentralized GIT model, it's also possible to use other git repositories for non-default extensions, however the Foswiki project will only track code released using the http://github.com/foswiki/ account.

Repositories

The GitHub foswiki project is organized into a number of repositories within the foswiki project account. The "distro" repository is the core foswiki product. We build and release from foswiki/distro

• There are two "classes" of repository within the project:

  "distro" Respository

  subjected to Release Management: This is the primary foswiki product. It Follow the release process: Changes to external behaviour, APIs, etc are subject to the FeatureProposal review process. Changes submitted by non-core developers should be done by forking the repo and submitting a "Pull Request".

  Repositories for contributed extensions

  This is "everything else". These repositories are subject to the guidelines of ModificationPolicy

Based upon this, a small subset of repositories is:

• Account: foswiki
  • Repository: distro: The core distribution complete with all default extensions and developer tools tied to a release.
  • Repository: AccessStatsPlugin: a non-default extension
  • Repository: ActionTrackerPlugin: another non-default extension
  • ...
  • ... all other extensions tracked by the foswiki project

Branching and Tagging

See GitBranchingAndTagging for more details.

• The master branch (formerly known as trunk) is used to check in all bug fixes and new code targeted at the next major release. Code on the master should always works, and most developers rely on it always working for their development environments. So development is preferably done in a separate branch and merged once reliable.

The code checked into the master branch runs as an active server at http://trunk.foswiki.org/. This checkout area is kept up to date by cron job that runs every 15 minutes. In the best case, the person doing the checkin will be able to almost immediately check the correct functioning of their checkin.

• The Release* branches are where we release Foswiki from. These must be kept in a "ready to release" state. Changes should always be staged first to a testing branch before merging into a Release.

• Any ItemNNNN* branch. These branches are created on demand to allow merging of unstable work related to a task. This is strongly recommended when commits are expected to be unstable, or need review before merge into a branch (master or Release*)

Branches & tags for "distro":

The "distro" repository, has all of the official branches managed by the project:

• master bleeding edge work used for changes that are ready for testing but have not quite "release quality"
• Release01x00: Original foswiki release, no longer maintained
• Release01x01:Current Foswiki release 1.1.x
  • Tag: FoswikiRelease01x01x08: 1.1.8 Patch release
  • Tag: FoswikiRelease01x01x09: 1.1.9 patch release
• Release01x02: (not yet branched. Next minor foswiki release)
• Item12477: Branch being used for development of a fix to Item12447.
• ... Other branches for features not yet ready for daylight

All other extension repositories will release from the master branch consistent with how extensions were built from trunk in subversion. Other branches can be created for new features not yet ready for release.

githooks

git hooks are used to enforce project standards for code contributed to the project. With subversion, we had these hooks on the server and check-ins were uniformly screened. With git / github, we are unable to install server side hooks, so all enforcement has to be done client side. We strongly recommend that the following two hooks are installed before commiting any changes.

pre-commit

This hook enforces coding standards. Code must be tidy, per the TIDY file setting, and dates, authors, versions set in topic META must be current and correct.

commit-msg

This hook enforces the Task standards. At least one Item number must be identified in the commit message. And any tasks identified must be in a state to accept check-ins.

The hook code is in core/tools/develop/githooks/ It must be copied to:

Each repository

Copy both hooks into the .git/hooks directory

For superprojects

Copy both hooks into .git/modules/<Extension>/hooks/ directory

What has changed with the conversion to git?

	subversion	git
Development model	One single centralized repository	Distributed development; Each developer has their own local repository for some or all of foswiki
Repository location	Central repository hosted on foswiki.org	Hosted by http://github.org/ (mirror kept on foswiki.org server)
Authority	Must request commit authority to contribute code	Optionally request commit authority. Anyone can fork the code and submit a "pull request" to contribute to the project
Repository structure	One large repository for the whole project. Very large download for extension developers	Very small repository per extension, plus the "distro" repository. Small, fast download
Bug tracking	Use http://foswiki.org/Tasks	(no change - continue with Tasks web)

Running from a Git repository

See GitBasedInstall and the GitCookbook for practical advice on using a git checkout as a live install.

Helpful hints using git

Adding the branch to your command prompt.

The following addition to your ~/.bashrc will add the current repo information to your command prompt

# Put Git branch name it shell prompt 
parse_git_branch() {
  git branch 2> /dev/null | sed -e '/^[^*]/d' -e 's/* \(.*\)/ (\1)/'
}
PS1='\u@\h:\w$(parse_git_branch)\$ '

Here is a normal "git status" from the

$: /home/GIT/distro (master)$ git status

On branch master
Your branch is up-to-date with 'origin/master'.

nothing to commit, working directory clean

Cleanup

Once done with testing, to discard any changes you don't intend to commit, and clean the checkout

• git reset HEAD --hard
• git clean -fd

Use the -fdx option to clean to remove files in the .gitignore settings.

How to select a branch for testing and development.

There is a branch for each foswiki release. There may be other developer branches as well, depending on what is going on.

Once you've cloned a repository, you can list the branches within that repository:

 $: /home/GIT/distro (master)$ git branch -va
  Release01x00                79c5cc9 Item10193: update to 1.0.10
  Release01x01                997bb5c Item11267: Remove $Rev keywords
* master                      5e490ad Item11267: Add FoswikirefsPlugin to trunk
  remotes/origin/HEAD         -> origin/master
  remotes/origin/Release01x00 79c5cc9 Item10193: update to 1.0.10
  remotes/origin/Release01x01 997bb5c Item11267: Remove $Rev keywords
  remotes/origin/master       5e490ad Item11267: Add FoswikirefsPlugin to trunk

To switch to a different branch, checkout that branch from your local repository.

$: /home/GIT/distro (master)$  git checkout -b test11 remotes/origin/Release01x01
Branch test11 set up to track remote branch Release01x01 from origin.
Switched to a new branch 'test11'
 

There is also a handy script in the core/tools directory that will list all of your local branches, and any remotes that you have not yet checked out.

myhost: /var/www/foswiki/distro (Item14530) $ core/tools/branches 
(Your local branches are listed first, along with the description from any task in the branch name)
    Item13594 (merged)
      Implement Feature AddConcatOptionToAttrs
       Implement Feature AddConcatOptionToAttrs - Waiting for Release 
  * Item14530
       Add new config "RootDir" to unify configuartions of file system paths - New 
    Release01x00
    Release01x01
    Release02x00
    Release02x01
    master

(And then all remote branches not covered by a local branch are also listed)
    remotes/origin/Item14448
       plugin list empty on page reload - Waiting for Release 
    remotes/origin/Item14454
       Bundle JsViews as an option with JsRender - Confirmed 

Before you do any development!

Ensure that the githooks were installed. again pseudo-install.pl developer takes care of this.

How to check in

Remember to run

git status
git diff

before committing!

Please run PerlTidy on your code before checking in, eg:

export PERL5LIB='path/to/BuildContrib/lib'
find . -name '*.pm' -exec perltidy -b '{}' \; 

We now enforce that .pm and .pl files are run through perltidy for some extensions, such as Foswiki's core/default extensions. See TIDY

For documentation topics, remove any stray Ctrl-M's, and ensure that the TOPICINFO metadata is set to author="ProjectContributor" and version="1" before checking in the file. Also the topic timestamp should be set to the current unix timestamp (Seconds since 1970).

date +%s
1395457984

Rules for checkins

Please keep these rules in sync with the error message coded into core/tools/develop/pre-commit.pl b/tools/develop/pre-commit.pl

The local git hooks should reject any commit that does not:

1. Have a comment...
2. ...with relevant ItemNNN topics in the first line, example:

   Item12345: Item12346: fixed foo, updated release notes

3. Use ItemNNN topics which are open at the time of checkin,
   1. E. not one of: Closed, Waiting For Release, No Action
4. Have "tidied" source code if the TIDY control file in the root of the extension calls for it, see: http://foswiki.org/Development/TIDY
5. Have a valid TOPICINFO for any topic.txt files: .txt files in web directories must have META:TOPICINFO with the author "ProjectContributor", a version of 1 and a date within 3 days of the checkin. Any FILEATTACHMENTs must have the "ProjectContributor" author, a version of 1 and a date with 3 days of the checkin.

This is to make sure that we have traceability of all changes. These comments are used to update the linkage between items in the Tasks web and checkins in git, so you should try hard to make sure they are accurate and complete.

We can't "enforce" these on github, so we depend upon "good developer behaviour"; installing the local hooks and not overriding them. Commits that don't follow the rules are subject to being reverted.

Generating Patches

Even if you can't check in, you can still contribute. The easiest way is to "fork" the repository on github, make your changes, commit them to your own github account / repository, and submit a "pull request" to the foswiki project. Other options:

• Submit a patch using email: For example, generating patches for your last two local commits:

$ git format-patch HEAD^^
0001-Item11267-Skip-updating-trunk-if-nothing-has-changed.patch
0002-Item10824-Implement-HEAD-support.patch

• Generate a diff of "uncommitted changes" and attach it to a task (This won't pick up new files)

git diff > mypatch.diff

Attach the file mypatch.diff to the relevant topic in Tasks web (there should be one topic for each change) and ask someone with checkin rights to merge it.

Please ensure the UnitTests are run if possible, and always run through the tests in the TestCases web in your checkout area.

Watching

Check-ins are automatically mailed to the foswiki-commits mailing list. See https://lists.foswiki.org/wws/info/foswiki-commits.

Scratch area in SVN

With git, it's no longer necessary to have a separate scratch branch. Just fork the code you want to experiment with into your own repository on github. The original subversion scratch branch still exists: https://github.com/foswiki/foswiki/tree/scratch but is no longer necessary.

FAQ

No questions yet.

Notes

Proxy Server

Proxy server settings are set by using the git config command:

git config --global http.proxy http://proxyuser:proxypwd@proxy.server.com:8080
git config --global https.proxy http://proxyuser:proxypass@proxy.server.com:8080

• proxyuser: Userid required to access the proxy server
• proxypass: Password for proxy server access
• proxy.server.com: The server
• :8080: Optional port of proxy server

Git tags

Tags are a lightweight pointer to the HEAD commit that was used to generate a release

After creating the tags, they can be pushed to the repository with "git push origin --tags" Other handy operations

• Checkout a tag: git checkout v1.1.6
• Compare two releases: git diff 2011-04-10a v1.1.7
• Show commits in release: git log FoswikiRelease01x01x08..FoswikiRelease01x01x09 --oneline
• Show tags with annotations: git tag -l -n1
• Delete a tag from the repository. git push --delete origin v1.1.6

Merge an extension into the distro repository

If an extension is added as a "Default" extension, it's possible to merge the repositories so that history remains intact. This process will rewrite the commit id's and invalidate pointers in the Tasks web.

• Clone the extension repo (if not already done), Either with git clone, or use pseudo-install.
• Move the cloned repo so that it won't collide with the existing directory when the extensions is merged.
• Rewrite the index, moving the root level directories into a subdirectory of the same name as the extension. Suggestion found in http://stackoverflow.com/questions/3142419/how-can-i-move-a-directory-in-a-git-repo-for-all-commits

git clone git@github.com:foswiki/FastCGIEngineContrib.git
mv FastCGIEngineContrib oldrepo
cd oldrepo
git filter-branch --index-filter \
  'git ls-files -s | sed "s-\t\"*-&FastCGIEngineContrib/-" |
   GIT_INDEX_FILE=$GIT_INDEX_FILE.new \
   git update-index --index-info &&
   mv "$GIT_INDEX_FILE.new" "$GIT_INDEX_FILE"' HEAD

• At this point, you should now have oldrepo/FastCGIEngineContrib/
• Check using git log, git status, etc. that everything is as expected.

• Incorporate the extension into distro.

cd ..   (into the root of the distro repository)
git remote add exten oldrepo
git fetch exten
git merge exten/master

• You should now have a FastCGIEngineContrib directory below distro.