How to use SolrSearch with multiple Foswikis on the same server.

  • Tip Category - Using Extensions
  • Tip Added By - ModAcOst - 21 Feb 2011 - 10:08
  • Extensions Used - SolrPlugin
  • Useful To - Experts
  • Tip Status - Outdated
  • Related Topics -
ALERT! WARNING: This best practice is outdated. Please rework.

Problem

SolrSearch is a powerful and feature-rich application. However in it's current state it is only meant to run with a single Foswiki-installation on the server. This article will describe how to set up Solr's multicore, ie. how to run SolrSearch with multiple wikis on the same machine.
Be warned, this text was not written by the author of SolrPlugin and might be wrong in some respects.
More information can be found here: http://wiki.apache.org/solr/CoreAdmin

This guide is based on SolrPlugin 1.10 (16. Dec. 2010)

Before you begin

You should be experienced on how to install SolrSearch "the usual way". Some files will be needed so it is suggested that you have a working install at hand.

Server

A number of changes will be applied server-wide. Those need to be done only once and apply to all wikis.

Solr-Home-Dir

Solr needs a home. All cores will be housed in that directory and need write permissions to it. For the rest of this document this directory is assumed to be
/solr
Into there copy the contents of a "normal" solr-home to be found in <foswiki>/solr and create a subdirectory called "multicore". Your result should look like this:

Contents of /solr:

60solr.policy
etc
.htaccess
lib
logs
multicore
solr
solr-context.xml
start.jar
webapps 

In your new multicore-directory create a file called "solr.xml" and copy the following contents into it:
<?xml version="1.0" encoding="UTF-8" ?>
<solr persistent="false">
        <cores adminPath="/admin/cores">
                <!-- ...Cores will be placed here... -->
        </cores>
</solr>
Each core will be added to this file later on.

Subdomains

After applying server-wide settings each wiki must be added and (unfortunately) be modified to a certain extend.

You need to install SolrPlugin in each wiki with configure or manually.
The following will assume your Solr-home is in /solr and your wiki will be called "example". Every following step must be applied to each Foswiki individually.

multicore

At first your "example"-wiki must be added to the configuration file "/solr/multicore/solr.xml". It should contain a reference to each active core:
Contents of /solr/multicore/solr.xml:

<?xml version="1.0" encoding="UTF-8" ?>
<solr persistent="false">
        <cores adminPath="/admin/cores">
                <!-- already existing other cores... -->
                <core name="aCore" instanceDir="aCore" />
                <core name="anotherCore" instanceDir="anotherCore" />
                <!-- our newly added core: --->
                <core name="example" instanceDir="example" />
        </cores>
</solr>

Each core has its own directory called instanceDir associated with it. You need to create this directory inside your multicore path:
# create new instance directory:
mkdir /solr/multicore/example
In order to keep thing easy you should name this directory after your wiki.
Inside the solr-home of a "normal" installation is a directory called "solr". Copy the contents of this directory into the instance-Dir:
# copy <foswiki>/solr/solr/* to /solr/multicore/example
cp -R /srv/www/htdocs/foswiki/solr/solr/* /solr/multicore/example

Contents of /solr/multicor/example:
conf
data
lib

Now one of the just copied files needs to be adapted to it's new environment. Edit the file "solrconfig.xml" inside "conf" and correct the "dataDir" to reflect the new location:
In /solr/multicore/beispiel/conf/solrconfig.xml change line 71:
  <dataDir>${solr.data.dir:./solr/data}</dataDir>
to
  <dataDir>${solr.data.dir:./multicore/example/data}</dataDir>

Plugin files

Currently some files of the SolrPlugin must be edited as well.

Webservices

After having installed SolrPlugin there will be a file "<foswiki>/lib/Webservices/Solr.pm". Change the given URL to the correct core:
Line 16 of <foswiki>/lib/Webservices/Solr.pm:
    default => sub { URI->new( 'http://localhost:8983/solr' ) }
becomes:
    default => sub { URI->new( 'http://localhost:8983/solr/example' ) }

solrstart

In Foswiki's tools directory is a file called solrstart. Adapt it's solr-home and also tell Solr to use multicore now:
Change line 10 of <foswiki>/tools/solrstart
test -z $SOLRHOME && SOLRHOME=$FOSWIKI_ROOT/solr
to
test -z $SOLRHOME && SOLRHOME=/solr

and Line 22
nohup java -jar start.jar >/dev/null &
to
nohup java -Dsolr.solr.home=multicore -jar start.jar >/dev/null &
(Don't forget to change the !# line and FOSWIKI_ROOT as you would in a normal installation.)

Configure

Now use configure to make some final adjustments. In the SolrPlugin settings change the SolrHome property to your actual solr home:
{SolrPlugin}{SolrHome} = /solr
(Note: not /solr/multicore or /solr/multicore/example I might be wrong here tough)
Also you must tell the plugin the URL where to find the core:
{SolrPlugin}{Url} = http://localhost:8983/solr
must be changed to
{SolrPlugin}{Url} = http://localhost:8983/solr/example
Also change your SolrStartCmd to use multicore as home-directory for solr (see solrstart).

Indexing

At last you must take care of your indexing-method of choice. Each core must be told to index but you can use solrjob as usual. However you should take care that only one indexing-operation runs at a time for better io-performance.---++ Problem

BestPracticeTipsForm edit

Category Using Extensions
Related Topics
Topic revision: r2 - 20 Apr 2015, MichaelDaum
The copyright of the content on this website is held by the contributing authors, except where stated elsewhere. See Copyright Statement. Creative Commons License    Legal Imprint    Privacy Policy