To do the upgrade it is required to change some ckan and solr config files in addition to the installation of a new version of solr. It is due to the new changes (e.g. support of old and new parameters) in newer versions of solr esp. after version 6.0.
The steps are explained as follows:
to start installing the new version of solr, it is required to Make sure java-8-openjdk is in use.
→ Errors are given saying 11 is too old which is not accurate.
Run this line to check which version of java is running:
user@virtual-machine:/home/user# java -version openjdk version "1.8.0_222" OpenJDK Runtime Environment (build 1.8.0_222-8u222-b10-1ubuntu1~16.04.1-b10) OpenJDK 64-Bit Server VM (build 25.222-b10, mixed mode)
Note: to upgrade the current version of java it is important to know in which version of ubuntu, the CKAN is running.
please make yourself familiar with the solr installation requirements considering the OS and solr version: e.g.: https://lucene.apache.org/solr/guide/7_7/solr-system-requirements.html
There are 2 options to get Solr to work if you want to run Ubuntu 14.x and 16.x (for higher versions it should be double checked):
- modify some
solr-jetty
package files - use Solr directly.
In our case, we have chosen using SOLR directly.
In this phase, we should first clean out all solr related "stuff" before installing a newer version and then installed from scratch. Careful with these commands
- stop jetty if it's running, remove related solr and jetty directories[2]:
sudo service jetty9 stop sudo apt-get purge --auto-remove solr-jetty sudo rm -rf /etc/default/jetty8 sudo rm -rf /etc/solr/ sudo rm -rf /etc/jetty8/ sudo rm /etc/init.d/jetty8 sudo rm -rf /var/solr sudo rm -rf /etc/default/solr.in.sh
Go to the Apache Solr website and download Solr, in this tutorial we are going to use the .tgz
format, but it works with .zip
as well.
Download the package from the web and put it somewhere in your file-system (e.g. /opt/)[1]
cd /opt wget https://www.apache.org/dyn/closer.lua/lucene/solr/7.7.2/solr-7.7.2.tgz
Now extract Apache Solr service installer shell script from the downloaded Solr archive file and run the installer using following commands.
tar xzf solr-7.7.2.tgz solr-7.7.2/bin/install_solr_service.sh --strip-components=2 sudo bash ./install_solr_service.sh solr-7.7.2.tgz
Solr is configured as a service the system. Using the following command to Start, Stop and check the status of Solr service:
sudo service solr stop sudo service solr start sudo service solr status
Switch to the solr
user and go to the bin
directory [3]
sudo su solr cd /opt/solr/bin
now create the ckan
core
./solr create -c ckan
solr creates all the configuration files and directories. At this point, we can see the core listed in our solr admin http://10.162.246.199:8983/solr/#/~cores/ckan and we can proceed to edit the configuration files.
Open /var/solr/data/ckan/conf/solrconfig.xml and update next sections[4]:
# //requestHandler[@name="/select"]/lst[@name="defaults"] # add two lines after `<int name"rows">10</int>` <str name="df">text</str> <str name="q.op">AND</str> # add next line before `<updateProcessor class="solr.UUIDUpdateProcessorFactory" name="uuid"/>` <schemaFactory class="ClassicIndexSchemaFactory" /> # change `default` attribute of `<updateRequestProcessorChain>` to `false`: <updateRequestProcessorChain name="add-unknown-fields-to-the-schema" default="${update.autoCreateFields:false}" processor="uuid,remove-blank,field-name-mutating,parse-boolean,parse-long,parse-double,parse-date,add-schema-fields">
From SOLR >= 7.0.0, schema.xml needs some changes as some of the parameters are no longer supported.
Replace the default schema.xml
file with a symlink to the CKAN schema file included in the sources:
sudo ln -s usr/lib/ckan/default/src/ckan/ckan/config/solr/schema.xml /var/solr/data/ckan/conf/schema.xml
Restart solr:
sudo service solr restart
Change the "solr_url" setting in the ckan configuration file (/etc/ckan/default/production.ini) to point to your Solr server:
solr_url=http://127.0.0.1:8983/solr/ckan
Rebuild the search index:
paster --plugin=ckan search-index rebuild --config=/etc/ckan/default/production.ini
Finally restart/reload the apache service:
sudo service apache2 reload
Extra test
To check whether everything works as expected you can run this:
http://10.162.246.199:8983/solr/ckan/select?indent=on&q=*:*&wt=json
There are different backends supported for the spatial search. In case that "ckanext.spatial.search_backend = solr-spatial-field", following steps need to be taken[6] :
- This option uses the spatial field introduced in Solr >=4.x, which allows to index points, rectangles and more complex geometries (complex geometries will require JTS)
- After upgrading or install a proper version of Solr, one may need to include JTS. In order to activate "JTS Topoligy Suite" :
- Download the JAR file: http://central.maven.org/maven2/org/locationtech/jts/jts-core/1.15.0/
- put that in a special location internal to Solr:
SOLR_INSTALL/server/solr-webapp/webapp/WEB-INF/lib/
[7] Set the
spatialContextFactory
attribute on the field type toJTS:
<fieldType name="location_rpt" class="solr.SpatialRecursivePrefixTreeFieldType" spatialContextFactory="JTS" autoIndex="true" validationRule="repairBuffer0" distErrPct="0.025" maxDistErr="0.001" distanceUnits="kilometers" /> <fields> <!-- ... --> <field name="spatial_geom" type="location_rpt" indexed="true" stored="true" multiValued="true" /> </fields>
- Restart Solr.