Upgrade from Solr3.4/jetty8 to Solr7.7.2

to start installing the new version of solr, it is required to Make sure java-8-openjdk is in use.

→ Errors are given saying 11 is too old which is not accurate. 

Run this line to check which version of java is running:

user@virtual-machine:/home/user# java -version
openjdk version "1.8.0_222"
OpenJDK Runtime Environment (build 1.8.0_222-8u222-b10-1ubuntu1~16.04.1-b10)
OpenJDK 64-Bit Server VM (build 25.222-b10, mixed mode)

Note: to upgrade the current version of java it is important to know in which version of ubuntu, the CKAN is running.

In this phase, we should first clean out all solr related "stuff" before installing a newer version and then installed from scratch. Careful with these commands

• stop jetty if it's running, remove related solr and jetty directories^[2]:

sudo service jetty9 stop
sudo apt-get purge --auto-remove solr-jetty
sudo rm -rf /etc/default/jetty8
sudo rm -rf /etc/solr/
sudo rm -rf /etc/jetty8/
sudo rm /etc/init.d/jetty8
sudo rm -rf /var/solr
sudo rm -rf /etc/default/solr.in.sh

Go to the Apache Solr website and download Solr, in this tutorial we are going to use the .tgz format, but it works with .zip as well.

Download the package from the web and put it somewhere in your file-system (e.g. /opt/)^[1]

cd /opt
wget https://www.apache.org/dyn/closer.lua/lucene/solr/7.7.2/solr-7.7.2.tgz

Now extract Apache Solr service installer shell script from the downloaded Solr archive file and run the installer using following commands.

tar xzf solr-7.7.2.tgz solr-7.7.2/bin/install_solr_service.sh --strip-components=2
sudo bash ./install_solr_service.sh solr-7.7.2.tgz

Solr is configured as a service the system. Using the following command to Start, Stop and check the status of Solr service:

sudo service solr stop
sudo service solr start
sudo service solr status

Switch to the solr user and go to the bin directory ^[3]

sudo su solr
cd /opt/solr/bin

now create the ckan core

./solr create -c ckan

solr creates all the configuration files and directories. At this point, we can see the core listed in our solr admin http://10.162.246.199:8983/solr/#/~cores/ckan and we can proceed to edit the configuration files.

Open /var/solr/data/ckan/conf/solrconfig.xml and update next sections^[4]:

# //requestHandler[@name="/select"]/lst[@name="defaults"]
# add two lines after `<int name"rows">10</int>`

<str name="df">text</str>
<str name="q.op">AND</str>

# add next line before `<updateProcessor class="solr.UUIDUpdateProcessorFactory" name="uuid"/>`

<schemaFactory class="ClassicIndexSchemaFactory" />

# change `default` attribute of `<updateRequestProcessorChain>` to `false`:

<updateRequestProcessorChain name="add-unknown-fields-to-the-schema" default="${update.autoCreateFields:false}"
processor="uuid,remove-blank,field-name-mutating,parse-boolean,parse-long,parse-double,parse-date,add-schema-fields">

From SOLR >= 7.0.0, schema.xml needs some changes as some of the parameters are no longer supported.

schema.xml.7.7.2

Replace the default schema.xml file with a symlink to the CKAN schema file included in the sources:

sudo ln -s usr/lib/ckan/default/src/ckan/ckan/config/solr/schema.xml /var/solr/data/ckan/conf/schema.xml

Restart solr:

sudo service solr restart

Change the "solr_url" setting in the ckan configuration file (/etc/ckan/default/production.ini) to point to your Solr server:

solr_url=http://127.0.0.1:8983/solr/ckan

Rebuild the search index:

paster --plugin=ckan search-index rebuild --config=/etc/ckan/default/production.ini

Finally restart/reload the apache service:

sudo service apache2 reload

There are different backends supported for the spatial search. In case that "ckanext.spatial.search_backend = solr-spatial-field", following steps need to be taken^[6] :

1. This option uses the spatial field introduced in Solr >=4.x, which allows to index points, rectangles and more complex geometries (complex geometries will require JTS)
2. After upgrading or install a proper version of Solr, one may need to include JTS. In order to activate "JTS Topoligy Suite" :
   1. Download the JAR file: http://central.maven.org/maven2/org/locationtech/jts/jts-core/1.15.0/
   2. put that in a special location internal to Solr:SOLR_INSTALL/server/solr-webapp/webapp/WEB-INF/lib/^[7]

   3. Set the spatialContextFactory attribute on the field type to JTS:

      <fieldType name="location_rpt"   class="solr.SpatialRecursivePrefixTreeFieldType"
                     spatialContextFactory="JTS"
                     autoIndex="true"
                     validationRule="repairBuffer0"
                     distErrPct="0.025"
                     maxDistErr="0.001"
                     distanceUnits="kilometers" />
      <fields>
          <!-- ... -->
          <field name="spatial_geom"  type="location_rpt" indexed="true" stored="true" multiValued="true" />
      </fields>

   4. Restart Solr.