Cacher is the code snippet organizer for pro developers

--- HOW TO CHANGE SOLR CONFIGURATION TO CREATE A SORT OF MASTER/SLAVE CONFIGURATION FOR GENERIC INDEXES

Based on the VM replication for APP31 a few simple steps I followed to add an extra core to Solr 4.4:


1) mkdir /opt/solr4.4/solr_example/solr_example
mv /opt/solr4.4/solr_example/conf /opt/solr4.4/solr_example/solr_example

2) cd /opt/solr4.4/solr_example/
cp -R solr_example/ solr_example_write

3) changing in "solr_example_write/solrconfig.xml":

from

<dataDir>/opt/indexes4.4/example</dataDir>

to

<dataDir>/opt/indexes4.4/example_write</dataDir>

4) changing the file /opt/solr4.4/solr_example/solr.xml to:

<?xml version="1.0" encoding="UTF-8" ?>
<solr persistent="true">
  <cores defaultCoreName="example" adminPath="/admin/cores" zkClientTimeout="20000" hostPort="8080">
    <core instanceDir="./solr_example" name="example"/>
    <core instanceDir="./solr_example_write" name="example_write"/>
  </cores>
</solr>

please note I removed <cores... hostContext="solr_example">

5) restart solr/tomcat
/etc/init.d/solr stop|start|restart (or whatever you use)

--- HOW I TESTED THE SWAP OF THE CORES

1) Created the main index:

curl http://localhost:8080/solr_example/update -H 'Content-type:application/json' -d '
[
{
  "id" : 1,
  "hash_id" : "foohash1",
  "source_type" : "FOO_EX_RES",
  "paf_udprn" : "foo udprn 2",
  "paf_postcode" : "foo postcode 2"
},
{
  "id" : 2,
  "hash_id" : "foohash2",
  "source_type" : "FOO_EX_RES",
  "paf_udprn" : "foo udprn 2",
  "paf_postcode" : "foo postcode 2"
}
]'

2) Created the "write" index:

curl http://localhost:8080/solr_example/example_write/update -H 'Content-type:application/json' -d '
[
{
  "id" : 1,
  "hash_id" : "barhash1",
  "source_type" : "BAR_EX_RES",
  "paf_udprn" : "bar udprn 2",
  "paf_postcode" : "bar postcode 2"
},
{
  "id" : 2,
  "hash_id" : "barhash2",
  "source_type" : "BAR_EX_RES",
  "paf_udprn" : "bar udprn 2",
  "paf_postcode" : "bar postcode 2"
}
]'

 

3) Querying the main index:

curl "http://localhost:8080/solr_example/query?q=*:*"
{                                                                                      
  "responseHeader":{                                                                  
  "status":0,                                                                       
  "QTime":2,                                                                         
  "params":{                                                                        
  "q":"*:*"}},                                                                    
  "response":{"numFound":2,"start":0,"docs":[                                         
      {                                                                               
        "id":2,                                                                       
        "hash_id":"foohash2",                                                         
        "source_type":"FOO_EX_RES",                                                   
        "paf_udprn":"foo udprn 2",                                                     
        "paf_postcode":"foo postcode 2",                                              
        "_version_":1489835798979149824},                                             
      {                                                                                
        "id":1,                                                                       
        "hash_id":"foohash1",                                                         
        "source_type":"FOO_EX_RES",                                                    
        "paf_udprn":"foo udprn 1",                                                    
        "paf_postcode":"foo postcode 1",                                              
        "_version_":1489838391616864256}]                                              
  }}                                                                                  

  

4) Swapping the cores:

curl "http://localhost:8080/solr_example/admin/cores?action=SWAP&core=example_write&other=example"

5) Querying the main index again:

curl "http://localhost:8080/solr_example/query?q=*:*"
{                                                                  
  "responseHeader":{                                               
    "status":0,                                                     
    "QTime":1,                                                     
    "params":{                                                     
      "q":"*:*"}},                                                 
  "response":{"numFound":2,"start":0,"docs":[                      
      {                                                            
        "id":1,                                                    
        "hash_id":"barhash1",                                      
        "source_type":"BAR_EX_RES",                                
        "paf_udprn":"bar udprn 2",                                 
        "paf_postcode":"bar postcode 2",                            
        "_version_":1490093803902599168},                          
      {                                                            
        "id":2,                                                    
        "hash_id":"barhash2",                                      
        "source_type":"BAR_EX_RES",                                
        "paf_udprn":"bar udprn 2",                                 
        "paf_postcode":"bar postcode 2",                           
        "_version_":1490093803933007872}]                          
  }}                                                               

So as we can see what it was written in "WRITE" now is on the MAIN index.

If we want to rollback just run again:

curl "http://localhost:8080/solr_example/admin/cores?action=SWAP&core=example_write&other=example"

Until the swap is completed, the reading requests from the "search handler" keep on going to the old index. Once the swap is completed, the requests will go to the new index.
SWAP should be atomic and in our case should be fast too.