carlessanagustin
12/1/2016 - 4:56 PM

HADOOP: Accumulo replicating blocks issue

HADOOP: Accumulo replicating blocks issue

accumulo_issue.md
content_copyfile_download
  • Rendered
  • Source

Accumulo replicating blocks issue

Problem

Accumulo is set to HDFS replica 5 by default and my cluster is replica 3 with 3 workers only. When Accumulo TServer is started, blocks are sent to HDFS with replica 5 and HDFS complains of under replicated blocks.

Solution

We must change the default value from 5 to 3.

Steps

  • Enter the shell:
LOCATION=/usr/hdp/2.5.0.0-1245/accumulo/bin
$LOCATION/accumulo shell -u root
  • Check replication settings:
root@hdp-accumulo-instance> config -t accumulo.root -f table.file.replication
-----------+---------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
SCOPE      | NAME                      | VALUE
-----------+---------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
default    | table.file.replication .. | 0
table      |    @override ............ | 5
-----------+---------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
root@hdp-accumulo-instance> config -t accumulo.metadata -f table.file.replication
-----------+---------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
SCOPE      | NAME                      | VALUE
-----------+---------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
default    | table.file.replication .. | 0
table      |    @override ............ | 5
-----------+---------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
root@hdp-accumulo-instance>
  • Set replication settings:
root@hdp-accumulo-instance> config -t accumulo.root -s table.file.replication=3
root@hdp-accumulo-instance> config -t accumulo.metadata -s table.file.replication=3
root@hdp-accumulo-instance> exit

Restart services and fix under replicated blocks following this gist.

More