Table of Contents
HBase Shell commands are broken down into 13 groups to interact with HBase Database via HBase shell, let’s see usage, syntax, description, and examples of each in this article. From the below tables, the first table describes groups and all its commands in a cheat sheet and the remaining tables provide the detail description of each group and its commands.
HBase Shell Commands by Group
On the below table click on links to check usage, description, and examples for each HBase shell group or commands. You can also get the usage of each by running help ‘<command>’ | ‘<group-name>’ or just entering command name without parameters on the HBase shell.
If you do not have HBase setup and running on your system, I would recommend to have the setup and start using the Hbase shell.
While trying these commands, make sure table names, rows, columns all enclosed in quote characters.
HBase General Shell Commands
These shell commands are commonly used to identify the version, status, of the database.
| Name | General HBase Shell Commands Usage |
|---|---|
status | 1 active master, 1 backup masters, 22 servers, 0 dead, 221.8182 average load |
version | 1.2.0-cdh5.14.4, rUnknown, Tue Jun 12 04:00:36 PDT 2018 |
whoami | Returns account and group information |
Data Manipulation Language (DML) Shell Commands
DML HBase shell commands include most commonly used commands to modify the data, for example, put – is used to insert the rows to the tables, get & scan – are used to retrieve the data, delete & truncate – are used to delete the data, append – is used to append the cells and there are many commands
| Command | Usage & Examples |
|---|---|
append | Appends a cell 'value' at specified table/row/column coordinates. hbase> append 't1', 'r1', 'c1', 'value', ATTRIBUTES=>{'mykey'=>'myvalue'} hbase> append 't1', 'r1', 'c1', 'value', {VISIBILITY=>'PRIVATE|SECRET'} The same commands also can be run on a table reference. Suppose you had a reference t to table 't1', the corresponding command would be: hbase> t.append 'r1', 'c1', 'value', ATTRIBUTES=>{'mykey'=>'myvalue'} hbase> t.append 'r1', 'c1', 'value', {VISIBILITY=>'PRIVATE|SECRET'} |
count | Count the number of rows in a table. Return value is the number of rows. This operation may take a LONG time (Run '$HADOOP_HOME/bin/hadoop jar hbase.jar rowcount' to run a counting mapreduce job). Current count is shown every 1000 rows by default. Count interval may be optionally specified. Scan caching is enabled on count scans by default. Default cache size is 10 rows. If your rows are small in size, you may want to increase this parameter. Examples: hbase> count 'ns1:t1' hbase> count 't1' hbase> count 't1', INTERVAL => 100000 hbase> count 't1', CACHE => 1000 hbase> count 't1', INTERVAL => 10, CACHE => 1000 The same commands also can be run on a table reference. Suppose you had a reference t to table 't1', the corresponding commands would be: hbase> t.count hbase> t.count INTERVAL => 100000 hbase> t.count CACHE => 1000 hbase> t.count INTERVAL => 10, CACHE => 1000 |
delete | Put a delete cell value at specified table/row/column and optionally timestamp coordinates. Deletes must match the deleted cell's coordinates exactly. When scanning, a delete cell suppresses older versions. To delete a cell from 't1' at row 'r1' under column 'c1' marked with the time 'ts1', do: hbase> delete 'ns1:t1', 'r1', 'c1', ts1 hbase> delete 't1', 'r1', 'c1', ts1 hbase> delete 't1', 'r1', 'c1', ts1, {VISIBILITY=>'PRIVATE|SECRET'} The same command can also be run on a table reference. Suppose you had a reference t to table 't1', the corresponding command would be: hbase> t.delete 'r1', 'c1', ts1 hbase> t.delete 'r1', 'c1', ts1, {VISIBILITY=>'PRIVATE|SECRET'} |
deleteall | Delete all cells in a given row; pass a table name, row, and optionally a column and timestamp. Examples: hbase> deleteall 'ns1:t1', 'r1' hbase> deleteall 't1', 'r1' hbase> deleteall 't1', 'r1', 'c1' hbase> deleteall 't1', 'r1', 'c1', ts1 hbase> deleteall 't1', 'r1', 'c1', ts1, {VISIBILITY=>'PRIVATE|SECRET'} The same commands also can be run on a table reference. Suppose you had a reference t to table 't1', the corresponding command would be: hbase> t.deleteall 'r1' hbase> t.deleteall 'r1', 'c1' hbase> t.deleteall 'r1', 'c1', ts1 hbase> t.deleteall 'r1', 'c1', ts1, {VISIBILITY=>'PRIVATE|SECRET'} |
get | Get row or cell contents; pass table name, row, and optionally a dictionary of column(s), timestamp, timerange and versions. Examples: hbase> get 'ns1:t1', 'r1' hbase> get 't1', 'r1' hbase> get 't1', 'r1', {TIMERANGE => [ts1, ts2]} hbase> get 't1', 'r1', {COLUMN => 'c1'} hbase> get 't1', 'r1', {COLUMN => ['c1', 'c2', 'c3']} hbase> get 't1', 'r1', {COLUMN => 'c1', TIMESTAMP => ts1} hbase> get 't1', 'r1', {COLUMN => 'c1', TIMERANGE => [ts1, ts2], VERSIONS => 4} hbase> get 't1', 'r1', {COLUMN => 'c1', TIMESTAMP => ts1, VERSIONS => 4} hbase> get 't1', 'r1', {FILTER => "ValueFilter(=, 'binary:abc')"} hbase> get 't1', 'r1', 'c1' hbase> get 't1', 'r1', 'c1', 'c2' hbase> get 't1', 'r1', ['c1', 'c2'] hbase> get 't1', 'r1', {COLUMN => 'c1', ATTRIBUTES => {'mykey'=>'myvalue'}} hbase> get 't1', 'r1', {COLUMN => 'c1', AUTHORIZATIONS => ['PRIVATE','SECRET']} hbase> get 't1', 'r1', {CONSISTENCY => 'TIMELINE'} hbase> get 't1', 'r1', {CONSISTENCY => 'TIMELINE', REGION_REPLICA_ID => 1} Besides the default 'toStringBinary' format, 'get' also supports custom formatting by column. A user can define a FORMATTER by adding it to the column name in the get specification. The FORMATTER can be stipulated: 1. either as a org.apache.hadoop.hbase.util.Bytes method name (e.g, toInt, toString) 2. or as a custom class followed by method name: e.g. 'c(MyFormatterClass).format'. Example formatting cf:qualifier1 and cf:qualifier2 both as Integers: hbase> get 't1', 'r1' {COLUMN => ['cf:qualifier1:toInt', 'cf:qualifier2:c(org.apache.hadoop.hbase.util.Bytes).toInt'] } Note that you can specify a FORMATTER by column only (cf:qualifier). You cannot specify a FORMATTER for all columns of a column family. The same commands also can be run on a reference to a table (obtained via get_table or create_table). Suppose you had a reference t to table 't1', the corresponding commands would be: hbase> t.get 'r1' hbase> t.get 'r1', {TIMERANGE => [ts1, ts2]} hbase> t.get 'r1', {COLUMN => 'c1'} hbase> t.get 'r1', {COLUMN => ['c1', 'c2', 'c3']} hbase> t.get 'r1', {COLUMN => 'c1', TIMESTAMP => ts1} hbase> t.get 'r1', {COLUMN => 'c1', TIMERANGE => [ts1, ts2], VERSIONS => 4} hbase> t.get 'r1', {COLUMN => 'c1', TIMESTAMP => ts1, VERSIONS => 4} hbase> t.get 'r1', {FILTER => "ValueFilter(=, 'binary:abc')"} hbase> t.get 'r1', 'c1' hbase> t.get 'r1', 'c1', 'c2' hbase> t.get 'r1', ['c1', 'c2'] hbase> t.get 'r1', {CONSISTENCY => 'TIMELINE'} hbase> t.get 'r1', {CONSISTENCY => 'TIMELINE', REGION_REPLICA_ID => 1} |
get_counter | Return a counter cell value at specified table/row/column coordinates. A counter cell should be managed with atomic increment functions on HBase and the data should be binary encoded (as long value). Example: hbase> get_counter 'ns1:t1', 'r1', 'c1' hbase> get_counter 't1', 'r1', 'c1' The same commands also can be run on a table reference. Suppose you had a reference t to table 't1', the corresponding command would be: hbase> t.get_counter 'r1', 'c1' |
get_splits | Get the splits of the named table: hbase> get_splits 't1' hbase> get_splits 'ns1:t1' The same commands also can be run on a table reference. Suppose you had a reference t to table 't1', the corresponding command would be: hbase> t.get_splits |
incr | Increments a cell 'value' at specified table/row/column coordinates. To increment a cell value in table 'ns1:t1' or 't1' at row 'r1' under column 'c1' by 1 (can be omitted) or 10 do: hbase> incr 'ns1:t1', 'r1', 'c1' hbase> incr 't1', 'r1', 'c1' hbase> incr 't1', 'r1', 'c1', 1 hbase> incr 't1', 'r1', 'c1', 10 hbase> incr 't1', 'r1', 'c1', 10, {ATTRIBUTES=>{'mykey'=>'myvalue'}} hbase> incr 't1', 'r1', 'c1', {ATTRIBUTES=>{'mykey'=>'myvalue'}} hbase> incr 't1', 'r1', 'c1', 10, {VISIBILITY=>'PRIVATE|SECRET'} The same commands also can be run on a table reference. Suppose you had a reference t to table 't1', the corresponding command would be: hbase> t.incr 'r1', 'c1' hbase> t.incr 'r1', 'c1', 1 hbase> t.incr 'r1', 'c1', 10, {ATTRIBUTES=>{'mykey'=>'myvalue'}} hbase> t.incr 'r1', 'c1', 10, {VISIBILITY=>'PRIVATE|SECRET'} |
put | Put a cell 'value' at specified table/row/column and optionally timestamp coordinates. To put a cell value into table 'ns1:t1' or 't1' at row 'r1' under column 'c1' marked with the time 'ts1', do: hbase> put 'ns1:t1', 'r1', 'c1', 'value' hbase> put 't1', 'r1', 'c1', 'value' hbase> put 't1', 'r1', 'c1', 'value', ts1 hbase> put 't1', 'r1', 'c1', 'value', {ATTRIBUTES=>{'mykey'=>'myvalue'}} hbase> put 't1', 'r1', 'c1', 'value', ts1, {ATTRIBUTES=>{'mykey'=>'myvalue'}} hbase> put 't1', 'r1', 'c1', 'value', ts1, {VISIBILITY=>'PRIVATE|SECRET'} The same commands also can be run on a table reference. Suppose you had a reference t to table 't1', the corresponding command would be: hbase> t.put 'r1', 'c1', 'value', ts1, {ATTRIBUTES=>{'mykey'=>'myvalue'}} |
scan | Scan a table; pass table name and optionally a dictionary of scanner specifications. Scanner specifications may include one or more of: TIMERANGE, FILTER, LIMIT, STARTROW, STOPROW, ROWPREFIXFILTER, TIMESTAMP, MAXLENGTH or COLUMNS, CACHE or RAW, VERSIONS, ALL_METRICS or METRICS If no columns are specified, all columns will be scanned. To scan all members of a column family, leave the qualifier empty as in 'col_family'. The filter can be specified in two ways: 1. Using a filterString - more information on this is available in the Filter Language document attached to the HBASE-4176 JIRA 2. Using the entire package name of the filter. If you wish to see metrics regarding the execution of the scan, the ALL_METRICS boolean should be set to true. Alternatively, if you would prefer to see only a subset of the metrics, the METRICS array can be defined to include the names of only the metrics you care about. Some examples: hbase> scan 'hbase:meta' hbase> scan 'hbase:meta', {COLUMNS => 'info:regioninfo'} hbase> scan 'ns1:t1', {COLUMNS => ['c1', 'c2'], LIMIT => 10, STARTROW => 'xyz'} hbase> scan 't1', {COLUMNS => ['c1', 'c2'], LIMIT => 10, STARTROW => 'xyz'} hbase> scan 't1', {COLUMNS => 'c1', TIMERANGE => [1303668804, 1303668904]} hbase> scan 't1', {REVERSED => true} hbase> scan 't1', {ALL_METRICS => true} hbase> scan 't1', {METRICS => ['RPC_RETRIES', 'ROWS_FILTERED']} hbase> scan 't1', {ROWPREFIXFILTER => 'row2', FILTER => " (QualifierFilter (>=, 'binary:xyz')) AND (TimestampsFilter ( 123, 456))"} hbase> scan 't1', {FILTER => org.apache.hadoop.hbase.filter.ColumnPaginationFilter.new(1, 0)} hbase> scan 't1', {CONSISTENCY => 'TIMELINE'} For setting the Operation Attributes hbase> scan 't1', { COLUMNS => ['c1', 'c2'], ATTRIBUTES => {'mykey' => 'myvalue'}} hbase> scan 't1', { COLUMNS => ['c1', 'c2'], AUTHORIZATIONS => ['PRIVATE','SECRET']} For experts, there is an additional option -- CACHE_BLOCKS -- which switches block caching for the scanner on (true) or off (false). By default it is enabled. Examples: hbase> scan 't1', {COLUMNS => ['c1', 'c2'], CACHE_BLOCKS => false} Also for experts, there is an advanced option -- RAW -- which instructs the scanner to return all cells (including delete markers and uncollected deleted cells). This option cannot be combined with requesting specific COLUMNS. Disabled by default. Example: hbase> scan 't1', {RAW => true, VERSIONS => 10} Besides the default 'toStringBinary' format, 'scan' supports custom formatting by column. A user can define a FORMATTER by adding it to the column name in the scan specification. The FORMATTER can be stipulated: 1. either as a org.apache.hadoop.hbase.util.Bytes method name (e.g, toInt, toString) 2. or as a custom class followed by method name: e.g. 'c(MyFormatterClass).format'. Example formatting cf:qualifier1 and cf:qualifier2 both as Integers: hbase> scan 't1', {COLUMNS => ['cf:qualifier1:toInt', 'cf:qualifier2:c(org.apache.hadoop.hbase.util.Bytes).toInt'] } Note that you can specify a FORMATTER by column only (cf:qualifier). You cannot specify a FORMATTER for all columns of a column family. Scan can also be used directly from a table, by first getting a reference to a table, like such: hbase> t = get_table 't' hbase> t.scan Note in the above situation, you can still provide all the filtering, columns, options, etc as described above. |
truncate | Disables, drops and recreates the specified table. |
truncate_preserve | Disables, drops and recreates the specified table while still maintaing the previous region boundaries. |
Data Definition Language (DDL) Shell Commands
DDL HBase shell commands are another set of commands used mostly to change the structure of the table, for example, alter – is used to delete column family from a table or any alteration to the table. before you run alter make sure you disable the table first. create – is used to create a table, drop – to drop the table and many more.
| Command | Usage & Examples |
|---|---|
alter | If the "hbase.online.schema.update.enable" property is set to false, then the table must be disabled (see help 'disable'). If the "hbase.online.schema.update.enable" property is set to true, tables can be altered without disabling them first. Altering enabled tables has caused problems in the past, so use caution and test it before using in production. You can use the alter command to add, modify or delete column families or change table configuration options. Column families work in a similar way as the 'create' command. The column family specification can either be a name string, or a dictionary with the NAME attribute. Dictionaries are described in the output of the 'help' command, with no arguments. For example, to change or add the 'f1' column family in table 't1' from current value to keep a maximum of 5 cell VERSIONS, do: hbase> alter 't1', NAME => 'f1', VERSIONS => 5 You can operate on several column families: hbase> alter 't1', 'f1', {NAME => 'f2', IN_MEMORY => true}, {NAME => 'f3', VERSIONS => 5} To delete the 'f1' column family in table 'ns1:t1', use one of: hbase> alter 'ns1:t1', NAME => 'f1', METHOD => 'delete' hbase> alter 'ns1:t1', 'delete' => 'f1' You can also change table-scope attributes like MAX_FILESIZE, READONLY, MEMSTORE_FLUSHSIZE, DURABILITY, etc. These can be put at the end; for example, to change the max size of a region to 128MB, do: hbase> alter 't1', MAX_FILESIZE => '134217728' You can add a table coprocessor by setting a table coprocessor attribute: hbase> alter 't1', 'coprocessor'=>'hdfs:///foo.jar|com.foo.FooRegionObserver|1001|arg1=1,arg2=2' Since you can have multiple coprocessors configured for a table, a sequence number will be automatically appended to the attribute name to uniquely identify it. The coprocessor attribute must match the pattern below in order for the framework to understand how to load the coprocessor classes: [coprocessor jar file location] | class name | [priority] | [arguments] You can also set configuration settings specific to this table or column family: hbase> alter 't1', CONFIGURATION => {'hbase.hregion.scan.loadColumnFamiliesOnDemand' => 'true'} hbase> alter 't1', {NAME => 'f2', CONFIGURATION => {'hbase.hstore.blockingStoreFiles' => '10'}} You can also remove a table-scope attribute: hbase> alter 't1', METHOD => 'table_att_unset', NAME => 'MAX_FILESIZE' hbase> alter 't1', METHOD => 'table_att_unset', NAME => 'coprocessor$1' You can also set REGION_REPLICATION: hbase> alter 't1', {REGION_REPLICATION => 2} There could be more than one alteration in one command: hbase> alter 't1', { NAME => 'f1', VERSIONS => 3 }, { MAX_FILESIZE => '134217728' }, { METHOD => 'delete', NAME => 'f2' }, OWNER => 'johndoe', METADATA => { 'mykey' => 'myvalue' } |
alter_async | Alter column family schema, does not wait for all regions to receive the schema changes. Pass table name and a dictionary specifying new column family schema. Dictionaries are described on the main help command output. Dictionary must include name of column family to alter. For example, To change or add the 'f1' column family in table 't1' from defaults to instead keep a maximum of 5 cell VERSIONS, do: hbase> alter_async 't1', NAME => 'f1', VERSIONS => 5 To delete the 'f1' column family in table 'ns1:t1', do: hbase> alter_async 'ns1:t1', NAME => 'f1', METHOD => 'delete' or a shorter version: hbase> alter_async 'ns1:t1', 'delete' => 'f1' You can also change table-scope attributes like MAX_FILESIZE MEMSTORE_FLUSHSIZE, READONLY, and DEFERRED_LOG_FLUSH. For example, to change the max size of a family to 128MB, do: hbase> alter 't1', METHOD => 'table_att', MAX_FILESIZE => '134217728' There could be more than one alteration in one command: hbase> alter 't1', {NAME => 'f1'}, {NAME => 'f2', METHOD => 'delete'} To check if all the regions have been updated, use alter_status |
alter_status | Get the status of the alter command. Indicates the number of regions of the table that have received the updated schema Pass table name. hbase> alter_status 't1' hbase> alter_status 'ns1:t1' |
create | Creates a table. Pass a table name, and a set of column family specifications (at least one), and, optionally, table configuration. Column specification can be a simple string (name), or a dictionary (dictionaries are described below in main help output), necessarily including NAME attribute. Examples: Create a table with namespace=ns1 and table qualifier=t1 hbase> create 'ns1:t1', {NAME => 'f1', VERSIONS => 5} Create a table with namespace=default and table qualifier=t1 hbase> create 't1', {NAME => 'f1'}, {NAME => 'f2'}, {NAME => 'f3'} hbase> # The above in shorthand would be the following: hbase> create 't1', 'f1', 'f2', 'f3' hbase> create 't1', {NAME => 'f1', VERSIONS => 1, TTL => 2592000, BLOCKCACHE => true} hbase> create 't1', {NAME => 'f1', CONFIGURATION => {'hbase.hstore.blockingStoreFiles' => '10'}} hbase> create 't1', {NAME => 'f1', IS_MOB => true, MOB_THRESHOLD => 1000000, MOB_COMPACT_PARTITION_POLICY => 'weekly'} Table configuration options can be put at the end. Examples: hbase> create 'ns1:t1', 'f1', SPLITS => ['10', '20', '30', '40'] hbase> create 't1', 'f1', SPLITS => ['10', '20', '30', '40'] hbase> create 't1', 'f1', SPLITS_FILE => 'splits.txt', OWNER => 'johndoe' hbase> create 't1', {NAME => 'f1', VERSIONS => 5}, METADATA => { 'mykey' => 'myvalue' } hbase> # Optionally pre-split the table into NUMREGIONS, using hbase> # SPLITALGO ("HexStringSplit", "UniformSplit" or classname) hbase> create 't1', 'f1', {NUMREGIONS => 15, SPLITALGO => 'HexStringSplit'} hbase> create 't1', 'f1', {NUMREGIONS => 15, SPLITALGO => 'HexStringSplit', REGION_REPLICATION => 2, CONFIGURATION => {'hbase.hregion.scan.loadColumnFamiliesOnDemand' => 'true'}} hbase> create 't1', {NAME => 'f1', DFS_REPLICATION => 1} You can also keep around a reference to the created table: hbase> t1 = create 't1', 'f1' Which gives you a reference to the table named 't1', on which you can then call methods. |
describe | Describe the named table. For example: hbase> describe 't1' hbase> describe 'ns1:t1' Alternatively, you can use the abbreviated 'desc' for the same thing. hbase> desc 't1' hbase> desc 'ns1:t1' |
disable | Start disable of named table: hbase> disable 't1' hbase> disable 'ns1:t1' |
disable_all | |
drop | Drop the named table. Table must first be disabled: hbase> drop 't1' hbase> drop 'ns1:t1' |
drop_all | Drop all of the tables matching the given regex: hbase> drop_all 't.*' hbase> drop_all 'ns:t.*' hbase> drop_all 'ns:.*' |
enable | Start enable of named table: hbase> enable 't1' hbase> enable 'ns1:t1' |
enable_all | Enable all of the tables matching the given regex: hbase> enable_all 't.*' hbase> enable_all 'ns:t.*' hbase> enable_all 'ns:.*' |
exists | Does the named table exist? hbase> exists 't1' hbase> exists 'ns1:t1' |
get_table | Get the given table name and return it as an actual object to be manipulated by the user. See table.help for more information on how to use the table. Eg. hbase> t1 = get_table 't1' hbase> t1 = get_table 'ns1:t1' returns the table named 't1' as a table object. You can then do hbase> t1.help which will then print the help for that table. |
is_disabled | Is named table disabled? For example: hbase> is_disabled 't1' hbase> is_disabled 'ns1:t1' |
is_enabled | Is named table enabled? For example: hbase> is_enabled 't1' hbase> is_enabled 'ns1:t1' |
list | List all tables TABLE hbase> list table1 table2 |
locate_region | Locate the region given a table name and a row-key hbase> locate_region 'tableName', 'key0' |
show_filters | hbas> show_filters DependentColumnFilter KeyOnlyFilter ColumnCountGetFilter SingleColumnValueFilter PrefixFilter SingleColumnValueExcludeFilter FirstKeyOnlyFilter ColumnRangeFilter TimestampsFilter FamilyFilter QualifierFilter ColumnPrefixFilter RowFilter MultipleColumnPrefixFilter InclusiveStopFilter PageFilter ValueFilter ColumnPaginationFilter |
Namespace Commands Syntax and Usage
This group contains commands to alter & create the namespace of the HBase database.
| Command | Usage & Examples |
|---|---|
alter_namespace | Alter namespace properties. To add/modify a property: hbase> alter_namespace 'ns1', {METHOD => 'set', 'PROPERTY_NAME' => 'PROPERTY_VALUE'} To delete a property: hbase> alter_namespace 'ns1', {METHOD => 'unset', NAME=>'PROPERTY_NAME'} |
create_namespace | Create namespace; pass namespace name, and optionally a dictionary of namespace configuration. Examples: hbase> create_namespace 'ns1' hbase> create_namespace 'ns1', {'PROPERTY_NAME'=>'PROPERTY_VALUE'} |
describe_namespace | Describe the named namespace. For example: hbase> describe_namespace 'ns1' |
drop_namespace | Drop the named namespace. The namespace must be empty. |
list_namespace | List all namespaces in hbase. Optional regular expression parameter could be used to filter the output. Examples: hbase> list_namespace hbase> list_namespace 'abc.*' |
list_namespace_tables | List all tables that are members of the namespace. Examples: hbase> list_namespace_tables 'ns1' |
Tool Syntax and Usage
| Commands | Usage & Examples |
|---|---|
assign | Assign a region. Use with caution. If region already assigned, this command will do a force reassign. For experts only. Examples: hbase> assign 'REGIONNAME' hbase> assign 'ENCODED_REGIONNAME' |
balance_switch | Enable/Disable balancer. Returns previous balancer state. Examples: hbase> balance_switch true hbase> balance_switch false |
balancer | Trigger the cluster balancer. Returns true if balancer ran and was able to tell the region servers to unassign all the regions to balance (the re-assignment itself is async). Otherwise false (Will not run if regions in transition). Command: balancer_enabled Query the balancer's state. Examples: hbase> balancer_enabled |
catalogjanitor_enabled | Query for the CatalogJanitor state (enabled/disabled?) Examples: hbase> catalogjanitor_enabled |
catalogjanitor_run | Catalog janitor command to run the (garbage collection) scan from command line. hbase> catalogjanitor_run |
catalogjanitor_switch | Enable/Disable CatalogJanitor. Returns previous CatalogJanitor state. Examples: hbase> catalogjanitor_switch true hbase> catalogjanitor_switch false |
close_region | Close a single region. Ask the master to close a region out on the cluster or if 'SERVER_NAME' is supplied, ask the designated hosting regionserver to close the region directly. Closing a region, the master expects 'REGIONNAME' to be a fully qualified region name. When asking the hosting regionserver to directly close a region, you pass the regions' encoded name only. A region name looks like this: TestTable,0094429456,1289497600452.527db22f95c8a9e0116f0cc13c680396. or Namespace:TestTable,0094429456,1289497600452.527db22f95c8a9e0116f0cc13c680396. The trailing period is part of the regionserver name. A region's encoded name is the hash at the end of a region name; e.g. 527db22f95c8a9e0116f0cc13c680396 (without the period). A 'SERVER_NAME' is its host, port plus startcode. For example: host187.example.com,60020,1289493121758 (find servername in master ui or when you do detailed status in shell). This command will end up running close on the region hosting regionserver. The close is done without the master's involvement (It will not know of the close). Once closed, region will stay closed. Use assign to reopen/reassign. Use unassign or move to assign the region elsewhere on cluster. Use with caution. For experts only. Examples: hbase> close_region 'REGIONNAME' hbase> close_region 'REGIONNAME', 'SERVER_NAME' hbase> close_region 'ENCODED_REGIONNAME' hbase> close_region 'ENCODED_REGIONNAME', 'SERVER_NAME' |
compact | Compact all regions in passed table or pass a region row to compact an individual region. You can also compact a single column family within a region. Examples: Compact all regions in a table: hbase> compact 'ns1:t1' hbase> compact 't1' Compact an entire region: hbase> compact 'r1' Compact only a column family within a region: hbase> compact 'r1', 'c1' Compact a column family within a table: hbase> compact 't1', 'c1' |
compact_mob | Run compaction on a mob enabled column family or all mob enabled column families within a table Examples: Compact a column family within a table: hbase> compact_mob 't1', 'c1' Compact all mob enabled column families hbase> compact_mob 't1' |
compact_rs | Compact all regions on passed regionserver. Examples: Compact all regions on a regionserver: hbase> compact_rs 'host187.example.com,60020' or hbase> compact_rs 'host187.example.com,60020,1289493121758' Major compact all regions on a regionserver: hbase> compact_rs 'host187.example.com,60020,1289493121758', true |
flush | Flush all regions in passed table or pass a region row to flush an individual region. For example: hbase> flush 'TABLENAME' hbase> flush 'REGIONNAME' hbase> flush 'ENCODED_REGIONNAME' |
major_compact | Run major compaction on passed table or pass a region row to major compact an individual region. To compact a single column family within a region specify the region name followed by the column family name. Examples: Compact all regions in a table: hbase> major_compact 't1' hbase> major_compact 'ns1:t1' Compact an entire region: hbase> major_compact 'r1' Compact a single column family within a region: hbase> major_compact 'r1', 'c1' Compact a single column family within a table: hbase> major_compact 't1', 'c1' |
major_compact_mob | Run major compaction on a mob enabled column family or all mob enabled column families within a table Examples: Compact a column family within a table: hbase> major_compact_mob 't1', 'c1' Compact all mob enabled column families within a table hbase> major_compact_mob 't1' |
merge_region | Merge two regions. Passing 'true' as the optional third parameter will force a merge ('force' merges regardless else merge will fail unless passed adjacent regions. 'force' is for expert use only). NOTE: You must pass the encoded region name, not the full region name so this command is a little different from other region operations. The encoded region name is the hash suffix on region names: e.g. if the region name were TestTable,0094429456,1289497600452.527db22f95c8a9e0116f0cc13c680396. then the encoded region name portion is 527db22f95c8a9e0116f0cc13c680396 Examples: hbase> merge_region 'ENCODED_REGIONNAME', 'ENCODED_REGIONNAME' hbase> merge_region 'ENCODED_REGIONNAME', 'ENCODED_REGIONNAME', true |
move | Move a region. Optionally specify target regionserver else we choose one at random. NOTE: You pass the encoded region name, not the region name so this command is a little different to the others. The encoded region name is the hash suffix on region names: e.g. if the region name were TestTable,0094429456,1289497600452.527db22f95c8a9e0116f0cc13c680396. then the encoded region name portion is 527db22f95c8a9e0116f0cc13c680396 A server name is its host, port plus startcode. For example: host187.example.com,60020,1289493121758 Examples: hbase> move 'ENCODED_REGIONNAME' hbase> move 'ENCODED_REGIONNAME', 'SERVER_NAME' |
normalize | Trigger region normalizer for all tables which have NORMALIZATION_ENABLED flag set. Returns true if normalizer ran successfully, false otherwise. Note that this command has no effect if region normalizer is disabled (make sure it's turned on using 'normalizer_switch' command). Examples: hbase> normalize |
normalizer_enabled | Query the state of region normalizer. Examples: hbase> normalizer_enabled |
normalizer_switch | Enable/Disable region normalizer. Returns previous normalizer state. When normalizer is enabled, it handles all tables with 'NORMALIZATION_ENABLED' => true. Examples: hbase> normalizer_switch true hbase> normalizer_switch false |
split | Split entire table or pass a region to split individual region. With the second parameter, you can specify an explicit split key for the region. Examples: split 'tableName' split 'namespace:tableName' split 'regionName' # format: 'tableName,startKey,id' split 'tableName', 'splitKey' split 'regionName', 'splitKey' |
trace | Start or Stop tracing using HTrace. Always returns true if tracing is running, otherwise false. If the first argument is 'start', new span is started. If the first argument is 'stop', current running span is stopped. ('stop' returns false on success.) If the first argument is 'status', just returns if or not tracing is running. On 'start'-ing, you can optionally pass the name of span as the second argument. The default name of span is 'HBaseShell'. Repeating 'start' does not start nested span. Examples: hbase> trace 'start' hbase> trace 'status' hbase> trace 'stop' hbase> trace 'start', 'MySpanName' hbase> trace 'stop' |
unassign | Unassign a region. Unassign will close region in current location and then reopen it again. Pass 'true' to force the unassignment ('force' will clear all in-memory state in master before the reassign. If results in double assignment use hbck -fix to resolve. To be used by experts). Use with caution. For expert use only. Examples: hbase> unassign 'REGIONNAME' hbase> unassign 'REGIONNAME', true hbase> unassign 'ENCODED_REGIONNAME' hbase> unassign 'ENCODED_REGIONNAME', true |
wal_roll | Roll the log writer. That is, start writing log messages to a new file. The name of the regionserver should be given as the parameter. A 'server_name' is the host, port plus startcode of a regionserver. For example: host187.example.com,60020,1289493121758 (find servername in master ui or when you do detailed status in shell) |
zk_dump | Dump status of HBase cluster as seen by ZooKeeper. |
Replication Syntax and Usage
Note: In order to use these tools, hbase.replication must be set to true and commands in these groups are mainly used to add or remove a peer from an HBase cluster.
| Commands | Usage & Examples |
|---|---|
add_peer | A peer can either be another HBase cluster or a custom replication endpoint. In either case an id must be specified to identify the peer. For a HBase cluster peer, a cluster key must be provided and is composed like this: hbase.zookeeper.quorum:hbase.zookeeper.property.clientPort:zookeeper.znode.parent This gives a full path for HBase to connect to another HBase cluster. An optional parameter for table column families identifies which column families will be replicated to the peer cluster. Examples: hbase> add_peer '1', "server1.cie.com:2181:/hbase" hbase> add_peer '2', "zk1,zk2,zk3:2182:/hbase-prod" hbase> add_peer '3', "zk4,zk5,zk6:11000:/hbase-test", "table1; table2:cf1; table3:cf1,cf2" hbase> add_peer '4', CLUSTER_KEY => "server1.cie.com:2181:/hbase" hbase> add_peer '5', CLUSTER_KEY => "zk1,zk2,zk3:2182:/hbase-prod", TABLE_CFS => { "table1" => [], "ns2:table2" => ["cf1"], "ns3:table3" => ["cf1", "cf2"] } For a custom replication endpoint, the ENDPOINT_CLASSNAME can be provided. Two optional arguments are DATA and CONFIG which can be specified to set different either the peer_data or configuration for the custom replication endpoint. Table column families is optional and can be specified with the key TABLE_CFS. hbase> add_peer '6', ENDPOINT_CLASSNAME => 'org.apache.hadoop.hbase.MyReplicationEndpoint' hbase> add_peer '7', ENDPOINT_CLASSNAME => 'org.apache.hadoop.hbase.MyReplicationEndpoint', DATA => { "key1" => 1 } hbase> add_peer '8', ENDPOINT_CLASSNAME => 'org.apache.hadoop.hbase.MyReplicationEndpoint', CONFIG => { "config1" => "value1", "config2" => "value2" } hbase> add_peer '9', ENDPOINT_CLASSNAME => 'org.apache.hadoop.hbase.MyReplicationEndpoint', DATA => { "key1" => 1 }, CONFIG => { "config1" => "value1", "config2" => "value2" }, hbase> add_peer '10', ENDPOINT_CLASSNAME => 'org.apache.hadoop.hbase.MyReplicationEndpoint', TABLE_CFS => { "table1" => [], "ns2:table2" => ["cf1"], "ns3:table3" => ["cf1", "cf2"] } hbase> add_peer '11', ENDPOINT_CLASSNAME => 'org.apache.hadoop.hbase.MyReplicationEndpoint', DATA => { "key1" => 1 }, CONFIG => { "config1" => "value1", "config2" => "value2" }, TABLE_CFS => { "table1" => [], "table2" => ["cf1"], "table3" => ["cf1", "cf2"] } Note: Either CLUSTER_KEY or ENDPOINT_CLASSNAME must be specified but not both. |
append_peer_tableCFs | Append a replicable table-cf config for the specified peer Examples: # append a table / table-cf to be replicable for a peer hbase> append_peer_tableCFs '2', { "ns1:table4" => ["cfA", "cfB"] } |
disable_peer | Stops the replication stream to the specified cluster, but still keeps track of new edits to replicate. Examples: hbase> disable_peer '1' |
disable_table_replication | Disable a table's replication switch. Examples: hbase> disable_table_replication 'table_name' |
enable_peer | Restarts the replication to the specified peer cluster, continuing from where it was disabled. Examples: hbase> enable_peer '1' |
enable_table_replication | Enable a table's replication switch. Examples: hbase> enable_table_replication 'table_name' |
get_peer_config | Outputs the cluster key, replication endpoint class (if present), and any replication configuration parameters |
list_peer_configs | No-argument method that outputs the replication peer configuration for each peer defined on this cluster. |
list_peers | List all replication peer clusters. hbase> list_peers |
list_replicated_tables | List all the tables and column families replicated from this cluster hbase> list_replicated_tables hbase> list_replicated_tables 'abc.*' |
remove_peer | Stops the specified replication stream and deletes all the meta information kept about it. Examples: hbase> remove_peer '1' |
remove_peer_tableCFs | Remove a table / table-cf from the table-cfs config for the specified peer Examples: # Remove a table / table-cf from the replicable table-cfs for a peer hbase> remove_peer_tableCFs '2', { "ns1:table1" => [] } hbase> remove_peer_tableCFs '2', { "ns1:table1" => ["cf1"] } |
set_peer_tableCFs | Set the replicable table-cf config for the specified peer Examples: # set all tables to be replicable for a peer hbase> set_peer_tableCFs '1', "" hbase> set_peer_tableCFs '1' # set table / table-cf to be replicable for a peer, for a table without # an explicit column-family list, all replicable column-families (with # replication_scope == 1) will be replicated hbase> set_peer_tableCFs '2', { "ns1:table1" => [], "ns2:table2" => ["cf1", "cf2"], "ns3:table3" => ["cfA", "cfB"] } |
show_peer_tableCFs | Show replicable table-cf config for the specified peer. hbase> show_peer_tableCFs |
update_peer_config | A peer can either be another HBase cluster or a custom replication endpoint. In either case an id must be specified to identify the peer. This command does not interrupt processing on an enabled replication peer. Two optional arguments are DATA and CONFIG which can be specified to set different values for either the peer_data or configuration for a custom replication endpoint. Any existing values not updated by this command are left unchanged. CLUSTER_KEY, REPLICATION_ENDPOINT, and TABLE_CFs cannot be updated with this command. To update TABLE_CFs, see the append_peer_tableCFs and remove_peer_tableCFs commands. hbase> update_peer_config '1', DATA => { "key1" => 1 } hbase> update_peer_config '2', CONFIG => { "config1" => "value1", "config2" => "value2" } hbase> update_peer_config '3', DATA => { "key1" => 1 }, CONFIG => { "config1" => "value1", "config2" => "value2" }, |
Snapshot Shell commands
This group of commands is used to take the snapshot of the database at any given time.
| Commands | Usage & Examples |
|---|---|
| clone_snapshot | Create a new table by cloning the snapshot content. There're no copies of data involved. And writing on the newly created table will not influence the snapshot data. Examples: hbase> clone_snapshot 'snapshotName', 'tableName' hbase> clone_snapshot 'snapshotName', 'namespace:tableName' |
| delete_all_snapshot | Delete all of the snapshots matching the given regex. Examples: hbase> delete_all_snapshot 's.*' |
| delete_snapshot | Delete a specified snapshot. Examples: hbase> delete_snapshot 'snapshotName', |
| list_snapshots | List all snapshots taken (by printing the names and relative information). Optional regular expression parameter could be used to filter the output by snapshot name. Examples: hbase> list_snapshots hbase> list_snapshots 'abc.*' |
| restore_snapshot | Restore a specified snapshot. The restore will replace the content of the original table, bringing back the content to the snapshot state. The table must be disabled. Examples: hbase> restore_snapshot 'snapshotName' |
| snapshot | Take a snapshot of specified table. Examples: hbase> snapshot 'sourceTable', 'snapshotName' hbase> snapshot 'namespace:sourceTable', 'snapshotName', {SKIP_FLUSH => true} |
Configuration Syntax and Usage
| Commands | Usage & Examples |
|---|---|
update_all_config | Reload a subset of configuration on all servers in the cluster. See http://hbase.apache.org/book.html?dyn_config for more details. Here is how you would run the command in the hbase shell: hbase> update_all_config |
update_config | Reload a subset of configuration on server 'servername' where servername is host, port plus startcode. For example: host187.example.com,60020,1289493121758 See http://hbase.apache.org/book.html?dyn_config for more details. Here is how you would run the command in the hbase shell: hbase> update_config 'servername' |
Quota Syntax and Usage
| Commands | Usage & Examples |
|---|---|
list_quotas | List the quota settings added to the system. You can filter the result based on USER, TABLE, or NAMESPACE. For example: hbase> list_quotas hbase> list_quotas USER => 'bob.*' hbase> list_quotas USER => 'bob.*', TABLE => 't1' hbase> list_quotas USER => 'bob.*', NAMESPACE => 'ns.*' hbase> list_quotas TABLE => 'myTable' hbase> list_quotas NAMESPACE => 'ns.*' |
set_quota | Set a quota for a user, table, or namespace. Syntax : set_quota TYPE => TYPE => THROTTLE User can either set quota on read, write or on both the requests together(i.e., read+write) The read, write, or read+write(default throttle type) request limit can be expressed using the form 100req/sec, 100req/min and the read, write, read+write(default throttle type) limit can be expressed using the form 100k/sec, 100M/min with (B, K, M, G, T, P) as valid size unit and (sec, min, hour, day) as valid time unit. Currently the throttle limit is per machine - a limit of 100req/min means that each machine can execute 100req/min. For example: hbase> set_quota TYPE => THROTTLE, USER => 'u1', LIMIT => '10req/sec' hbase> set_quota TYPE => THROTTLE, THROTTLE_TYPE => READ, USER => 'u1', LIMIT => '10req/sec' hbase> set_quota TYPE => THROTTLE, USER => 'u1', LIMIT => '10M/sec' hbase> set_quota TYPE => THROTTLE, THROTTLE_TYPE => WRITE, USER => 'u1', LIMIT => '10M/sec' hbase> set_quota TYPE => THROTTLE, USER => 'u1', TABLE => 't2', LIMIT => '5K/min' hbase> set_quota TYPE => THROTTLE, USER => 'u1', NAMESPACE => 'ns2', LIMIT => NONE hbase> set_quota TYPE => THROTTLE, NAMESPACE => 'ns1', LIMIT => '10req/sec' hbase> set_quota TYPE => THROTTLE, TABLE => 't1', LIMIT => '10M/sec' hbase> set_quota TYPE => THROTTLE, THROTTLE_TYPE => WRITE, TABLE => 't1', LIMIT => '10M/sec' hbase> set_quota TYPE => THROTTLE, USER => 'u1', LIMIT => NONE hbase> set_quota TYPE => THROTTLE, THROTTLE_TYPE => WRITE, USER => 'u1', LIMIT => NONE hbase> set_quota USER => 'u1', GLOBAL_BYPASS => true |
HBase Security Syntax and Usage
Note: Security commands are only applicable if running with the AccessController coprocessor. These HBase shell commands are mostly used by an admin to make & provide security to the database and tables.
| Commands | Usage & Examples |
|---|---|
| grant | Grant users specific rights. permissions is either zero or more letters from the set "RWXCA". READ('R'), WRITE('W'), EXEC('X'), CREATE('C'), ADMIN('A') Note: Groups and users are granted access in the same way, but groups are prefixed with an '@' character. In the same way, tables and namespaces are specified, but namespaces are prefixed with an '@' character. For example: hbase> grant 'bobsmith', 'RWXCA' hbase> grant '@admins', 'RWXCA' hbase> grant 'bobsmith', 'RWXCA', '@ns1' hbase> grant 'bobsmith', 'RW', 't1', 'f1', 'col1' hbase> grant 'bobsmith', 'RW', 'ns1:t1', 'f1', 'col1' |
| list_security_capabilities | List supported security capabilities Example: hbase> list_security_capabilities |
| revoke | Revoke a user's access rights. Note: Groups and users access are revoked in the same way, but groups are prefixed with an '@' character. In the same way, tables and namespaces are specified, but namespaces are prefixed with an '@' character. For example: hbase> revoke 'bobsmith' hbase> revoke '@admins' hbase> revoke 'bobsmith', '@ns1' hbase> revoke 'bobsmith', 't1', 'f1', 'col1' hbase> revoke 'bobsmith', 'ns1:t1', 'f1', 'col1' |
| user_permission | Show all permissions for the particular user. Syntax : user_permission table Note: A namespace must always precede with '@' character. For example: hbase> user_permission hbase> user_permission '@ns1' hbase> user_permission '@.*' hbase> user_permission '@^[a-c].*' hbase> user_permission 'table1' hbase> user_permission 'namespace1:table1' hbase> user_permission '.*' hbase> user_permission '^[A-C].*' |
Procedure Syntax and Usage
| Name | Procedure Hbase Shell Commands Usage |
|---|---|
abort_procedure | Given a procedure Id (and optional boolean may_interrupt_if_running parameter, default is true), abort a procedure in hbase. Use with caution. Some procedures might not be abortable. For experts only. If this command is accepted and the procedure is in the process of aborting, it will return true; if the procedure could not be aborted (eg. procedure does not exist, or procedure already completed or abort will cause corruption), this command will return false. Examples: hbase> abort_procedure proc_id hbase> abort_procedure proc_id, true hbase> abort_procedure proc_id, false |
list_procedures | List all procedures in hbase. For example: hbase> list_procedures |
Visibility Label Syntax and Usage
| Commands | Usage & Examples |
|---|---|
add_labels | Add a set of visibility labels. Syntax : add_labels [label1, label2] For example: hbase> add_labels ['SECRET','PRIVATE'] |
clear_auths | Clear visibility labels from a user or group Syntax : clear_auths 'user',[label1, label2] For example: hbase> clear_auths 'user1', ['SECRET','PRIVATE'] hbase> clear_auths '@group1', ['SECRET','PRIVATE'] |
get_auths | Get the visibility labels set for a particular user or group Syntax : get_auths 'user' For example: hbase> get_auths 'user1' hbase> get_auths '@group1' |
list_labels | List the visibility labels defined in the system. Optional regular expression parameter could be used to filter the labels being returned. Syntax : list_labels For example: hbase> list_labels 'secret.*' hbase> list_labels |
set_auths | Add a set of visibility labels for a user or group Syntax : set_auths 'user',[label1, label2] For example: hbase> set_auths 'user1', ['SECRET','PRIVATE'] hbase> set_auths '@group1', ['SECRET','PRIVATE'] |
set_visibility | Set the visibility expression on one or more existing cells. Pass table name, visibility expression, and a dictionary containing scanner specifications. Scanner specifications may include one or more of: TIMERANGE, FILTER, STARTROW, STOPROW, ROWPREFIXFILTER, TIMESTAMP, or COLUMNS If no columns are specified, all columns will be included. To include all members of a column family, leave the qualifier empty as in 'col_family:'. The filter can be specified in two ways: 1. Using a filterString - more information on this is available in the Filter Language document attached to the HBASE-4176 JIRA 2. Using the entire package name of the filter. Examples: hbase> set_visibility 't1', 'A|B', {COLUMNS => ['c1', 'c2']} hbase> set_visibility 't1', '(A&B)|C', {COLUMNS => 'c1', TIMERANGE => [1303668804, 1303668904]} hbase> set_visibility 't1', 'A&B&C', {ROWPREFIXFILTER => 'row2', FILTER => "(QualifierFilter (>=, 'binary:xyz')) AND (TimestampsFilter ( 123, 456))"} This command will only affect existing cells and is expected to be mainly useful for feature testing and functional verification. |
Rsgroup HBase Shell Commands
Note: The rsgroup Coprocessor Endpoint must be enabled on the Master else commands fail with:
UnknownProtocolException: No registered Master Coprocessor Endpoint found for RSGroupAdminService
| Commands | Usage & Examples |
|---|---|
add_rsgroup | Create a new RegionServer group. Example: hbase> add_rsgroup 'my_group' |
balance_rsgroup | Balance a RegionServer group Example: hbase> balance_rsgroup 'my_group' |
get_rsgroup | Get a RegionServer group's information. Example: hbase> get_rsgroup 'default' |
get_server_rsgroup | Get the group name the given RegionServer is a member of. Example: hbase> get_server_rsgroup 'server1:port1' |
get_table_rsgroup | Get the RegionServer group name the given table is a member of. Example: hbase> get_table_rsgroup 'myTable' |
list_rsgroups | List all RegionServer groups. Optional regular expression parameter can be used to filter the output. Example: hbase> list_rsgroups hbase> list_rsgroups 'abc.*' |
move_servers_rsgroup | Reassign a region server from one RSGroup to another. hbase> move_servers_rsgroup 'dest',['server1:port','server2:port'] |
move_tables_rsgroup | Reassign tables from one RSGroup to another. hbase> move_tables_rsgroup 'dest',['table1','table2'] |
remove_rsgroup | Remove a RegionServer group. hbase> remove_rsgroup 'my_group' |
Conclusion:
We have seen HBase shell commands are broken down into several different groups, each serves a different purpose and also have seen examples, usage, and description of each command to interact with HBase. I hope it helps !!
Happy Learning !!
