java.lang.RuntimeException:Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient

R

18

35

I have configured my Hive as given on link: http://www.youtube.com/watch?v=Dqo1ahdBK_A, but I am getting the following error while creating a table in Hive. I am using hadoop-1.2.1 and hive-0.12.0.

hive> create table employee(emp_id int,name string,salary double);
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient

Refuse answered 28/3, 2014 at 11:17 Comment(0)

P

46

Looks like problem with your metastore. If you are using the default hive metastore embedded derby. Lock file would be there in case of abnormal exit. if you remove that lock file this issue would be solved

rm   metastore_db/*.lck

Paring answered 28/3, 2014 at 14:8 Comment(5)

I have to run the rm command every time I use pyspark, any ideas why? – Churchwell 9/9, 2016 at 18:32

Using default derby database as hive back end is not recommended, With derby as Hive backend only one session can established at a time. The best option is fix this issue is to use MySQL/Postgres/Oracle server as Hive backend instead of derby. – Paring 12/9, 2016 at 14:46

Thanks Sachin. It worked for me. But it is not good to execute this command when you gave lot of data stored in Hive. What should be an alternative in that case – Frawley 3/5, 2017 at 6:51

@KanavNarula, It's not recommended to use derby for a clustered environment. Recommended options is to migrate from derby to Postgres or MySQL. – Paring 3/5, 2017 at 18:31

Great. Thanks @sachin for the quick response. – Frawley 5/5, 2017 at 4:14

I

10

This could be an issue with the metastore like sachinjose described or a connection issue. Run hive console in debug mode like so:

hive -hiveconf hive.root.logger=DEBUG,console

Then execute a simple query like show tables; and see what happens. I ran into this issue after restarting a namenode and it was due to the wrong host being set in the configuration files (ec2 generates new private IP/hostname when restarted).

Interment answered 23/4, 2014 at 10:4 Comment(0)

T

8

Run this command from the console:

./hive -hiveconf hive.root.logger=DEBUG,console

Now run

show databases;

If you see an exception as below:

java.sql.SQLException: A read-only user or a user in a read-only database is not permitted to disable read-only mode on a connection.

It means there is some permissions issue with the metastore_db. You need to change the permissions of the complete folder. I am running my hive as hdfs user. So, i used the command

chown -R hdfs:hadoop *

in the hive directory. This resolved the issue.

Tropopause answered 27/8, 2014 at 3:55 Comment(0)

M

2

hive> show databases;
FAILED: Error in metadata: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask

To resolve this issue start the hadoop services first.

$ start-all.sh

Then I run

hive> show database;

It works fine for me.

Manilla answered 19/7, 2015 at 13:28 Comment(1)

Where can I find the script start-all.sh? – Moonfaced 26/6, 2019 at 3:5

H

1

The answer is located in http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/5.0/CDH5-Installation-Guide/cdh5ig_hive_schema_tool.html

To suppress the schema check and allow the metastore to implicitly modify the schema, you need to set the hive.metastore.schema.verification configuration property to false in hive-site.xml.

Hilten answered 22/7, 2014 at 13:35 Comment(0)

F

1

rm metastore_db/*.lck

It works for me, too. It can be found in your home/user directory. You can use the locate command to find it: locate metastore_db

After removing the lock files, close the current session. Call hive shell in new session

Freitag answered 1/7, 2015 at 3:7 Comment(0)

S

1

make hive.metastore.schema.verification=false in hive-site.xml file this error will be removed. In my case it is working well.

Suffrage answered 13/1, 2016 at 3:58 Comment(0)

K

1

I was getting the same error but I could not even start the hive shell. After trying the very useful hive debug command:

hive -hiveconf hive.root.logger=DEBUG,console

I was able to see that Hive could not find a valid Kerberos TGT. You will see in the debug info something about SASL negotiation failed and no valid Kerberos TGT. My solution was to run

kinit

before running the hive CLI.

Kaye answered 28/12, 2018 at 18:1 Comment(0)

P

0

Restarting the virtual machine or system should also release the lock.

Pocky answered 1/4, 2014 at 7:58 Comment(0)

S

0

this could due to more than one "metastore_db". Remove "metastore_db", restart hadoop cluster and open hive shell from $HIVE_HOME/bin folder

Spandex answered 4/5, 2014 at 7:34 Comment(0)

A

0

Being a newbie i got the same set of errors.
It was found that one of the daemons in my case namenode was not initiated. On installing hadoop it would be wise to make it a habit to hit he following commands :
ps -ef | grep "namenode"
ps -ef | grep "datanode"
ps -ef | grep "tracker"

One needs to check the relevant log if any of the daemon is not working.

Avina answered 19/5, 2015 at 13:51 Comment(0)

C

0

If it is in local machine, looks like you have another terminal opened with hive shell/session. You can have only one session using embedded derby data base. Close all other hive sessions and try.

Calculating answered 16/7, 2015 at 11:49 Comment(0)

R

0

delete code in hive-site.xml and paste this code in hive-site.xml.

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?><!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<configuration>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:derby:;databaseName=/home/hadoop/Desktop/apache-hive-2.1.1-bin/metastore_db;create=true</value>
<description>
JDBC connect string for a JDBC metastore.
To use SSL to encrypt/authenticate the connection, provide database-specific SSL flag in the connection URL.
For example, jdbc:postgresql://myhost/db?ssl=true for postgres database.
</description>
</property>
<property>
<name>hive.metastore.warehouse.dir</name>
<value>/user/hive/warehouse</value>
<description>location of default database for the warehouse</description>
</property>
<property>
<name>hive.metastore.uris</name>
<value/>
<description>Thrift URI for the remote metastore. Used by metastore client to connect to remote metastore.</description>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>org.apache.derby.jdbc.EmbeddedDriver</value>
<description>Driver class name for a JDBC metastore</description>
</property>
<property>
<name>javax.jdo.PersistenceManagerFactoryClass</name>
<value>org.datanucleus.api.jdo.JDOPersistenceManagerFactory</value>
<description>class implementing the jdo persistence</description>
</property>
</configuration>

Reichstag answered 3/2, 2017 at 4:10 Comment(0)

A

0

I was facing the same issue there are some important point's that could be resolve this problem.

1.Put the following at the beginning of hive-site.xml

  <property>
    <name>system:java.io.tmpdir</name>
    <value>/tmp/hive/java</value>
  </property>
  <property>
    <name>system:user.name</name>
    <value>${user.name}</value>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:derby:,databaseName=$HIVE_HOME/metastore_db;create=true</value>
    <description>JDBC connect string for a JDBC metastore </description>
  </property>

this is set relative path in absolute URI and configuring Metastore where the database is stored

2.Remove $HIVE_HOME/metastore_db But Be aware, this will remove your schema completely!

3.Now,You must Initialize Derby database.By default, Hive uses Derby database

$HIVE_HOME/bin/schematool -initSchema -dbType derby

also I suppose that your environment variable has been set correctly ,if not please check them that are looking like below:

export HADOOP_HOME=/usr/local/hadoop 
export PATH=$PATH:$HADOOP_HOME/bin
export PATH=$PATH:$HADOOP_HOME/sbin
export HIVE_HOME=/usr/lib/hive
export PATH=$PATH:$HIVE_HOME/bin

and then run hive command and type show databases;

Assassin answered 24/7, 2018 at 12:46 Comment(0)

N

0

I'm surprised to see no one has pointed out the actual issue which is most often the case.

System.setSecurityManager(null)

Put this line at the start of your application.

Neville answered 26/11, 2018 at 21:13 Comment(2)

I'm using windows and pyspark. I tried your solution but System is not defined ?! @Abhinandan Dubey – Ornithology 17/12, 2019 at 14:6

Try importing java.lang.System – Neville 17/12, 2019 at 16:48

S

0

When I used jdk 11, I deploymented hive on my master node and then those exceptions were thrown. Many methods I have tried but useless. Eventually I changed the version of jdk from 11 to 8, which used in master node. Then I started the hive successfully.

Sty answered 24/3, 2019 at 4:55 Comment(0)

C

0

remove metastore_db/*.lck
restart your Virtual Machine (CentOS, or some other)
start the mysql sevice: service mysqld start
test: hive -e 'show tables;'

Note that the mysql-connector-xxx-bin.jar has been stored in hive/lib/ and the xml file hive-site.xml has been configurated correctly. Check the username and password about MySQL carefully.

Complected answered 18/5, 2023 at 15:44 Comment(0)

H

-1

I was facing the same issue--used the below steps to resolve it:

Create an file hive-site.xml and input the details (for Local/Prod mode). Make sure the below location exist /home/hadoop/bhishm/warehouse

Example:

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
    <name>hive.metastore.warehouse.dir</name>
    <value>/home/hadoop/bhishm/warehouse</value>
    <description>
    Local or HDFS directory where Hive keeps table contents.
    </description>
</property>
<property>
    <name>hive.metastore.local</name>
    <value>true</value>
    <description>
    Use false if a production metastore server is used.
    </description>
</property>
<property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:derby:;databaseName=/home/hadoop/bhishm/warehouse/metastore_db;create=true</value>
    <description>
    The JDBC connection URL.
    </description>
</property>
</configuration>

Edit the hive-env.sh--> add the java path as the first line after reducing memory usage:

Example:

# Hive Client memory usage can be an issue if a large number of clients
# are running at the same time. The flags below have been useful in 
# reducing memory usage:

# The java implementation to use.  Required.

export JAVA_HOME=/usr/lib/jvm/java-1.6.0-openjdk

Run the hive query.

Hyperthermia answered 14/2, 2016 at 17:50 Comment(0)

Recommended topics

Hot tags