I use Spring-Boot and want to connect it to Hive by spring-data-hadoop 2.3.0.M1-hdp23. The problem is that my application doesn't start if I add hive-jdbc to my pom.xml. The errorlog (partly) is: ...

I need to replace the square brackets around a key:value pair similar to the following. Any help is much appreciated! data in 'properties' looks like this: name: property1 value: [12345667:97764...

On an AWS EMR cluster, I'm trying to write a query result to parquet using Pyspark but face the following error: Caused by: java.lang.RuntimeException: Parquet record is malformed: empty fields ar...

I am trying to create partition for my Table inorder to update a value. This is my sample data 1,Anne,Admin,50000,A 2,Gokul,Admin,50000,B 3,Janet,Sales,60000,A I want to update Janet's Departme...

Using Hive 2.3.7 on AWS EMR (5.33.1) I have created a database which shows correctly when calling show databases;. I then create a table which seems to work correctly (no exceptions). When I call d...

How to execute hive commands directly from .hql file?

I have a scenario to compare two different tables source and destination from two separate remote hive servers, can we able to use two SparkSessions something like I tried below:- val spark = Spa...

I tried to install hive on a raspberry pi 2. I installed Hive by uncompress zipped Hive package and configure $HADOOP_HOME and $HIVE_HOME manually under hduser user-group I created. When running hi...

I have orc table in hive I want to drop column from this table ALTER TABLE table_name drop col_name; but I am getting the following exception Error occurred executing hive query: OK FAILED: ...

I want to create a hive table using my Spark dataframe's schema. How can I do that? For fixed columns, I can use: val CreateTable_query = "Create Table my table(a string, b string, c double)" spa...

I want to convert a .sas7bdat file to a .csv/txt format so that I can upload it into a hive table. I'm receiving the .sas7bdat file from an outside server and do not have SAS on my machine.

I need to deploy Big Data Cluster on our servers. But I just know about knowledge of Apache Spark. Now I need to know whether Spark SQL can completely replace Apache Impala or Apache Hive. I...

I recently ran into the following error "AthenaQueryError: Athena query failed: "NOT_SUPPORTED: Unsupported Hive type", and for this I followed this stack overflow link: converting t...

I created a 3 node Hadoop cluster with 1 namenode and 2 datanode. I can perform a read/write query from Hive shell, but not beeline. I found many suggestions and answers related to this issue. In...

I want to select all columns in a table except StudentAddress and hence I wrote following query: select `(StudentAddress)?+.+` from student; It gives following error in Squirrel Sql client. org....

Is there any way to identify which database the user is using while using the command line interface ?

Since we can get the row number assigned using the row_number() and if we want to find the rank of each row without skipping of any number within the partition using dense_rank(), why do we need ra...

I'm trying to save dataframe in table hive. In spark 1.6 it's work but after migration to 2.2.0 it doesn't work anymore. Here's the code: blocs .toDF() .repartition($"col1", $"col2", $"col3", ...

In hive, I can set the prompt to be the currently selected database. Is it possible to do the same in MySQL? I want to replace mysql with kundoor in the image below.

I tried dropping a HIVE table. Screen hangs. I tried in both HIVE CLI and beeline. Neither the table getting dropped nor the data getting inserted into the table. Need help!

I am interested in being able to retrieve the location value of a Hive table given a Spark object (SparkSession). One way to obtain this value is by parsing the output of the location via the follo...

I am trying to create a table which has a complex data type. And the data types are listed below. array map array< map < String,String> > I am trying to create a data structure of 3 type ...

I'm trying to use Apache Iceberg for writing data to a specified location(S3/local). Following is the configuration used below. SBT: libraryDependencies += "org.apache.spark" %% "spa...

In the spark-shell (scala), we import, org.apache.spark.sql.hive.thriftserver._ for starting Hive Thrift server programatically for a particular hive context as HiveThriftServer2.startWithContext(h...

How to enable the %sql Magic string on jupyter notebook and how to use %sql magic string on a cell with the below line of code. spark.sql('select * from test').show()