How to change Tez job name when running query in HIVE

Question

When I submit a Hive SQL using Tez like below:

hive (default)> select count(*) from simple_data;

In Resource Manager UI the job name shows something like HIVE-9d1906a2-25dd-4a7c-9ea3-bf651036c7eb Is there a way to change the job name tomy_job_nam?

If I am not using Tez and running the job in MR, I can set the job name using set mapred.job.name.

Are there any Tez parameters I need to set, to change the job name?

Any input is appreciated.

Coke

MR will run a new job for each query; but TEZ will try to maintain its "session" open to avoid de-allocating and re-allocating YARN containers all the time. So your question is rather how to set the current (or future) TEZ session name. See fr.hortonworks.com/blog/introducing-tez-sessions >> Unfortunately HIVE is not aware of how TEZ works and has no specific parameter for that... – Tawannatawdry

By the way: there is a long-standing request to support custom tags in YARN and TEZ jobs (especially for OOZIE job management) but these JIRAs don't show much progress, see issues.apache.org/jira/browse/TEZ-909 – Tawannatawdry

Thank you very much Samson for the info. – Coke

Answer 1

You can use "set hiveconf hive.query.name=myjobname"
But you will be able to see the name only in TEZ view. Not in Yarn.
See the link below:
https://community.hortonworks.com/questions/5309/how-to-set-tez-job-name.html

I`m looking into this issue also. If I find the solution I update the question.

Bonded

Answer 2

Got this figured out. Using the property hive.session.id the name could be changed. Below is an example.

hive --hiveconf hive.session.id=test_$(date '+%Y%m%d_%H%M%S') \
      -e "select month, max(sale) from simple_data group by month;"

Coke

Answer 3

Good question. There is a JIRA for Hive on Spark for a very similar thing that you're asking: HIVE-12811 - you could use spark.app.name there; landing in Hive 2.1.

Can't find anything specific for Hive on Tez.. perhaps somebody needs to submit a Hive jira/patch similar to Hive-12811 but for Tez.

Wallasey

Answer 4

set hive.query.name="test_query";

Will work in hive with TEZ

Gamp

Answer 5

set mapred.job.name = more helpful name

Lundquist

The original question already notes that as a possibility if not using Tez, but wants a solution using Tez. – Washwoman