udf Questions
4
I have some sql that I want to pass into a mysql stored procedure. I'm using the json functions in mysql-json-udfs-0.4.0-labs-json-udfs-linux-glibc2.5-x86_64. We are running a mysql 5.5.4 server. U...
3
Solved
I'm wondering if it would be possible to make a api call to the google maps geocoding api within a UDF in BigQuery?
I have Google analytics geo fields such as
{
"geoNetwork_continent": "Europe",...
Au asked 6/1, 2016 at 12:34
3
I have a query in Spark SQL like
select count(ts), truncToHour(ts)
from myTable
group by truncToHour(ts).
Where ts is of timestamp type, truncToHour is a UDF that truncates the timestamp to hou...
Junket asked 9/7, 2015 at 22:23
8
Solved
I'm wondering if anyone knows of a way to measure string similarity in BigQuery.
Seems like would be a neat function to have.
My case is i need to compare the similarity of two urls as want to ...
Publius asked 30/10, 2015 at 10:34
3
I have a requirement to select * from all columns from a hive struct.
Hive create table script is here below
Create Table script
Select * from the table displays each struct as a column
select ...
Janenejanenna asked 16/3, 2017 at 22:54
2
I have a DataFrame that has multiple columns of which some of them are structs. Something like this
root
|-- foo: struct (nullable = true)
| |-- bar: string (nullable = true)
| |-- baz: string ...
Man asked 29/6, 2017 at 17:44
2
I'm using a custom Add-in which adds functions to Excel, like UDF1(param1, param2).
(1) example call in cell:
=UDF1(param1, param2)
When saving workbook1, which implements UDF(), Excel silently a...
2
Solved
I am trying to install the following package on my mac in order to test my API on my local environment but thus far I have not succeeded.
https://github.com/spachev/mysql_udf_bundle
I have tried ...
3
The issues:
1) Spark doesn't call UDF if input is column of primitive type that contains null:
inputDF.show()
+-----+
| x |
+-----+
| null|
| 1.0|
+-----+
inputDF
.withColumn("y",
udf { (x: D...
Larner asked 14/3, 2017 at 16:49
4
Solved
Imagine the following code:
def myUdf(arg: Int) = udf((vector: MyData) => {
// complex logic that returns a Double
})
How can I define the return type for myUdf so that people looking at the...
Gastroenterostomy asked 31/5, 2017 at 18:8
2
I have previously registered a UDF with hive. It is permanent not TEMPORARY. It works in beeline.
CREATE FUNCTION normaliseURL AS 'com.example.hive.udfs.NormaliseURL' USING JAR 'hdfs://udfs/hive-u...
Jamnis asked 18/8, 2016 at 16:52
2
Solved
I would like to build one UDF from two already working functions. I'm trying to calculate a md5 hash as a new column to an existing Spark Dataframe.
def md5(s: String): String = { toHex(MessageDig...
Poppo asked 29/6, 2017 at 7:37
1
Solved
I know I can define a User Defined Function in order to perform some custom calculation. I also know I can use the 'out-of-the-box' aggregation functions to reduce a collection of values to a singl...
Recipe asked 15/5, 2017 at 8:15
3
I have a "StructType" column in spark Dataframe that has an array and a string as sub-fields. I'd like to modify the array and return the new column of the same type. Can I process it with UDF? Or ...
Vertigo asked 21/3, 2017 at 15:43
2
Understanding that side-effecting operators (like "insert") are disallowed in user-defined functions, how does one log (or otherwise track) calls to a specific user-defined function? I'd also like ...
Pothunter asked 17/1, 2017 at 16:47
1
Solved
I am trying to pass a list of tuples to a udf in scala. I am not sure how to exactly define the datatype for this. I tried to pass it as a whole row but it can't really resolve it. I need to sort t...
Euphony asked 9/1, 2017 at 15:32
1
When I work with DataFrames in Spark, I have to sometimes edit only the values of a particular column in that DataFrame. For eg. if I have a count field in my dataframe, and If I would like to add ...
Hawkeyed asked 19/8, 2016 at 12:28
2
In my Project, I want to achieve ADD(+) function, but my parameter maybe LongType, DoubleType, IntType. I use sqlContext.udf.register("add",XXX), but I don't know how to write XXX, which is to make...
Sidran asked 28/4, 2016 at 12:44
2
Solved
We had a simple UDF in BigQuery that somehow throws an error that keeps returning
Query Failed
Error: An internal error occurred and the request could not be completed.
The query was simply try...
Endocardial asked 29/3, 2016 at 3:16
1
Solved
I managed to write a few scalar functions with Python in AmazonRedshift, i.e. taking one or a few columns as input and returning a single value based on some logic or transformation.
But is there ...
Rainer asked 25/10, 2015 at 13:0
2
Solved
I have a macro that goes through column(s) and removed numbers from all cells in the range. I would like to add an optional parameter, so I can call the sub while telling it which columns to run on...
3
I would like to know how to call UDF function from function of domain-specific language(DSL) in Spark SQL using JAVA.
I have UDF function (just for example):
UDF2 equals = new UDF2<String, Str...
Overdue asked 20/11, 2015 at 14:45
1
I have a serialized blob and a function that converts it into a java Map.
I have registered the function as a UDF and tried to use it in Spark SQL as follows:
sqlCtx.udf.register("blobToMap", Util...
Kata asked 4/1, 2016 at 9:1
1
I have a custom distance metric that I need to use for KNN, K Nearest Neighbors.
I tried following this, but I cannot get it to work for some reason.
I would assume that the distance metric is ...
Selenodont asked 22/12, 2015 at 3:31
2
Solved
I have successfully created a Python UDF that accepts a varchar value from a table and extracts a substring of that value based on a regex.
The max size of that varchar column in the DDL is set to...
Millet asked 9/11, 2015 at 10:45
1 Next >
© 2022 - 2024 — McMap. All rights reserved.