I'm trying to write a pig latin script to pull the count of a dataset that I've filtered. Here's the script so far: /* scans by title */ scans = LOAD '/hive/scans/*' USING PigStorage(',') AS (t...

I have a file in hdfs with 100 columns, which i want to proces using pig. I want to load this file into a tuple with columns names in a separate pig script, and reuse this script from other pig scr...

How can I check in piglatin, if a bag contains an element? Example : In a bag of chararray, how can I check if a token is present?

I'm about to start playing around with PIG-latin, and I was hoping to get some text highlighting and such for it in Eclipse. Doing a quick Google search, I saw a couple of Eclipse plugins for it. A...

I wrote a program to execute a embeded Pig sentence in Java. I executed the java sentence registryQuery. But when I try on to store the result, I give a error of org.apache.hadoop.mapred.localClien...

I'm using CurrentTime(), which is a datetime data type. However, I need it as a chararray. I have the following: A = LOAD ... B = FOREACH A GENERATE CurrentTime() AS todaysDate; I've tried vario...

I am using Pig (0.9.1) with UDFs written in Python. The Python scripts import modules from the standard Python library. I have been able to run the Pig scrips that call the Python UDFs successfully...

Let's say I JOIN two relations like: -- part looks like: -- 1,5.3 -- 2,4.9 -- 3,4.9 -- original looks like: -- 1,Anju,3.6,IT,A,1.6,0.3 -- 2,Remya,3.3,EEE,B,1.6,0.3 -- 3,akhila,3.3,IT,C,1.3,0.3 j...

I like to generate multiple tuples from a single tuple. What I mean is: I have file with following data in it. >> cat data ID | ColumnName1:Value1 | ColumnName2:Value2 so I load it by the...

I am relatively new to PigScript. I would like to know if there is a way of passing parameters to Java UDFs in Pig? Here is the scenario: I have a log file which have different columns (each repre...

I understood Group didn't work with multiple tuples and hence we had COGROUP in PIG. However, while checking today the GROUP command works for me. I am using PIG-0.12.0. My commands and outputs are...

Can Some one explain me on getting this below output in Pigscript my input file is below a.txt aaa.kyl,data,data bbb.kkk,data,data cccccc.hj,data,data qa.dff,data,data I am writing the pig sc...

I can control the number of reducers by using PARALLEL clause in the statements which result in reducers. I want to control the number of mappers. The data source is already created, and I can not...

I'm having a problem using the python library simplejson in jython to write a Pig UDF. I need because jython-standalone-2.5.2.jar doesn't come with a JSON library. I'm using Apache Pig version 0.11...

Let's say I have a data set of restaurant reviews: User,City,Restaurant,Rating Jim,New York,Mecurials,3 Jim,New York,Whapme,4.5 Jim,London,Pint Size,2 Lisa,London,Pint Size,4 Lisa,London,Rabbit Wh...

From this: (1, {(1,2), (1,3), (1,4)} ) (2, {(2,5), (2,6), (2,7)} ) ...How could we generate this? ((1,2),(1,3),(1,4)) ((2,5),(2,6),(2,7)) ...And how could we generate this? (1, 2, 3, 4) (2, ...

I am currently debugging a pig script. I'd like to define a tuple in the Pig file directly (instead of the basic "Load" function). Is there a way to do it? I am looking for something like that: ...

I have data that's already grouped and aggregated, it looks like so: user value count ---- -------- ------ Alice third 5 Alice first 11 Alice second 10 Alice fourth 2 ... Bob second 20 Bob third 1...

Am using Pig 0.11.0 rank function and generating ranks for every id in my data. I need ranking of my data in a particular way. I want the rank to reset and start from 1 for every new ID. Is it pos...

I have a data file and a corresponding schema file stored in separate locations. I would like to load the data using the schema in the schema-file. I tried using A= LOAD '<file path>' USING ...

I'm loading a tsv file with a datetime column and long column with: A = LOAD 'tweets-clean.txt' USING PigStorage('\t') AS (date:datetime, userid:long); DUMP A; An example line of input: Tue Feb...

I am trying to edit a large file on Hadoop cluster and trim white spaces and special characters like ¦,*,@," etc from the file. I dont want to copyToLocal and use a sed as i have 1000's of such fil...

I installed Hadoop (1.0.2) for a single node on Windows 7 with Cygwin, and it is working. However, I cannot get PIG (0.10.0) to see the Hadoop. 1) "Error: JAVA_HOME is not set." I added this lin...

First of all I am relatively new to Big Data and the Hadoop world and I have just started to experiment a little with the Hortonworks Sandbox (Pig and Hive so far). I was wondering in which c...

Versions: Hadoop 2.2, Hbase 0.96.1, Pig 0.12 Whenever I run this pig script raw_data = LOAD 'sample_data.csv' USING PigStorage( ',' ) AS ( listing_id: chararray, fname: chararray, lname: chara...