Select previous row which satisfies a condition in hive

I have product data like this

Product   Date            Sales   Availbility
    xyz      2017-12-31      724.5   6.0
    xyz      2018-01-07      362.25  7.0
    xyz      2018-01-14      281.75  7.0
    xyz      2018-01-21      442.75  7.0
    xyz      2018-01-28      442.75  6.0
    xyz      2018-02-04      402.5   7.0
    xyz      2018-02-11      201.25  3.0
    xyz      2018-02-18      120.75  0.0
    xyz      2018-02-25      40.25   0.0
    xyz      2018-03-11      201.25  0.0
    xyz      2018-03-18      483.0   5.0
    xyz      2018-03-25      322.0   7.0
    xyz      2018-04-01      241.5   7.0
    xyz      2018-04-08      281.75  7.0
    xyz      2018-04-15      523.25  7.0
    xyz      2018-04-22      241.5   7.0
    xyz      2018-04-29      362.25  7.0

The data is not ordered ( a minor issue) , what I want to do is that wherever we have 0 in the availbility column (4th column) I want to take previous 3 weeks ( which have full availability i.e 7 ) average

something like below:

xyz      2017-12-31      724.5   6.0     Null 
xyz      2018-01-07      362.25  7.0     362.25 ( Same value for weeks with availbility = 7) 
xyz      2018-01-14      281.75  7.0     281.75
xyz      2018-01-21      442.75  7.0     442.75 
xyz      2018-01-28      442.75  6.0     361 (362 + 281 + 362/3)the prior fully availble week avg which is avilble)
xyz      2018-02-04      402.5   7.0     402
xyz      2018-02-11      201.25  3.0     375 (402 + 442 + 281 /3)
xyz      2018-02-18      120.75  0.0     375 ( Same since 375 is the most recent 4 fully availble average)
xyz      2018-02-25      40.25   0.0     375
xyz      2018-03-11      201.25  0.0     375
xyz      2018-03-18      483.0   5.0     375
xyz      2018-03-25      322.0   7.0     322
xyz      2018-04-01      241.5   7.0     241
xyz      2018-04-08      281.75  7.0     281
xyz      2018-04-15      523.25  7.0     523
xyz      2018-04-22      241.5   7.0     241
xyz      2018-04-29      362.25  7.0     362

I approached it by trying to find the 3 weeks average of only the fully available weeks and union it with the rest of the weeks, later tries using lag function to retrieve the most recent average .

select a.*,lag(case when a.Full_availble_sales >0 then a.Full_availble_sales end,1) over (partition by a.asin order by a.week_beginning) as Four_wk_avg  from (select asin,week_beginning,avg(sales) as weekly_sales,sum(available_to_purchase) as weekly_availbility,0 as Full_availble_sales from t1 where asin = 'xyz' group by asin,week_beginning having sum(available_to_purchase) < 7
union all
select t.asin,t.week_beginning,t.weekly_sales,t.weekly_availbility,avg(t.weekly_sales) over (partition by t.asin order by t.week_beginning rows between 3 preceding and current row ) as Full_availble_sales from 
(select asin,week_beginning,avg(sales) as weekly_sales,sum(available_to_purchase) as weekly_availbility from t1 where asin = 'xyz' group by asin,week_beginning having sum(available_to_purchase) = 7)t ) a  order by a.week_beginning

O/P was

xyz      2017-12-31      724.5   6.0     0.0     NULL
xyz      2018-01-07      362.25  7.0     362.25  NULL
xyz      2018-01-14      281.75  7.0     322.0   362.25
xyz      2018-01-21      442.75  7.0     362.25  322.0
xyz      2018-01-28      442.75  6.0     0.0     362.25
xyz      2018-02-04      402.5   7.0     372.3125        NULL
xyz      2018-02-11      201.25  3.0     0.0     372.3125
xyz      2018-02-18      120.75  0.0     0.0     NULL
xyz      2018-02-25      40.25   0.0     0.0     NULL
xyz      2018-03-11      201.25  0.0     0.0     NULL
xyz      2018-03-18      483.0   5.0     0.0     NULL
xyz      2018-03-25      322.0   7.0     362.25  NULL
xyz      2018-04-01      241.5   7.0     352.1875        362.25
xyz      2018-04-08      281.75  7.0     311.9375        352.1875
xyz      2018-04-15      523.25  7.0     342.125 311.9375
xyz      2018-04-22      241.5   7.0     322.0   342.125
xyz      2018-04-29      362.25  7.0     352.1875        322.0

which was not what I intended.

This will do the job (using aggregate functions avg and max_by on a window)

WITH
   tt1  (Product,Date_week_beginning,Sales,Availbility) AS 
      ( SELECT * FROM ( VALUES 
      ('xyz','2017-12-31',  724.5   ,6.0),
      ('xyz','2018-01-07',  362.25  ,7.0),
      ('xyz','2018-01-14',  281.75  ,7.0),
      ('xyz','2018-01-21',  442.75  ,7.0),
      ('xyz','2018-01-28',  442.75  ,6.0),
      ('xyz','2018-02-04',  402.5   ,7.0),
      ('xyz','2018-02-11',  201.25  ,3.0),
      ('xyz','2018-02-18',  120.75  ,0.0),
      ('xyz','2018-02-25',  40.25   ,0.0),
      ('xyz','2018-03-11',  201.25  ,0.0),
      ('xyz','2018-03-18',  483.0   ,5.0),
      ('xyz','2018-03-25',  322.0   ,7.0),
      ('xyz','2018-04-01',  241.5   ,7.0),
      ('xyz','2018-04-08',  281.75  ,7.0),
      ('xyz','2018-04-15',  523.25  ,7.0),
      ('xyz','2018-04-22',  241.5   ,7.0),
      ('xyz','2018-04-29',  362.25  ,7.0) )  
   ), tt2 AS (
      SELECT *, avg(sales) OVER (partition by Product order by if(Availbility = 7.0,1),Date_week_beginning ROWS BETWEEN 2 PRECEDING AND CURRENT ROW) avg3
      FROM tt1
   )
   SELECT Product,Date_week_beginning,Sales,Availbility, 
          CASE WHEN Availbility = 7.0 THEN Sales
             ELSE
                 max_by(if(Availbility = 7.0,avg3),(Availbility = 7.0, Date_week_beginning) ) OVER (partition by Product order by Date_week_beginning)
             END new_col   
   FROM tt2
   ORDER BY Product,Date_week_beginning

And the resuls are exactly like requested:

Product Date_week_beginning Sales    Availbility  new_col
xyz     2017-12-31          724.5    6.0          NULL
xyz     2018-01-07          362.25   7.0          362.25
xyz     2018-01-14          281.75   7.0          281.75
xyz     2018-01-21          442.75   7.0          442.75
xyz     2018-01-28          442.75   6.0          362.25
xyz     2018-02-04          402.5    7.0          402.5
xyz     2018-02-11          201.25   3.0          375.6666666666667
xyz     2018-02-18          120.75   0.0          375.6666666666667
xyz     2018-02-25          40.25    0.0          375.6666666666667
xyz     2018-03-11          201.25   0.0          375.6666666666667
xyz     2018-03-18          483.0    5.0          375.6666666666667
xyz     2018-03-25          322.0    7.0          322.0
xyz     2018-04-01          241.5    7.0          241.5
xyz     2018-04-08          281.75   7.0          281.75
xyz     2018-04-15          523.25   7.0          523.25
xyz     2018-04-22          241.5    7.0          241.5
xyz     2018-04-29          362.25   7.0          362.25

I assume two typos in the question (as can be verified from the example):

This line in the question is mistake: 361 (362 + 281 + 362/3)the prior fully availble week avg which is avilble) should be 361 (442 + 281 + 362/3)the prior fully availble week avg which is avilble)

The sentence: "what I want to do is that wherever we have 0 in the availbility column (4th column).." should be "what I want to do is that wherever we do not have 7.0 in the availbility column (4th column).."

Recommended topics

Hot tags