Posts

Showing posts with the label hive

Subtract n months from a date field in hive

Subtract n months from a date field in hive I am trying to limit a table by a specific number of months, 24 for example in my WHERE clause. I have used the following with no success: Where month(EVENT_START_DT_TM) > add_months(from_unixtime(unix_timestamp()),-1) EVENT_START_DT_TM is a true datetime field and I want to have a 24 month look back period from the current date matching up against that field. Looking at the HIVE manual, a lot of date functions require date STRINGS, but what a true DATE field? 1 Answer 1 use Current_date() and add_months() function to get date before 24 months then use date_format() function to change the format as per your needs(matches to EVENT_START_DT_TM field format) hive> select date_format(add_months(current_date(),-24),"yyyy-MM-dd HH:mm:ss.SSS"); +--------------------------+--+ | _c0 | +--------------------------+--+ | 2016...

Why is Spark saveAsTable with bucketBy creating thousands of files?

This summary is not available. Please click here to view the post.