Converting cftime.DatetimeJulian to datetime

Question

I'm trying to convert an xarray data array to a pandas dataframe for a machine learning project, but the time data appears to be in a cftime.DatetimeJulian format, which is not convertible using the pandas to_datetime() method. Suggestions? Thanks.

nor_xr.time

<xarray.DataArray 'time' (time: 1372)>
array([cftime.DatetimeJulian(2015, 3, 31, 0, 0, 0, 0, 0, 90),
       cftime.DatetimeJulian(2018, 12, 31, 0, 0, 0, 0, 6, 365)], dtype=object)
Coordinates:
  * time     (time) object 2015-03-31 00:00:00 ... 2018-12-31 00:00:00
Attributes:
    standard_name:  time
    axis:           T

nor_df = nor_xr.to_dataframe().reset_index()
nor_df.head()

    time
0   2015-03-31 00:00:00
1   2015-04-01 00:00:00

pd.to_datetime(nor_df.time)

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-104-1f0fc00ad825> in <module>
      2 
      3 #|nor_df.time.unique()
----> 4 pd.to_datetime(nor_df.time)

~\AppData\Local\Continuum\anaconda3A\lib\site-packages\pandas\core\tools\datetimes.py in to_datetime(arg, errors, dayfirst, yearfirst, utc, box, format, exact, unit, infer_datetime_format, origin, cache)
    449         else:
    450             from pandas import Series
--> 451             values = _convert_listlike(arg._values, True, format)
    452             result = Series(values, index=arg.index, name=arg.name)
    453     elif isinstance(arg, (ABCDataFrame, MutableMapping)):

~\AppData\Local\Continuum\anaconda3A\lib\site-packages\pandas\core\tools\datetimes.py in _convert_listlike(arg, box, format, name, tz)
    366                     dayfirst=dayfirst,
    367                     yearfirst=yearfirst,
--> 368                     require_iso8601=require_iso8601
    369                 )
    370 

pandas\_libs\tslib.pyx in pandas._libs.tslib.array_to_datetime()

pandas\_libs\tslib.pyx in pandas._libs.tslib.array_to_datetime()

pandas\_libs\tslib.pyx in pandas._libs.tslib.array_to_datetime()

TypeError: <class 'cftime._cftime.DatetimeJulian'> is not convertible to datetime

Burkitt

Answer 1

Indeed this is a common request. The recommended way of doing it is to pull out the index (a CFTimeIndex) and use its built-in to_datetimeindex method:

datetimeindex = nor_xr.indexes['time'].to_datetimeindex()

You can then replace the CFTimeIndex with its DatetimeIndex counterpart:

nor_xr['time'] = datetimeindex

Fluttery

Confirming that this solution works for a similar issue with cftime.DatetimeNoLeap as well. – Heroin

Answer 2

I found a simple way to convert from cftime.DatetimeNoLeap to datetime object. Say for eg datetimes is a list of cftime.DatetimeNoLeap objects with datetimes[0] = cftime.DatetimeNoLeap(1970, 1, 1, 12, 0, 0, 0, has_year_zero=True)

datetime.strptime(str(datetimes[0]),'%Y-%m-%d %H:%M:%S')

This will give you datetime.datetime(1970, 1, 1, 12, 0).

Metaprotein

Answer 3

A simple and clean solution using iso format

from datetime import datetime

datetime(cfdatetime.isoformat())

Cichlid