Is there a faster alternative to Python's strftime?

Question

I'm processing hundreds of thousands of dates in Python and noticing that the strftime function is pretty slow.

I used timeit to check and it tells me that it takes roughly 0.004 which is fine for a small number but becomes problematic for processing a couple thousand for example.

print(min(timeit.Timer("now.strftime('%m/%d/%Y')", setup=setup).repeat(7,1000))

Is there any faster alternative?

Raama

I don't think so. You could go for caching if the entries repeat themselves, or splittingg it into threads (which is very error-prone however). – Surat

https://mcmap.net/q/54601/-a-faster-strptime this may help. – Banuelos

@AjaySingh: That question is about strptime, the string-to-timestamp function. This question is about strftime, the timestamp-to-string function. – Cockrell

@Surat Splitting into threads will not help CPU-bound tasks in most implementations of Python since there is typically a global lock that prevents threads from running concurrently. – Counterinsurgency

Answer 1

Since you have a rigid format you can just access directly the fields of the datetime object and use Python string formatting to construct the required string:

'{:02d}/{:02d}/{}'.format(now.month, now.day, now.year)

In Python 3 this is about 4 times faster than strftime(). It's also faster in Python 2, about 2-3 times as fast.

Faster again in Python 3 is the "old" style string interpolation:

'%02d/%02d/%d' % (now.month, now.day, now.year)

about 5 times faster, but I've found this one to be slower for Python 2.

Another option, but only 1.5 times faster, is to use time.strftime() instead of datetime.strftime():

time.strftime('%m/%d/%Y', now.timetuple())

---

Finally, how are you constructing the datetime object to begin with? If you are converting strings to datetime (with strptime() for example), it might be faster to convert the incoming string version to the outgoing one using string slicing.

Gynaecology

Thanks a lot, trying these out now. Essentially I have a datetime object and would like to output something like 'Thursday 10th Dec at 10:44AM'. The datetime comes directly from the db. – Raama

Mapping the day of week to the day name is more difficult and will slow down these alternative methods. Ditto for month name. Assuming that you do not actually need a datetime object, perhaps you can select the date from the database as a string in the required format? (or you could do both: get a datetime and a string representation of it) – Gynaecology

Right so I realised I couldn't do that as there's some additional timezone conversion that happens after the db call. Instead of meddling with that I opted with your suggestion of using the time.strftime instead which is pretty impressive and is a sufficient performance improvement and frankly works well :). Appreciate your help! – Raama

Don't suppose you know any other techniques? Just curious if there's anything else that could result in some quick wins. – Raama

The best way, probably, is to do the work in the database if you can. That depends on the database in use - which one? What is the timezone conversion? Is it converting UTC to local time? Other than that I don't know of any faster way to produce the output string from a datetime object. – Gynaecology

Answer 2

This is marginally faster on my box (python 2.7.x)

def fmt(date):
    return "%02d/%02d/%d" % (date.month, date.day, date.year)

Kizzee