How can I parse a time string containing milliseconds in it with python?

Question

Hi, I am able to parse strings containing date/time with time.strptime

>>> import time
>>> time.strptime('30/03/09 16:31:32', '%d/%m/%y %H:%M:%S')
(2009, 3, 30, 16, 31, 32, 0, 89, -1)

How can I parse a time string that contains milliseconds?

>>> time.strptime('30/03/09 16:31:32:123', '%d/%m/%y %H:%M:%S')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.5/_strptime.py", line 333, in strptime
    data_string[found.end():])
ValueError: unconverted data remains: :123

Answer 1

My first thought was to try passing it '30/03/09 16:31:32.123' (with a period instead of a colon between the seconds and the milliseconds.) But that didn't work. A quick glance at the docs indicates that fractional seconds are ignored in any case...

Ah, version differences. This was reported as a bug and now in 2.6+ you can use "%S.%f" to parse it.

Answer 2

from python mailing lists: parsing millisecond thread. There is a function posted there that seems to get the job done, although as mentioned in the author's comments it is kind of a hack. It uses regular expressions to handle the exception that gets raised, and then does some calculations.

You could also try do the regular expressions and calculations up front, before passing it to strptime.

Answer 3

To give the code that nstehr's answer refers to (from its source):

def timeparse(t, format):
    """Parse a time string that might contain fractions of a second.

    Fractional seconds are supported using a fragile, miserable hack.
    Given a time string like '02:03:04.234234' and a format string of
    '%H:%M:%S', time.strptime() will raise a ValueError with this
    message: 'unconverted data remains: .234234'.  If %S is in the
    format string and the ValueError matches as above, a datetime
    object will be created from the part that matches and the
    microseconds in the time string.
    """
    try:
        return datetime.datetime(*time.strptime(t, format)[0:6]).time()
    except ValueError, msg:
        if "%S" in format:
            msg = str(msg)
            mat = re.match(r"unconverted data remains:"
                           " \.([0-9]{1,6})$", msg)
            if mat is not None:
                # fractional seconds are present - this is the style
                # used by datetime's isoformat() method
                frac = "." + mat.group(1)
                t = t[:-len(frac)]
                t = datetime.datetime(*time.strptime(t, format)[0:6])
                microsecond = int(float(frac)*1e6)
                return t.replace(microsecond=microsecond)
            else:
                mat = re.match(r"unconverted data remains:"
                               " \,([0-9]{3,3})$", msg)
                if mat is not None:
                    # fractional seconds are present - this is the style
                    # used by the logging module
                    frac = "." + mat.group(1)
                    t = t[:-len(frac)]
                    t = datetime.datetime(*time.strptime(t, format)[0:6])
                    microsecond = int(float(frac)*1e6)
                    return t.replace(microsecond=microsecond)

        raise

Answer 4

Python 2.6 added a new strftime/strptime macro %f, which does microseconds. Not sure if this is documented anywhere. But if you're using 2.6 or 3.0, you can do this:

time.strptime('30/03/09 16:31:32.123', '%d/%m/%y %H:%M:%S.%f')

I assume the colon between seconds and milliseconds was a typo, but you can of course change the format string to match, if that was intentional.

Edit: I never really work with the time module, so I didn't notice this at first, but it appears that time.struct_time doesn't actually store milliseconds/microseconds. You may be better off using datetime, like this:

>>> from datetime import datetime
>>> a = datetime.strptime('30/03/09 16:31:32.123', '%d/%m/%y %H:%M:%S.%f')
>>> a.microsecond
123000

Answer 5

I know this is an older question but I'm still using Python 2.4.3 and I needed to find a better way of converting the string of data to a datetime.

The solution if datetime doesn't support %f and without needing a try/except is:

    (dt, mSecs) = row[5].strip().split(".") 
    dt = datetime.datetime(*time.strptime(dt, "%Y-%m-%d %H:%M:%S")[0:6])
    mSeconds = datetime.timedelta(microseconds = int(mSecs))
    fullDateTime = dt + mSeconds 

This works for the input string "2010-10-06 09:42:52.266000"