cx_Oracle MemoryError when reading lob

Question

When trying to read data from a lob field using cx_Oralce I’m receiving “exceptions.MemoryError”. This code has been working, this one lob field seems to be too big.

Example:
xml_cursor = ora_connection.cursor()
xml_cursor.arraysize = 2000
try:
    xml_cursor.execute(“select xml_data from xmlTable where id = 1”)
    for row_data in xml_cursor.fetchall():
        str_xml = str(row_data[0])  #this throws “exceptions.MemoryError”

Answer 1

Yep, if Python is giving MemoryError it means that just that one field of that just one row takes more memory than you have (quite possible with a LOB of course). You'll have to slice it up and get it in chunks (with select dbms_lob.substr(xml_data, ... repeatedly) and feed it to an incremental XML parser (or write it out to a file, or whatever is it that you're trying to do with that multi-GB LOB). DBMS_LOB is a well-documented Oracle-supplied package, and you can find its docs in many places, e.g. here.