Object for storing strings in Python

Question

class MyWriter:

    def __init__(self, stdout):
        self.stdout = stdout
        self.dumps = []

    def write(self, text):
        self.stdout.write(smart_unicode(text).encode('cp1251'))
        self.dumps.append(text)

    def close(self):
        self.stdout.close()   

writer = MyWriter(sys.stdout)
save = sys.stdout
sys.stdout = writer 

I use self.dumps list to store data obtained from prints. Is there a more convenient object for storing string lines in memory? Ideally I want dump it to one big string. I can get it like this "\n".join(self.dumps) from code above. May be it's better to just concatenate strings - self.dumps += text?

Answer 1

I am quite sure, that a single '\n'.join(self.dumps) will be much faster than self.dumps += text.

Explanation: In Python, strings are immutable, so if you concat two strings, a new string is generated and the two other strings are copied into it. That's not a problem if you do it once, but inside a loop, this will copy the whole text in every iteration. join() on the other hand is a builtin function written in C, which has the ability to reallocate memory efficiently and change the end of the string. So, it should be much faster.

So, you source is perfectly fine. Great work!

PS: the flush() function is missing

Answer 2

A list of strings to be joined with ''.join is just fine. However, if you prefer a more direct solution:

import cStringIO

class MyWriter(object):

    def __init__(self, stdout):
        self.stdout = stdout
        self.dumps = cStringIO.StringIO()
        self.final = None

    def write(self, text):
        self.stdout.write(smart_unicode(text).encode('cp1251'))
        self.dumps.write(text)

    def close(self):
        self.stdout.close()
        self.final = self.dumps.getvalue()
        self.dumps.close()

    def getvalue(self):
        if self.final is not None:
            return self.final
        return self.dumps.getvalue()

getvalue cannot be called on a string-io object after it's closed (closing the object makes it drop its own buffer memory) which is why I make self.final just before that happens. Apart from the getvalue, a string-io object is a pretty faithful implementation of the "file-like object" interface, so it often comes in handy when you just want to have some piece of code, originally designed to print results, keep them in memory instead; but it's also a potentially neat way to "build up a string by pieces" -- just write each piece and getvalue when done (or at any time during the process to see what you've built up so far).

Modern Python style for this task is often to prefer the lower-abstraction approach (explicitly build a list of strings and join them up at need), but there's nothing wrong with the slightly higher-abstraction "string I/O" approach either.

(A third approach that seems a bit out of favor is to keep extending an array.array of characters, just to be comprehensive in listing these;-).