wx.TextCtrl.LoadFile()

Question

I am trying to display search result data quickly. I have all absolute file paths for files on my network drive(s) in a single, ~50MB text file. The python script makes a single pass over every line in this file [kept on the local drive] in a second or less, and that is acceptable. That is the time it takes to gather results.

However, the results are given in a wx.TextCtrl widget. Appending them line by line to a wx TextCtrl would be ridiculous. The best method I have come up with is to write the results to a text file, and call wx.TextCtrl's LoadFile native, which, depending on the number of results, loads the lines of text into the pane in between 0.1 to 5 seconds or so. However there must be a faster way for 10+MB of text inbound. The results are immediately calculated and available in the same process as the GUI... so please, tell me is there any way I can pipe/proxy/hack that data directly into the TextCtrl? Would mmap help this transfer?

Answer 1

Are people really going to read (or need) all 10MB in a text control? Probably not.

Suggest that, you load on demand by paging in portions of the data.

Or better still, provide some user search functionality that narrows down the results to the information of interest.

Answer 2

You can load all the data at once with AppendText, why you need to do it line by line, but still it will take seconds as 10MB is huge. If you use wx.RichTextCtrl it is bit faster in my test it loaded 10 MB in 6 secs instead of 9 sec for TextCtrl.

I do not see the reason why you need to set all the data at once? and who is going to read 10MB?

So depending on the purpose there can be better ways.

If you need to display all data in super fast way, write a custom control which keeps a list of lines and only renders the lines visible in the view.

here is a test app where you can try various things

import wx
import wx.richtext
import string
import time

# create a big text
line = string.ascii_letters+"\n"
bigText = line*200000

app = wx.PySimpleApp()
myframe = wx.Frame(None)
#tc = wx.TextCtrl(myframe, style=wx.TE_MULTILINE)
tc = wx.richtext.RichTextCtrl(myframe)
def loadData():
    s = time.time()
    tc.SetMaxLength(len(bigText))
    tc.AppendText(bigText)
    print time.time()-s

# load big text after 5 secs
wx.CallLater(5000, loadData)

app.SetTopWindow(myframe)
myframe.Show()
app.MainLoop()

If you do not want to paint everything youself in a custom control, you can just use a textctrl with separate scrollbar and update text of textcntrl on scrolling, so at a time you will be loading few lines only.

Edit: as you said in your comment that data may be 1-2 MB, 1MB data with AppendText takes only .5 sec I think that is ok