What is the lightest way of doing this task?

Question

I have a file whose contents are of the form:

.2323  1
.2327  1
.3432  1
.4543  1

and so on some 10,000 lines in each file. I have a variable whose value is say a=.3344

From the file I want to get the row number of the row whose first column is closest to this variable...for example it should give row_num='3' as .3432 is closest to it.

I have tried in a method of loading the first columns element in a list and then comparing the variable to each element and getting the index number

If I do in this method it is very much time consuming and slow my model...I want a very quick method as this need to to called some 1000 times minimum...

I want a method with least overhead and very quick can anyone please tell me how can it be done very fast. As the file size is maximum of 100kb can this be done directly without loading into any list of anything...if yes how can it be done.

Any method quicker than the method mentioned above are welcome but I am desperate to improve the speed -- please help.

def get_list(file, cmp, fout):
    ind, _ = min(enumerate(file), key=lambda x: abs(x[1] - cmp))
    return fout[ind].rstrip('\n').split(' ')

#root = r'c:\begpython\wavnk'
header = 6
for lst in lists:
    save = database_index[lst]
    #print save
    index, base,abs2, _ , abs1 = save
    using_data[index] = save

    base = 'C:/begpython/wavnk/'+ base.replace('phone', 'text')
    fin, fout = base + '.pm', base + '.mcep'
    file = open(fin)
    fout = open(fout).readlines()
    [next(file) for _ in range(header)]
    file = [float(line.partition(' ')[0]) for line in file]
    join_cost_index_end[index] = get_list(file, float(abs1), fout)
    join_cost_index_strt[index] = get_list(file, float(abs2), fout)

this is the code i was using..copying file into a list.and all please give better alternarives to this

Answer 1

Load it into a list then use bisect.

Answer 2

Building on John Kugelman's answer, here's a way you might be able to do a binary search on a file with fixed-length lines:

class SubscriptableFile(object):
    def __init__(self, file):
        self._file = file
        file.seek(0,0)
        self._line_length = len(file.readline())
        file.seek(0,2)
        self._len = file.tell() / self._line_length
    def __len__(self):
        return self._len
    def __getitem__(self, key):
        self._file.seek(key * self._line_length)
        s = self._file.readline()
        if s:
            return float(s.split()[0])
        else:
            raise KeyError('Line number too large')

This class wraps a file in a list-like structure, so that now you can use the functions of the bisect module on it:

def find_row(file, target):
    fw = SubscriptableFile(file)
    i = bisect.bisect_left(fw, target)
    if fw[i + 1] - target < target - fw[i]:
        return i + 1
    else:
        return i

Here file is an open file object and target is the number you want to find. The function returns the number of the line with the closest value.

I will note, however, that the bisect module will try to use a C implementation of its binary search when it is available, and I'm not sure if the C implementation supports this kind of behavior. It might require a true list, rather than a "fake list" (like my SubscriptableFile).