Remove linebreak at specific position in textfile

Question

I have a large textfile, which has linebreaks at column 80 due to console width. Many of the lines in the textfile are not 80 characters long, and are not affected by the linebreak. In pseudocode, this is what I want:

• Iterate through lines in file
• If line matches this regex pattern: ^(.{80})\n(.+)
  • Replace this line with a new string consisting of match.group(1) and match.group(2). Just remove the linebreak from this line.
• If line doesn't match the regex, skip!

Maybe I don't need regex to do this?

Answer 1

Here's some code which should to the trick

def remove_linebreaks(textfile, position=81):
    """
    textfile : an file opened in 'r' mode
    position : the index on a line at which \n must be removed

    return a string with the \n at position removed
    """
    fixed_lines = []
    for line in textfile:
        if len(line) == position:
            line = line[:position]
        fixed_lines.append(line)
    return ''.join(fixed_lines)

Note that compared to your pseudo code, this will merge any number of consecutive folded lines.

Answer 2

Here is an example of how to use regular expressions to archive this. But regular expressions aren't the best solution everywhere and in this case, i think not using regular expressions is more efficient. Anyway, here is the solution:

text = re.sub(r'(?<=^.{80})\n', '', text)

You can also use the your regular expression when you call re.sub with a callable:

text = re.sub(r'^(.{80})\n(.+)', lambda m: m.group(1)+m.group(2), text)

Answer 3

f=open("file")
for line in f:
    if len(line)==81:
       n=f.next()
       line=line.rstrip()+n
    print line.rstrip()
f.close()

Answer 4

Consider this.

def merge_lines( line_iter ):
    buffer = ''
    for line in line_iter:
        if len(line) <= 80:
            yield buffer + line
            buffer= ''
        else:
            buffer += line[:-1] # remove '\n'

with open('myFile','r') as source:
    with open('copy of myFile','w') as destination:
        for line in merge_lines( source ):
            destination.write(line)

I find that an explicit generator function makes it much easier to test and debug the essential logic of the script without having to create mock filesystems or do lots of fancy setup and teardown for testing.