How can I keep a file in memory during editing?

Question

Short version : echo "testing" | vim - | grep "good"

This doesn't work as vim won't output to a pipe. It says : "Vim: Warning: Output is not to a terminal". Any way to do this? Cross-editor support would be nice too.

I've tried named pipes, but vim won't open them.

Long version : echo $passw | gpg -q -d --passphrase-fd 0 $filename | vim - | "somehow gpg encrypt it using $passw and store it back in $filename".

I'm trying to edit a gpg encrypted file but would like to not have the decrypted file on disk at any time.

The full script is here : http://github.com/ptarjan/gpg-encrypt/tree/master

Answer 1

You could mount a tmpfs filesystem (which stores data in memory) somewhere and do your work there.

EDIT (!): Sorry, make that ramfs. The difference is tmpfs can be paged out to swap space, which you don't want. Alternatively, you could turn off all swap devices (with swapoff) and use tmpfs.

Answer 2

You cannot hide anything from the root user, even in memory (they have access to /dev/mem). That's a fact that can't be avoided.

So I'd just use temp files in your home directory which should be protected from everyone except root. On one line, enter:

echo "testing" >~/proc$$ ; vim ~/proc$$ ;
    grep "good" ~/proc$$ ; rm -f ~/proc$$

If you want real security, move the file to a box where you are the only root user and do it there. Then move the encrypted file back.

Answer 3

That is correct: Vim won't output to a pipe. Vim doesn't take its input and output it to a pipe, rather it outputs it into an editor window where you can edit it. You can't continue the pipe sequence after vim. So try:

echo $passw | gpg -q -d --passphrase-fd 0 $filename | vim -

Or don't use vim at all.

If you want to edit the file before passing back to gpg you'll have to do this in a two step process.

Answer 4

Install the moreutils package, it has the vipe program, which edits a pipe in vi.

echo $passw | gpg -q -d --passphrase-fd 0 $filename | ...

Ick! Are you sure your password doesn't show up in a ps process listing? Are you really really really sure it never will?

Answer 5

You could just have vim encrypt it for you.

Before you save the file in vim, filter the content through your gpg encrypter:

    :{range}![!]{filter} [!][arg]      *:range!*
          Filter {range} lines through the external program
          {filter}.  Vim replaces the optional bangs with the
          latest given command and appends the optional [arg].
          Vim saves the output of the filter command in a
          temporary file and then reads the file into the
          buffer.  Vim uses the 'shellredir' option to redirect
          the filter output to the temporary file.
          However, if the 'shelltemp' option is off then pipes
          are used when possible (on Unix).
          When the 'R' flag is included in 'cpoptions' marks in
          the filtered lines are deleted, unless the
          |:keepmarks| command is used.  Example: >
            :keepmarks '<,'>!sort
    <      
          When the number of lines after filtering is less than
          before, marks in the missing lines are deleted anyway.
        w 

So, if you want to filter it through gpg (I'm guessing at the flags here):

:%!gpg -q -d -p $password
:w $filename

You'll need to export the $password and $filename environment variables so vim has access to them if you want to use them within vim.

Answer 6

Keep in mind that by default Vim will create a swap file on disk. Start vim with the following command to avoid it:

vim "+set noswapfile"

Answer 7

You can use the %p command to print the file being edited to the standard output.

In this example, Vim reads its standard input and sends it to the standard output:

$ (echo one; echo two; echo three; echo four) | \
  vim - -nes -u NONE  -c ':%p' -c ':q!' | \
  tail -n +2 | \
  grep t
two
three

Notes:

• vim -: read from standard input
• -n: don't create swapfile, use memory only
• -e: start in ex mode
• -s: silent or batch mode
• -u NONE: don't read the .vimrc (If you want your .vimrc to be read, skip this option. But different people have different .vimrc-s, so if you don't write -u NONE, your code may not work for a different person or even for you if you change your .vimrc.)
• -c <something>: run something as a command
• -c ':%p': print the content of the buffer to the standard output
• -c 'q!': quit without saving
• tail -n +2: throw the first line of the Vim output away (that is the line 'Vim: Reading from stdin...')

In the following example, Vim does actually something useful: it deletes the first column of the standard input.

$ (echo one; echo two; echo three; echo four) | \
  vim - -nes -u NONE  -c ':exec "normal G\<c-v>ggx"' -c ':%p' -c ':q!' | \
  tail -n +2
ne
wo
hree
our

Notes:

• :exec: execute the following command (command-line command, not normal mode command)
• normal: a command-line command that executes the given normal mode commands
• G: go to the last line of the file
• \<c-v>: start block selection
• gg: go to the first line
• x: delete the selected characters

EDIT:

Can I tell vim issue the ":%p" right after I ":wq" from the interactive session?

You can tell Vim things like "run command X before exiting", using e.g. the VimLeave autocommand (see :help autocommand, :help VimLeave):

$ (echo "line 1"; echo "line 2") | \
  vim - -nes -u NONE -c ':autocmd VimLeave * :%p'
Vim: Reading from stdin...
:q!        # you type :q!
line 1
line 2

The problem is with the ":%p" command. If you use the "-e" argument, you won't get the visual editor. If you don't use "-e", Vim won't print the result of ":%p" to the standard output but to the terminal.

I will make a different suggestion in another post.

Answer 8

A solution to the original problem (as far as I see it):

secret.txt contains the encrypted text.

./encode.sh is the symmetric encoder script.

The following command will read the text from secret.txt, decode, make it editable in Vim, encode, and write it back to secret.txt:

$ cp secret.txt | \
  ./encode.sh | \
  vim - -n -u NONE \
      -c ':autocmd VimLeave * :exec "%!./encode.sh" | write! tmp'; \
  mv tmp secret.txt

Notes:

• :autocmd VimLeave * <command>: perform <command> before leaving Vim on any file
• :exec "%!./encode.sh" | write! secret.txt: run ./encode.sh on the whole content of the buffer as a filter, then write the buffer into secret.txt
• "As a filter" means that the content of the buffer will be fed into ./encode.sh. The content of the buffer will be replaced with the standard output of ./encode.sh after it finished its execution.

But I have to say, this solution is ugly. I would recommend the same as rampion: have a look at the autocommands mentioned (:help autocommand) and try to do the whole edit process within Vim.

Finally a side note: Vim has its own encryption command which does exactly what you described on your project web page: "It simply decrypts the file into a file readable only by you, you edit it in your favorite editor, and then it rencrypts it and saves the file back where it came from."

The Vim help (:help :X) says about the algorithm:

• The algorithm used is breakable. A 4 character key in about one hour, a 6 character key in one day (on a Pentium 133 PC). This requires that you know some text that must appear in the file. An expert can break it for any key. When the text has been decrypted, this also means that the key can be revealed, and other files encrypted with the same key can be decrypted.
• Pkzip uses the same encryption, and US Govt has no objection to its export. Pkzip's public file APPNOTE.TXT describes this algorithm in detail.