How can I alter a file and write only the changes to disk - basically, sed (python)?
Let's say I have a file /etc/conf1
it's contents are along the lines of
option = banana name = monkey operation = eat
and let's say I want to replace "monkey" with "ostrich". How can I do that without reading the file to memory, altering it and then just writing it all back? Basically, how can I modify the file "in place"?
You can't. "ostrich" is one letter more than "monkey", so you'll have to rewrite the file at least from that point onwards. File systems do not support "shifting" file contents upwards or downwards.
If it's just a small file, there's no reason to bother with even this, and you might as well rewrite the whole file.
If it's a really large file, you'll need to reconsider the internal design of the file's contents, for example, with a block-based approach.
You should look at the fileinput module:
There's an option to perform inplace editing via the input method:
UPDATE - example code:
import fileinput import re import sys for line in fileinput.input(inplace=True): sys.stdout.write(re.sub(r'monkey', 'ostrich', line))
Using sys.stdout.write so as not to add any extra newlines in.
It depends on what you mean by "in place". How can you do it if you want to replace monkey with supercalifragilisticexpialidocious? Do you want to overwrite the remaining file? If not, you are going to have to read ahead and shift subsequent contents of the file forwards.
CPU instructions operate on data which come from memory.
The portion of the file you wish to read must be resident in memory before you can read it; before you write anything to disk, that information must be in memory.
The whole file doesn't have to be there at once, but to do a search-replace on an entire file, every character of the file will pass through RAM at some point.
What you're probably looking for is something like the mmap() system call. The above fileinput module sounds like a plausible thing to use.
In-place modifications are only easy if you don't alter the size of the file or only append to it. The following example replaces the first byte of the file by an "a" character:
fd = os.open("...", os.O_WRONLY | os.O_CREAT) os.write(fd, "a") os.close(fd)
Note that Python's file objects don't support this, you have to use the low-level functions. For appending, open file file with the open() function in "a" mode.
sed -i.bak '/monkey$/newword/' file