Re: Flushing buffer on file copy on linux
J <dreadpiratejeff <at> gmail.com> writes:
> Now, the problem I have is that linux tends to buffer data writes to a
> device, and I want to work around that. When run in normal non-stress
> mode, the program is slow enough that the linux buffers flush and put
> the file on disk before the hash occurs. However, when run in stress
> mode, what I'm finding is that it appears that the files are possibly
> being hashed while still in the buffer, before being flushed to disk.
Your analysis is partly wrong. It is right that the files can be hashed from
in-memory buffers; but even if you flush the buffers to disk using standard
techniques (such as fsync()), those buffers still exist in memory, and
therefore the file will still be hashed from memory (for obvious efficiency
I don't think there's a portable solution to get away entirely with the
in-memory buffers, but under Linux you can write "1" to the special file
$ sudo sh -c "echo 1 > /proc/sys/vm/drop_caches"
Or, to quote the /proc man page:
/proc/sys/vm/drop_caches (since Linux 2.6.16)
Writing to this file causes the kernel to drop clean
caches, dentries and inodes from memory, causing that mem‐
ory to become free.
To free pagecache, use echo 1 > /proc/sys/vm/drop_caches;
to free dentries and inodes, use echo 2 >
/proc/sys/vm/drop_caches; to free pagecache, dentries and
inodes, use echo 3 > /proc/sys/vm/drop_caches.
Because this is a nondestructive operation and dirty
objects are not freeable, the user should run sync(8)
Software development and contracting: http://pro.pitrou.net
|All times are GMT. The time now is 06:13 PM.|
Powered by vBulletin®. Copyright ©2000 - 2013, vBulletin Solutions, Inc.
SEO by vBSEO ©2010, Crawlability, Inc.