I've observed that different systems and filesystems behave very differently with respect to their interactions with the page cache.

When I was using ext4 and overwrote an existing file (eg. dd if=source of=target, where target already exists) the close() system call is made and there was a code path taken in the kernel that went into ext4_release_file() that caused the pages for the file to be flushed synchronously when ext4 called filemap_flush() and it killed performance. If I mimic the behavior that SAS has when you update a dataset in place this didn't happen. The IO used the page cache and I was basically stuck behind the speed of CPU as the kernel shuffled data pages around in the page cache. Everyone was super fast. When I used ZFS on Linux I would see the throughput fluctuate slightly based on how close to the periodic sync ZFS does that I began the write.

I'd say the moral of the story is experiment, trace and tune.