[ Date Index ] [ Thread Index ] [ <= Previous by date / thread ] [ Next by date / thread => ]
On 24/03/14 15:49, Martijn Grooten wrote: > On Sun, Mar 23, 2014 at 05:48:44PM +0000, Simon Waters wrote: >> However file system is rarely dominant in IO performance. >> >> Sure if everything is perfect, then file system can make a slight >> difference, but they tend to all try and optimize the common cases. > > Which of course make sense. > > Looks like I'll have to see if I can find some other ways to improve IO > performance. Such as by yet again looking at my code. > > Thanks. > > Martijn. > > Are you free to rebuild the kernel on the current host? My main workstation spends most of it's life absolutely battering the disks which is partly why I eventually had to start building my own. Amongst others, I pull in this patchset: https://pf.natalenko.name/ The interesting bit for you is the BFQ IO scheduler to replace the standard Linux CFQ elevator. BFQ has much better responsiveness under heavy disk load - there are some benchmarks and explanations here: http://algo.ing.unimo.it/people/paolo/disk_sched/ So on my system: ghost@failbot:~$ !491 cat /sys/block/sda/queue/scheduler noop deadline cfq [bfq] You may benefit even further from noop as an elevator instead, especially if your server has enough RAM for aggressive caching. Speaking of RAM, I don't know how big the dataset is that you're hitting so hard on the server, but is it small enough to be mounted on a RAM disk? All my compiles take place within a TMPFS that is whipped up on demand (a trick I originally learnt from temerge vs emerge on Gentoo) by a script. The performance gains are spectacular, unsurprisingly. Along with tuning sysctl, etc, there's still a LOT of optimisation that can be wrung out of your machine yet... Regards -- The Mailing List for the Devon & Cornwall LUG http://mailman.dclug.org.uk/listinfo/list FAQ: http://www.dcglug.org.uk/listfaq