[ Date Index ] [ Thread Index ] [ <= Previous by date / thread ] [ Next by date / thread => ]
On 02/10/20 23:41, Simon Waters wrote: > Been too long out of hands on system administration. > > When my desktop box starts backing stuff up it occasionally glitches the > interactive response (I thought it was memory before because I was usually > killing the memory), reading from one disk, writing to another. > > Probably the proper response is buy another SSD (because it is 2020 and 1TB > SSD are under £100) and shuffle the bulk of the home file system onto it and > make sure the IO is hellishly faster and the bottleneck will likely be moved > elsewhere (probably CPU). I may leave that till I buy a new PC. > > However is there anything to do in software to make interactive response > better under this sort of high IO load, because the backup is important but > not urgent, where as me watching YouTube, or playing chess online, is urgent > but not important. > > # cat /sys/block/sda/queue/scheduler > [mq-deadline] none > > (same on sdb) > > # cat /proc/version > Linux version 4.19.0-10-amd64 (debian-kernel@xxxxxxxxxxxxxxxx) (gcc version > 8.3.0 (Debian 8.3.0-6)) #1 SMP Debian 4.19.132-1 (2020-07-24) > > # dpkg -l | grep linux > ....snip... 4.19.0-11 ...snip.. > > I read those as Debian kernel maintainer is opinionated as to my scheduler > options, and I'm guessing they know more than me on this topic. > > I'm due a reboot because I'm a point released behind the Debian kernel. And > purging a load of stale kernels I'll never want again. (Surely someone > automated the apt command to remove old enough kernels by now in Debian?). > > Memory 8GB, ~6GB is available for buffer cache when I'm not running virtual > servers, or similar. ~4GB is I disable the swap partition. > > The backup is "tar cfz /usbmountpoint/SOMEDATEDFILE.tar.gz /home" which is > probably not optimal. It is quite big compared to RAM (~200GB compressed each > backup). > > SuSE documentation is (a) readable (b) says the mq-deadline schedule has the > same tunable parameters as the deadline scheduler. > > https://documentation.suse.com/sles/12-SP5/html/SLES-all/cha-tuning-io.html#sec-tuning-io-schedulers-deadline > > Looking in /sys/block/sda/queue/iosched/ > > I see my values are the same as the SuSE defaults except write_starves is "2" > not "3" (hmm). > > I have "ionice" installed, I don't understand the manual page. I think it is > saying "wrong scheduler" so probably not the tool of choice. > > "top" suggests that the backup is sometimes taking a fair bit of CPU, but CPU > scheduling looks plausibly fair. > > "iotop" suggests tar is reading, and gzip is writing, and there is hardly any > data written to "sdb". > > Trying to interpret "iostat" output is interesting, but it suggests that "sda" > is the disk being backed up to (once various redirections are sorted) and > "sdb" is the source disk, and it is doing a lot of reading from sdb (well it > would) and a lot of writing to "sdb", then sporadically dumping the next chunk > to an almost idle USB disk on "sda". > > Both volumes are encrypted of course. > > Most of the time it rolls along 10 to 20MB/s read by tar, 10 to 20MB/s written > by gzip, and 1 core is 100% running gzip, and responsiveness is okay, but > every so often the read by tar jumps to 50MB/s, I'm assuming this is possibly > large files or something fancy like sparse files? Doesn't seem to correspond > with the problem mind. > > Digging further, the writes to "sdb" were largely swapping out things like the > content of idle browser tabs (even though not memory stressed), and then the > swapping data back in is contending with the backup reads from the same disk. > If I disable swap, it helps a bit with things like recovering browser tabs, > but because other things are not in RAM (code?) it is still stalling to > display things waiting on IO which makes everything slow and unpleasant to > use. > > Tempted to "rsync" all the data to the backup disk first, and/or choose a > method which is more aware of what hasn't changed to either reduce the work or > move it off the main disk. I'd prefer to abandon compression but I don't get > than many backups to the terabyte as it is. > > > Yeah that's a scheduler issue .. probably a bit of both cpu scheduler AND io scheduler. Although I can't really recommend what settings you might try to improve things, because I'm on a somewhat dated and exotic setup here myself. I shall instead sit and wait for meowski to impart some of his wisdom unless anybody else beats him to it ... ;P Best. veremitz/Michael.
Attachment:
signature.asc
Description: OpenPGP digital signature
-- The Mailing List for the Devon & Cornwall LUG https://mailman.dcglug.org.uk/listinfo/list FAQ: http://www.dcglug.org.uk/listfaq