[ Date Index ] [ Thread Index ] [ <= Previous by date / thread ] [ Next by date / thread => ]
On 02/10/2020 23:41, Simon Waters wrote:
Been too long out of hands on system administration.
*appears in puff of magic smoke*
# cat /sys/block/sda/queue/scheduler [mq-deadline] none
You're not that out of practice as you've immediately gone to the right place :]
The BFQ IO scheduler is ideally what you want. Tailored for desktop user responsiveness over any other consideration. Unfortunately not available in super-conservative 4 series Debian kernels but you might have CFQ available?
Your SSD scheduler can be left as is - you didn't actually specify the nature of your backup drive though. Internal SATA HDD(s) I presume? The problem is massively intensified through USB obviously as everyone who's ever had their powerful Linux box unaccountably judder and lag through a simple USB disk copy knows all too well.
You need a kernel with proper IO scheduler support though and you probably can't have that without upgrading to the 5.x series from Debian, building your own or installing an alternative. You want either CFQ (technically depreacted) or ideally, BFQ. Set your backup drive(s) IO scheduler to the desired choice like so:
echo bfq | sudo tee -a /sys/block/sdX/queue/scheduler I set my workstation like this every boot, for reasons*: echo none | sudo tee -a /sys/block/sd*/queue/schedulerYou're doing yourself a bit of a mischief here by being so conservative with your kernels. Don't use old kernels unless you specifically have to for valid reasons. Newer kernels exist for a reason and are no less stable than old ones, use them!
For Debian I hugely recommend Zen or Liquorix 3rd party kernels. Investigate them and make your own mind up but these aren't silly "riced" kernels made up by kids for fun - they are expertly built advanced kernels. If you don't want to make your own but want modern hardware support and features these are the go to choice.
The IO scheduler is the obvious first thing to change - tuning it would be unwise before correcting or tuning other aspects of your system first though. Second would be to correct the CPU scheduler. I boot my kernel like so:
comrade@failbot:~$ cat /proc/cmdlineBOOT_IMAGE=/BOOT/ubuntu@/vmlinuz-5.8.0-pf6-coldwar+ root=ZFS=rpool/ROOT/ubuntu ro bmq.timeslice=2000
BMQ is to CPU scheduling what BFQ is to IO scheduling. The distro defaults for all Linux variants I know are poorly chosen "best average fit" for both of these important components.
The *reasons* I mentioned above for setting ALL of my workstation's disk schedulers to "none" - normally a poor choice - is because I use ZFS and ZFS has it's own much more sophisticated internal IO scheduler so I set "none" to allow it to take over.
The rules is:non-ZFS systems or disks not handled by ZFS have their IO scheduler set to "none" if SSD/NVME and "bfq" if HDD. This is for "responsive" systems, aka PCs. Servers can be different if needed, but usually can be treated with the same rules and will perform better than the defaults.
Speaking of ZFS, it's trivially officially installable on Debian and has been for ages. It solves all of your issues and countless more you didn't even know you had. Your tar backup routine is horrible! ZFS optionally transparently compresses and is a much better choice specifically for bulk backup/data disks.
So I would: Install a proper kernel (at least for testing options if nothing else) Switch backup disk(s) to BFQ scheduler Evaluate switching CPU scheduler to BMQ Next: Buy 8Gb more RAM for that system and another HDD Install ZFS and use it manage that the new HDD Compare ZFS vs your old system Promptly switch all bulk storage to ZFS as inevitable resultSystem tuning is very definitely my cup of tea and there's a lot of other sysctl stuff to start on (vm.swappiness= etc etc) if you want to really get into it but pick the low hanging fruit first. Upgraded kernel provides many advantages including more sophisticated schedulers and tuning bits so you're going to need that as bare minimum.
Actually, looking again and more carefully at your system specs, exactly what you're doing and the fact you have a 4Gb swap partition active on that 8Gb of RAM then yeah, it's not surprising your poor computer isn't very happy with you. I would definitely expect it to be pretty badly impacted if you've just got default everything's set. Ouch!
tar cfz /usbmountpoint/SOMEDATEDFILE.tar.gz /homeis a rude thing to do to that poor machine if it's 200Gb and all disks are encrypted as well. I'm not going to tell a man how to do his own backups but I humbly suggest you could do a lot better, as you suggested yourself. Please at least consider incremental rsync you monster :]
Apologies, that was as "brief" as I could possibly keep it believe it or not.
-- The Mailing List for the Devon & Cornwall LUG https://mailman.dcglug.org.uk/listinfo/list FAQ: http://www.dcglug.org.uk/listfaq