D&C GLug - Home Page

[ Date Index ] [ Thread Index ] [ <= Previous by date / thread ] [ Next by date / thread => ]

Re: [LUG] OT: [Fwd: Hard drive evolution could hit XP]

 

george parker wrote:
>
> ...I seem to remember having this argument about sector sizes maybe 10
> or 12 years ago (maybe 15 years ago) when the problem was the size of disc 
> the operating system could handle.

Hehe, having used IBM mainframes that thought they had IBM3380 disks in,
thousands and thousands of them, when the disks were orders of magnitude
larger and doing whatever was efficient to do under the hood and
ignoring their confused upstream masters, this really looks like the
disk drive makers problem to me and definitely the same issue was
addressed there decades ago.

Although in that case they hit a slightly different problem, that the
I/O layer had a limit of 2^16 bytes per block, but it was no longer
efficient to move data in chunks that small, so every read/write of more
than 32KiB was less efficient (in some arcane way) than the hardware was
capable off (at both ends). I dare say they have moved on since then.

> Or am I having a senior moment?

Yes, you are confusing sectors and blocks to start with (although given
the history an easy thing to mix up).

Since the default block size in ext3 is 4KiB already, most people are
already wasting disk space when storing small files in the manner you
describe, and the world hasn't ended yet.

I usually use ReiserFS at work where we have a LOT of small files, which
does tail packing (optional, defaulted on - which is why it rules the
small file benchmarks) and thus avoids this problem.

ZFS has block level deduplication, choice of compression algorithms
(which can be changed dynamically at any time) for block level
compression, and variable block size, you don't even want to start
thinking about the implications of this for disk space efficiency, or
I/O efficiency. Let us say it makes sense for hosting virtual operating
systems and many other tasks. Although if SUN's^W Oracle's marketing
plan is selling to people who understand file systems they aren't going
to sell much to most purchasers of IT systems.

 Simon with a bad case of file system envy.



-- 
The Mailing List for the Devon & Cornwall LUG
http://mailman.dclug.org.uk/listinfo/list
FAQ: http://www.dcglug.org.uk/linux_adm/list-faq.html