D&C GLug - Home Page

[ Date Index ] [ Thread Index ] [ <= Previous by date / thread ] [ Next by date / thread => ]

Re: [LUG] Clustering Mboards...

 

On Mon, 23 Feb 2009, Tom Potts wrote:

> I'm just wondering how much use clustering might be today:
> A single (?multi-core) CPU can perform as well as Gordons cluster!

That was 1994.. I suspect a quad core Xeon is faster these days... As for 
disk capacity... I think we had one of the first batches of 4GB drives off 
seagate at the time. My camera now has more storage on solid state, and 
it's faster then those compute nodes had!

> OK you need a cluster to run Vista but most other 'tasks' are partitionable to
> run on multiple different spec machines - more cloud than cluster but that
> works out a lot cheaper to maintain!

I've been involved in one way or another with a couple of "supercomputer" 
companies in Bristol for some time since I left my homeland (Edinburgh!) 
just over 20 years ago now...

The biggest issue is programming them. I remember way back people who'd 
written big fortran programs 30 years ago, and they didn't want to change 
them, they just wanted to make them go faster with bigger data sets! 
Parallelising these problems is sometimes impossible, so making one big 
really really fast processor is a valid solution for them... (cf. Cray)

The other solution would be to parallelise the libraries these program use 
- and there are a standard set of them for most scientific applications, 
so the program doesn't change, you just link it with the new go-faster 
libraries which have been re-written to use parallel processing and the 
user is happy.

Ultimately the best way might be to re-write the application to use the 
full powers of the beast, but that's hard and when someone's nurtured 
their (now)  40 year old Fortran program, they might be reluctant to 
change )-:

Compiler technology can play a huge part here too. A (fortran) compiler 
that can recognise a loop written by the programmer that's doing some 
standard operation and output the relevant code can almost be seen to work 
miracles...

> Having said that I'd still like a machine where I could plug in extra
> processors to beef up a couple of things but for me its getting like
> virtualisation - don't need it no more.

Oddly enough there are some occasions when virtualisation can make a task 
appear to perform faster when you'd think it was the reverse - however 
these tasks should always be made to go faster natively if they are 
re-coded.

Tasks for the home user to do with clusters (or even multi-processor) 
systems that work well include graphics rendering - ray tracing - give the 
same data-set to all processors and just tell them to do a particular area 
then merge the results at the end - sort of thing. (You can simulate all 
sorts of things like this - eg. a weather) Fractal generation was always 
the classic demo here too.

I think it's relatively trivial to build a "cluster" these days, but it's 
still as hard as ever to effectively program them.

Oh well...

Gordon

-- 
The Mailing List for the Devon & Cornwall LUG
http://mailman.dclug.org.uk/listinfo/list
FAQ: http://www.dcglug.org.uk/linux_adm/list-faq.html