[ Date Index ] [ Thread Index ] [ <= Previous by date / thread ] [ Next by date / thread => ]
On 21/05/12 14:36, Kai Hendry wrote:
> I found the GNU parallel command a bit cumbersome myself.
>
> I prefer Makefiles :-)
>
> http://natalian.org/archives/2010/01/08/Parallelized_processes/
Hmm, on reflection I think I prefer "parallel" for most tasks, although
I don't use it currently. In particular it lets you do substitutions
like "find" does e.g. {}, but not just on file names.
Make is good for the "bring everything up to date" type task, since that
dependency handling is built in, I use it for Postfix tables (where
Postfix needs them put through postmap to hash them), aside from
compilation type tasks. We have a few tasks at work which a Makefile
would have been good for, but someone wrote a Perl script that does too
much instead.
I note that "parallel" comes with GNU sql and GNU sem, GNU sql provides
a command line for SQL which is database agnostic, GNU sem provides a
tool to keep N jobs running at once. Both I think address common
patterns for Linux command line work which are not well enough served
currently. I've tried to use "batch" to achieve a certain amount of
sensitivity to load, that sem might give you, but it didn't work well.
I didn't know xargs could do some of the stuff parallel does, although
the not keeping I/O sequential is something I've hit with other attempts
to run things in parallel, which parallel does right.
Simon
--
The Mailing List for the Devon & Cornwall LUG
http://mailman.dclug.org.uk/listinfo/list
FAQ: http://www.dcglug.org.uk/listfaq