D&C Lug - Home Page
Devon & Cornwall Linux Users' Group

[ Date Index ][ Thread Index ]
[ <= Previous by date / thread ] [ Next by date / thread => ]

Re: [LUG] Anyone fancy helping?



-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Neil Williams wrote:
On Wednesday 24 Sep 2003 11:27 pm, Neil Stone wrote:

Neil Williams wrote:

In for a penny.. in for a pound... I have a few spare machines here...
frobably the friendliest sysadmin i know... (me)


:-))


Any ideas how much space might be needed initially ? I can always get a
new hdd should the need arise...


Initially, the HTML and other content is tiny - it'll fit on a floppy. The MySQL database will gradually build with time. I'm trying to improve the efficiency (currently about 20/hr added, 24/7 via cron) but as to how large it gets, that's down to your own configuration of the scripts and the popularity of the site. I'm currently estimating a database of at least 5,000 records before the site could become self-sustaining. That is a real shot-in-the-dark estimate though - pick a number and double it stuff. It could easily be more than double that before the number of failed searches becomes 'acceptable'.

34,000 records of an old style (larger record size) database took up 49MB just for the MySQL data. I'm hoping that the new database structure will be 50% of the size of the equivalent old style, so 8MB/10,000 records. Not large, but then it's hard to tell just how big the database will need to be. Perhaps 10,000 is a wildly low estimate. Maybe 250,000 would be needed - that's about the most I can envisage on any one server. Certainly my first active site, isbn.org.uk is setup with a 250MB account. Before it gets anywhere near that I'll have some kind of distributed export and synchronisation protocol for these servers. A bit like GnuPG keyservers, ISBN's are never deleted or re-assigned or reused. Even if a book is out of print, the ISBN is still valid because the book will be available second-hand somewhere and there are already online sites that will attempt to find these books. So the database needs to be distributed and probably needs to co-opt an ever larger array of servers.

It's v.v.v.early days and it's hard to see that far ahead. If the project is to meet the eventual target of all ISBN's there'll need to be quite a few of these servers! (Anyone like to hazard guess of how many ISBN's might be out there? The people who issue ISBN's don't know! (or even care, probably).)
How many new books are published each year? Anyone know? (Not how many books are sold, how many new titles or new editions?)




somewhere in the region of 10 thousand million ISBN's should be available !! (roughish guess)


250Mb ?? no problem... I can at least match that if not quadruple it... (well a little less as it will need the OS in that space too...)

How bandwidth hungry is it likely to be ? I can't really see it being too hungry as it is going to be text transfers...


- -- Neil Stone


Flash on telnet://TR3.Org:3000 and irc://irc.r-t-f-m.org.uk/DCLUG

******************************************************************************

SELECT * FROM windows.users WHERE (clue != NULL AND clue > 0);

0 row(s) returned.


"Backups are for wimps. Real men upload their data to an FTP site and have everyone else mirror it." - Linus Torvalds
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org


iD8DBQE/c2Jnz3Av8JKgzxQRAg8nAKCtVhWfFNdCtOGOhScogfLzzu+XxwCfZCN3
9T4X57fmcgfLRFOjR6MQtrw=
=xe9j
-----END PGP SIGNATURE-----


-- The Mailing List for the Devon & Cornwall LUG Mail majordomo@xxxxxxxxxxxx with "unsubscribe list" in the message body to unsubscribe.


Lynx friendly