Hey friends
I am just a beginner in MySQL, I need to know how much data can be stored in MySQL. I am developing a web crawler, can I store all the data in MySQL, or do I need to use another Database? Which is more faster? What I mean is, which has the highest Writing/Reading Rate? Do I need to reconfigure to add more data?
views:
133answers:
1
+3
A:
Depends on the operating system.
**Operating System** **File-size Limit**
Win32 w/ FAT/FAT32 2GB/4GB
Win32 w/ NTFS 2TB (possibly larger)
Linux 2.2-Intel 32-bit 2GB (LFS: 4GB)
Linux 2.4+ 4TB
Solaris 9/10 16TB
MacOS X w/ HFS+ 2TB
NetWare w/NSS file system 8TB
http://dev.mysql.com/doc/refman/5.0/en/full-table.html
Your write/read rate is of pretty much no concern to you, your bottleneck is going to be your internet connection.
Tom Gullen
2010-08-05 16:55:18
i am using a win32/NTFS system,what will i do if the data goes higher than 4tb??its already 3TB
Alex Mathew
2010-08-05 16:58:59
Find a different DBS, SQL server on a 64bit machine can have 16TB. http://msdn.microsoft.com/en-us/library/ms143432.aspx Just shop around and see what tickles your onion jar. Oracle can handle 8589541376 G on a 'BFT' addressing scheme. I don't think MYSql was designed with your needs in mind.
Tom Gullen
2010-08-05 17:02:29
@Tom Gullen: So you are suggesting SQL Server,is there any Open Source DB for my needs?what DB did google using?
Alex Mathew
2010-08-05 17:04:23
@Alex: google is using homegrown BigTable (http://stackoverflow.com/questions/362956/what-database-does-google-use).
Wrikken
2010-08-05 17:08:45
@Wrikken : Thks for the info,in my case what all DB can use?what are the solutions? what about Hadoop?is it a DB,can i use that for a webcrawler?
Alex Mathew
2010-08-05 17:11:01
.. and the MySQL manual actually suggests some solutions for a bigger table, you could just employ one of those.
Wrikken
2010-08-05 17:11:31
@Wrikken:what are they?can you please say?
Alex Mathew
2010-08-05 17:14:28
Wrikken
2010-08-05 17:21:33
Alex, Check out Hadoop.
Tom Gullen
2010-08-05 17:43:21
@Tom : Can i use Hadoop as a DB?
Alex Mathew
2010-08-05 18:00:32
No idea sorry but check it out might be good
Tom Gullen
2010-08-05 20:54:14