bigtable

Bigtable implementations

I would like to know if there is any fast free/opensource BigTable implementations. ...

Bigtable database design theory

I am very well versed in the theory and practice of relational database design. I know what works and what doesn't, what is performant and what is maintainable (almost - there's always place to tweak when you start having real data). It seems I can't find a substantial body of knowledge regarding distributed scalable databases such as ...

30 million records a day, SQL Server can't keep up, other kind of database system needed?

Some time ago I thought an new statistics system over, for our multi-million user website, to log and report user-actions for our customers. The database-design is quite simple, containing one table, with a foreignId (200,000 different id's), a datetime field, an actionId (30 different id's), and two more fields containing some meta-inf...

Life without JOINs... understanding, and common practices

Lots of "BAW"s (big ass-websites) are using data storage and retrieval techniques that rely on huge tables with indexes, and using queries that won't/can't use JOINs in their queries (BigTable, HQL, etc) to deal with scalability and sharding databases. How does that work when you have lots and lots of data that is very related? I can on...

storing massive ordered time series data in bigtable derivatives

I am trying to figure out exactly what these new fangled data stores such as bigtable, hbase and cassandra really are. I work with massive amounts of stock market data, billions of rows of price/quote data that can add up to 100s of gigabytes every day (although these text files often compress by at least an order of magnitude). This d...

How is MegaStore different from BigTable?

It's been noted that Google App Engine is moving its datastore implementation from BigTable to MegaStore. What's the difference between the two? ...

Facebooks Big Table Implementation

I found out that facebook use whats called a big table. Is there a way that I can use this "Big table" database in an application that I am using? ...

Is HBase meaningful if it's not running in a distributed environment?

I'm building an index of data, which will entail storing lots of triplets in the form (document, term, weight). I will be storing up to a few million such rows. Currently I'm doing this in MySQL as a simple table. I'm storing the document and term identifiers as string values than foreign keys to other tables. I'm re-writing the software...

Is this a suitable (or possible) use of HBase?

I want to use HBase as a store where I can push in a few million entries of the format {document => {term => weight}} e.g. "Insert term X into document Y with weight Z" and then issue a command like "Select the top 1000 terms for this document" or "Select the top 1000 term for each document". This works in my current MySQL implementation...

Move or copy an entity to another kind

Is there a way to move an entity to another kind in appengine. Say you have a kind defines, and you want to keep a record of deleted entities of that kind. But you want to separate the storage of live object and archived objects. Kinds are basically just serialized dicts in the bigtable anyway. And maybe you don't need to index the arch...

iphone table view delete entry and update app engine db

Hi, I have a tableview with data, that i post to the app engine database. Whenever i delete an entry in the table, i want to delelte the item in the app engine database as well. How do i know which entry to delete? I was thinking of this: for every item i save in the GAE store, i make a reference to a model with the iphone unique devi...

In a BigTable datastore, with regards to concurrency, how do I "lock" an entity?

I am not sure how to handle this in a BigTable datastore. Imagine the following example (just to explain the concept. The example does not match my actual data model): I have a Counter entity that keeps track of the number of Transactions in my dataStore. Let's say the current 'count' is 100. Now two web requests read this value at t...

Is BigTable object oriented database?

I want to know that the distributed database system Bigtable is object oriented? ...

Join operation with NOSQL

Hi, I have gone through some articles regarding Bigtable and NOSQL. It is very interesting that they avoid JOIN operations. As a basic example, let's take Employee and Department table and assume the data is spread across multiple tables / servers. Just want to know, if data is spread across multiple servers, how do we do JOIN or UNI...

On google's BigTable, if my entities belong to the same Entity Group (parent), will the entities be fully consistent?

I have started learning about Eventual Consistency. My entities on google's BigTable belong to specific Entity Groups (parents) to support transactionality. My question is: if my entities belong to the correct Entity Groups, and if I do reads/updates in a Transaction, will Reads be fully consistent? (as opposed to Eventually Consistent?...

BigTable Concept

I need Books material or tutorial related to BigTable. ...

Google Appengine: Is This a Good set of Entity Groups?

Hi, I am trying to wrap my head around Entity Groups in Google AppEngine. I understand them in general, but since it sounds like you can not change the relationships once the object is created AND I have a big data migration to do, I want to try to get it right the first time. I am making an Art site where members can sign up as regul...

What aspect of relational databases makes it difficult for them to scale sufficiently on services like Google App Engine?

Apparently the reason for the BigTable architecture has to do with the difficulty scaling relational databases when you're dealing with the massive number of servers that Google has to deal with. But technically speaking what exactly makes it difficult for relational databases to scale? In the enterprise data centers of large corporat...

How build a scalable (relational) database for Petabytes+ of data?

More a general brainstorming question: what is the state-of-the-art in designing a (relational) database which scales to very large amounts of data? And given today's technology trends, how do we expect to design them in 5-10 years? By scalabiliy, I mean in particualar the ability to increase capacity with linear cost by adding hardware...

app engine big table.

what is bigtable. Is any authentication require to create table in bigtable.where the data will be store. it is possible to view the table. we can view all the tables in bigtable, which was created by others. ...