I'm developing an application for Google App Engine which uses BigTable for its datastore. It's an application about writing a story collaboratively. It's a very simple hobby project that I'm working on just for fun. It's open source and you can see it here: http://story.multifarce.com/ The idea is that anyone can write a paragraph, wh...
I'm in the process of spiking a conversion from MySQL to Cassandra for PenWag.com. In Cassandra, I'm storing Users keyed off of a GUID, but users sign in with their email, not the GUID (obviously). GUID as a key for Users makes sense to me more than email for two reasons. From a practical perspective it seems that it's too cumbersome ...
I've been working with Google App Engine and I'm running into some slow performance with some of my data queries. I've read that designing an App Engine datastore is a different mindset from working with SQL databases and I'm not sure I'm doing this the best way. I have two questions to try to get on the right track: Specifically: I ...
What are recommended resources for learning HBase? The only one I can think of is HBase wiki and one chapter in book "Hadoop: The Definitive Guide", are there any other good resources? I'm looking for links, books, wikis, etc. Stuff about BigTable is also welcome. Thanks. ...
How does the performance of db.get() compare with that of db.get_by_key_name()? ...
bulkoader.yaml: transformers: - kind: ExampleModel connector: csv property_map: - property: __key__ external_name: key export_transform: transform.key_id_or_name_as_string - property: data external_name: data - property: type external_name: type ...
Hi. I'm trying to upload some records to my local data store using appcfg.py Only a small number of records are actually inserted and I get the following output (with a ton of errors): $ appcfg.py upload_data --config_file=bulkloader.yaml --filename=/output.csv --kind=AutoCompleteIndex --url=http://localhost:8084/remote_api . ...
Hi guys. When I try to upload data using appcfg.py I get an authentication error regardless of my login details being completely correct. appcfg.py upload_data --config_file=bulkloader.yaml --filename=/output.csv --kind=AutoCompleteIndex . Application: qwertifyy; version: dev. Uploading data records. [INFO ] Logging to bulkloader...
Hi, by default, GAE adds a unique key to your db entries. Is it possible to have a numeric unique key in addition? Elias ...
A sample model: class Foo(db.Model): id = db.IntegerProperty() bar = db.ListProperty(int, required=True) How can I query using either Query or GqlQuery to return all Foo entities that have a given value in their bar property? If I have a list of ids, is there a single filter that will return all entities whose id property...
Hi, I have the following data model: class StadiumOccupant { String stadiumname; String username; } class Friend { String username; String friendname; } I want to find all users at a stadium that are also my friends. I can do that like this: List<String> friendsAtStadium; String stadiumId = 'xyz'; List<Friend> friends = se...
I'm kinda new to modeling my data model for efficient querying with GAE, but have extensive knowledge with RDBMS. Here's the problem: I got roughly a million terms (strings) and need to query and compare associated numerical values as a time series with weekly data points. Think of it as a graph with time on the X axis and a linear...
Is there any bridge/native method to use the Bigtable / Cassandra DB arhitecture? ...
We have BigTable from Google, Hadoop, actively contributed by Yahoo, Dynamo from Amazon all aiming towards one common goal - making data management as scalable as possible. By scalability what I understand is that the cost of the usage should not go up drastically when the size of data increases. RDBMS's are slow when the amount of...
This is a shot from Google BigTable paper What can be the kind of scenarios in which instead of having something like Oracle's redo logs, I will need to store multiple versions of the same data within the database? Coming specifically to this example, why do I need to store multiple versions of a html page in my database? It cannot ...
Hi. See the pagination on the app gallery? It has page numbers and a 'start' parameter which increases with the page number. Presumably this app was made on GAE. If so, how did they do this type of pagination? ATM I'm using cursors but passing them around in URLs is as ugly as hell. ...
any insight as to making/optimizing full-text searches on bigtable using java? best practices and such? how do u guys do it? ...
I am trying to understand how Django and Appengine work together? First, question: Is this a good team? Experience, what is possible and what not, would be great. I also read some modules like auth, admin wont work. But the article is rather old, so maybe there is an update. And in that tutorial one has to import bforms. What is tha...
I like to work with data saved in one GAE application in other GAE applications. Basically share the datastore between multiple web applications in Google App Engine (Python) Development and Production. Also if possible with: http://localhost:####/_ah/admin/datastore I like to view data in other applications not runnings and/or running...
Hi, I'm new in google-app-engine and google datastore (bigtable) and I've some doubts in order of which could be the best approach to design the required data model. I need to create a hierarchy model, something like a product catalog, each domain has some subdomains in deep. For the moment the structure for the products changes less t...