By Upper tens of thousands requests/second I want to see 60,000 -> +90,000 requests/second. My Setup consists of the following: user ---> web app --> message queue --> parser --> database? I should mention that the parser currently can parse/stuff around 18750 records/second using COPY so we are limited on that end until we start addi...
I'm looking for general experiences from people who have used both, particularly on how the two compare on handling large numbers of records, transaction/concurrency/deadlock handling, and juicy stories about database corruption and backup procedures. ...
I am looking for python bindings for Table engine of Tokyo cabinet. I tried Pytc but can only find Hash and B-tree engine support. Is there any other bindings available? ...
I am evaluating tokyo cabinet Table engine. The insert rate slows down considerable after hitting 1 million records. Batch size is 100,000 and is done within transaction. I tried setting the xmsiz but still no use. Has any one faced this problem with tokyo cabinet? Details Tokyo cabinet - 1.4.3 Perl bindings - 1.23 OS : Ubuntu...
I was wondering if anyone has seen a library that abstracts away SQLite and Tokyo-Cabinet to the same API. Basically I'm looking for something that will allow me to test and dev with SQLite, and later move to TC for production. Anyways the language I'm using specifically is Lua, but I'm sure such a library doesn't exist for Lua, so a L...
Hi what do you suggest as a cross platform "almost all encompassing" abstraction toolkit/library, not necessarily gui oriented? the project should at some point include an extremely minimal web server and a "db" of some sort (basically to have indexes/btrees, maybe relations, so a rdbms is desiderable but avoidable if necessarily, sql ...
Has anyone successfully used Tokyo Cabinet / Tokyo Tyrant with large datasets? I am trying to upload a subgraph of the Wikipedia datasource. After hitting about 30 million records, I get exponential slow down. This occurs with both the HDB and BDB databases. I adjusted bnum to 2-4x the expected number of records for the HDB case with onl...
I have implemented a queue on top of TC (Tokyo Tyrant to be specific). I am using memcache compatible function increment to keep track of queue head and tail. I just want to know what is the limit for the integers stored (64 bit?) by TC and what happens when increment function is called after the number stored is at maximum value allowed...
Is there a C++ wrapper for the Tokyo Cabinet API? Obviously I can use the basic C API from C++, but I'd much rather use something that abstracted the raw functions and structures into classes. I'm tempted to write this myself, at least for the bits of the API that I'd want to use, but it'll save time if there's one already. I checked th...
Is it possible to use Tokyo Cabinet with any of the .Net Languages, preferably VB10? How about Tokyo Tyrant? Preferably without going through LUA/Ruby or other wrappers.. If there's no easy way in doing so, can you recommend other fast Key-Value Database which can be used? ...
Hey guys, Has anyone used Tokyo Cabinet on the iPhone? I'm interested to see if there are any real world performance differences between it and SQLite3. Also, SQLite 3 has the expressive power of SQL, does Tokyo Cabinet have any kind of query language? Any input would be greatly appreciated, thanks. ...
i quote from the tokyo cabinet docs... As for database of hash table, each key must be unique within a database, so it is impossible to store two or more records with a key overlaps. or does tokyocabinet allow tuple based keys ? what would be the best way to set a one-to-many store ( like a crawler 1 kw<->many docids ) ~B ...
Hello, I am writing a ruby script to be used as Postfix SMTP access policy delegation. The script needs to access a Tokyo Tyrant database. I am using EventMachine to take care of network connections. EventMachine needs a EventMachine::Connection class that is instantiated by EventMachine‘s processing loop whenever a new connection is cr...
I'm building a system, with C++, that uses Tokyo Cabinet (original API in C). The problem is I want to store a class such as: class Entity { public: string entityName; short type; vector<another_struct> x; vector<another_struct> y vector<string> z; }; The problem is that vectors an...
Hi, I'm stuck at a Segmentation fault after inserting about 8 million records in a TC Hash Database. After everything is inserted I close the DB but I caught a Segmentation Fault at this part of the code (tchdb.c): static void tchdbsetflag(TCHDB *hdb, int flag, bool sign){ assert(hdb); char *fp = (char *)hdb->map + HDBFLAGSOFF; if...
Let's say I have N > 1 TCP-based, connection-oriented (read: not a website) services handling connections from end users in some load-balanced/sharing configuration. These users do things that cause updates to one or more keys in the centralized Tokyo Tyrant datastore. What do you recommend to push these changes to interested users c...
Helle there, compiling the perl api for tokyo cabinet is diving me crazy ! i've always this error message : ld: warning: in /usr/local/lib/libtokyocabinet.dylib, file is not of required architecture i've tried de rebuild tokyop cabinet en 64bit with -arch x86_64 but nothing changes .. i'm using mac os x 10.6 (snow leop...
I'm writing a small Java-based server around Tokyo Cabinets JNI-bindings, and I am a bit confused regarding opening and closing of the db file. The server is multi-threaded, and I'm therefore wondering if it is best to open the db file upon start-up and then let the threads use get/put unsynchronized (I think that the tokyo docs state th...
Has anybody compared SimpleDB and Tokyo Cabinet for performance and scalability? I'm coding my project against SimpleDB at the moment and considering benchmarking TC, be nice if somebody had already done it and could tell me whether it's worth testing my specific storage and searching operations. If not I'll run some direct comparisons a...
I have 2 servers running tokyo cabinet in a cluster (2 hard drives so 4 instances of Tokyo Cabinet). if it's just one instance of tokyo cabinet it's fairly straight forward to use php memcache protocol to communicate. My question is how do you connect php if it's cluster? Option 1: $memcache->connect('memcache_host', 11211); Using thi...