partitioning

MySQL Proxy Alternatives for Database Sharding

Are there any alternatives for MySQL Proxy. I don't want to use it since it's still in alpha. I will have 10 MySQL servers with table_1 table_2 table_3 table_4 ... table_10 spread across the 10 servers. Each table is identical in their structure, their just shards with different data sets. Is there a alternative to MySQL Proxy, where...

Separating SQL Results Into Ranges

I would like to take a simple query on a list of members that are indexed by a number and group them into 'buckets' of equal size. So the base query is: select my_members.member_index from my_members where my_members.active=1; Say I get 1000 member index numbers back, now I want to split them into 10 equally sized groups by a max and...

Algorithm for finding nearby points?

Given a set of several million points with x,y coordinates, what is the algorithm of choice for quickly finding the top 1000 nearest points from a location? "Quickly" here means about 100ms on a home computer. Brute force would mean doing millions of multiplications and then sorting them. While even a simple Python app could do that in ...

Format a device with parted into a specific pattern using parted

Hi, This is really a parted usage question, but other ideas about how to accomplish this are welcome. I need to create a boot device set up as follows: ( up to 4MB erase block size (EBS)): partitions aligned on 4MB blocks by using 32 sectors/track and 128 heads and using odd start numbers (counting from 1) MBR: s...

How to partition a problem into smaller understandable portions?

I'm not sure if it's possible to give general advice on this topic, but please try. It's hard to explain my case because it's too complex to explain. And that's exactly the problem. I seem to constantly stumble on a situation where I try to design some part of my project, but it has so many things to take into consideration that I'm una...

Find all possible factors in KenKen puzzle 'multiply' domain

A KenKen puzzle is a Latin square divided into edge-connected domains: a single cell, two adjacent cells within the same row or column, three cells arranged in a row or in an ell, etc. Each domain has a label which gives a target number and a single arithmetic operation (+-*/) which is to be applied to the numbers in the cells of the do...

Techniques for removing old data on Oracle databases

We have a mature Oracle database application (in production for over 10 years), and during that time, we have been using scripts of our own devising to remove old data that is no longer needed. They work by issuing delete statements against the appropriate tables, in a loop with frequent commits, in order to avoid overloading the system...

Approaches to table partitioning in SQL Server

The database I'm working with is currently over 100 GiB and promises to grow much larger over the next year or so. I'm trying to design a partitioning scheme that will work with my dataset but thus far have failed miserably. My problem is that queries against this database will typically test the values of multiple columns in this one ...

what is a good way to horizontal shard in postgresql

Hey guys what is a good way to horizontal shard in postgresql 1. pgpool 2 2. gridsql which is a better way to use sharding also is it possible to paritition without changing client code It would be great if some one can share a simple tutorial or cookbook example of how to setup and use sharding ...

How can I use an index on a partitioned table in postgresql 8.3.7

I have situation, where running a query that filters by an indexed column in a partitioned table, performs a full table scan. Apparently , this is a known issue in postgresql, and it's explained in detail here. Is there a more elegant way around this other than performing a query on each partition, and then performing a UNION on all of...

How do we do horizontal sharding/partitioning in Postgresql using pgpool-II?

Grid sql and pgpool-ii are partitioning tools for postgresql. gridsql is designed for reporting business applications. PGPool-II for transactional systems. Can some one show me how to do a horizontal partition on the bigint column uid on table users? thanks ...

Distributed Key-Value Data Store with Offline Access (Static Partitioning)

Need to be able to set server(s) that replicate all information, as a master data store that has all the data. Also need servers that specifically store/replicate certain data, available in local LANs, so that when the internet connection goes down, they can still access their local data. Under normal circumstances, the clients will ac...

Are there any cleverly efficient algorithms to perform a calculation over the space of partitionings of a string?

I'm working on a statistical project that involves iterating over every possible way to partition a collection of strings and running a simple calculation on each. Specifically, each possible substring has a probability associated with it, and I'm trying to get the sum across all partitions of the product of the substring probability in ...

Breaking down webapp into multiple pieces (tomcat)

I have a large monolithic webapp, and whenever I make changes to a particular section of the webapp I am forced to deploy the entire app. I would like to break this down into smaller chunks based on functionality (e.g. In a Banking website - I want to seperate out informational stuff from the online banking functionality, so if I make a...

Oracle 10g Partitioning: Can I partition a subtable?

I have a main table with a primary key and a partition column (a date) and five subtables which use foreign key references to the primary key. Is there a way with Oracle 10g to partition the subtables in the same way as the main table without duplicating the partition column? ...

How should I range partition an index with a varchar2 column in Oracle? Is it a bad idea?

I am using Oracle 10g Enterprise edition. A table in our Oracle database stores the soundex value representation of another text column. We are using a custom soundex implementation in which the soundex values are longer than are generated by traditional soundex algorithms (such as the one Oracle uses). That's really beside the point. ...

Is a globally partitioned index better (faster) than a non-partitioned index?

I'm interested to find out if there is a performance benefit to partitioning a numeric column that is often the target of a query. Currently I have a materialized view that contains ~50 million records. When using a regular b-tree index and searching by this numeric column I get a cost of 7 and query results in about 0.8 seconds (with no...

Unable to delete oldest table partition - Oracle 11g

Hi, I'm using the 11g interval partitioning feature in one of my tables. I set it up to create 1 day partitions on a timestamp field and created a job to delete data 3 months old. When I try to delete the oldest partition I get the following error: ORA-14758: Last partition in the range section cannot be dropped I would have thought t...

PHP: Caching ordered integer partition algorithm

Hello! First: The problem's name in Wikipedia is "ordered partition of a set". I have an algorithm which counts possible partitions. To speed it up, I use a cache: function partition($intervalSize, $pieces) { // special case of integer partitions: ordered integer partitions // in Wikipedia it is: ordered partition of a set global $...

Oracle Partitioning

I have 1 table which contain 8 subtables. Each subtable contains another 3 to 5 subtables. Now my question is that is it possible to partition tables by hierarchy? That means i like to keep all the tables info in one table and then create partitioning on them. ...