duplicate-data

Vim duplicate selection

If i have something selected in VIM in visual mode .... how can i duplicate that selection and place it below or above the selection. ...

Duplicate WCF service calls being saved in database

We have a C# WCF service (3.0) that takes in data and then make another web service call to a third party with that same data. Before the third party call, the entry data is saved as a record in a database, and then updated with response data from the thrid party web service. We have start doing some Load/Stress testing, and noticed th...

Efficient checking of possible duplicate entities

I have a requirement to produce a list of possible duplicates before a user saves an entity to the database and warn them of the possible duplicates. There are 7 criteria on which we should check the for duplicates and if at least 3 match we should flag this up to the user. The criteria will all match on ID, so there is no fuzzy string ...

Finding Rows with Duplicate Data in MS Access

I have a large table that includes a last name and date of birth among other things. I need to write a query that returns only those rows where the last names and dates of birth are equivalent to those in another row of the table. This a table that contains information about people who have made an application. I want to identify thos...

Using Perl to cleanup a filesystem with one or more duplicates

I have two disks, one an ad-hoc backup disk, which is a mess with duplicates everywhere and another disk in my laptop which is an equal mess. I need to backup unique files and delete duplicates. So, I need to do the following: Find all non-zero size files Calculate the MD5 digest of all files Find files with duplicate file names Separa...

data deduplication algorithms

I'd like to find data deduplication algorithms, mostly to find duplicate files. Looks like the first step is to identify the files with the same timestamps, sizes and file names. I can do the md5 chechsum on those files and compare. Addition to that it is possible to compare the contents of files. What else should I watch for? ...

Map implementation with duplicate keys

I want to have Map with duplicate keys, I know there are many Map implementations(eclipse shows me about 50), so I bet there must be one that allows this. I know its easy to write your own Map that does this, but i would rather use some existing solution. Maybe something in commons-collections or google-collections? ...

Is there a standard way to duplicate a row in a database table?

I want to duplicate a row, not the keys of course, without explicity using the field names. Is there a SQL way or do I have to enumerate the field names through code? I don't want to explicity use field names because I want to minimize code and db dependencies. I am going to use it in the Ms Access 2003. I mention it in case that no s...

Remove dupes/sort from a Array of Associative Arrays in PHP

I have a array of associative arrays aa[] = ('Tires'=>100, 'Oil'=>10, 'Spark Plugs'=>4 ); aa[] = ('Tires'=>454, 'Oil'=>43, 'Spark Plugs'=>3 ); aa[] = ('Tires'=>34, 'Oil'=>55, 'Spark Plugs'=>44 ); aa[] = ('Tires'=>454, 'Oil'=>43, 'Spark Plugs'=>45 ); aa[] = ('Tires'=>34, 'Oil'=>55, 'Spark Plugs'=>433 ); aa[] = ('Tires'=>23, 'Oil'=>33...

Eliminating duplicate same-level elements in XSLT

I'm looking to display a list of same-level node names, without duplicates. Let's say I have <a> <b> <c /> <d /> <d /> </b> <b> <e /> <c /> <f /> </b> </a> I'd want c,d,e,f to be displayed. I've found several solutions to a similar problem, eliminating duplicate siblings...

Tips for deduping a list based on priority column

I have a set of rows that contain duplicate entries because the data originates from multiples sources. I also have a separate reference table that indicates the priority of those data sources. Does anyone have good tips for the most effective t-SQL to deduplicate this list? Basically I have: SELECT a.*, b.priority FROM tableA as a ...

counting duplicate entries in Data base

I want to count the amount of times that a zip code is entered into a database. I'm not sure if I am using the right function or not. Also eventually I need to separate the zip codes by the year they were entered int the data base. I know how to separate the years-I think. What I really need help on is counting duplicate entries Here is ...

C# Remove Duplicate Lines From Text File?

Does anyone have any sample code of how to do this? ...

Duplicates in a QIF File?

Does anyone have a good way of deleting duplicate transactions (same date, amount, biller, etc) in a QIF file? I looked at PERL's Finance:QIF, but it appear to have delete a record function. Alternatively, does someone have a good QIF --> CSV converter? Although I am looking at a PERL solution, I am open to other ideas. Thanks in...

what are the fast algorithms to find duplicate elements in a collection and group them?

Say you have a collection of elements, how can you pick out those with duplicates and put them into each group with least amount of comparison? preferably in C++, but algorithm is more important than the language. For Example given {E1,E2,E3,E4,E4,E2,E6,E4,E3}, I wish to extract out {E2,E2}, {E3,E3}, {E4,E4,E4}. what data structure and...

SharePoint: How to have a unique constraint in a list?

In SQL its possible to have fields that cannot contain duplicate data. How is this possible in SharePoint? Lets say you have a list containing values like ORANGES APPLES PEARS How do you ensure that the user cannot enter ORANGES APPLES PEARS ORANGES Thanks ...

PHP - Highlight duplicate values in a multidimensional array

I'm outputting a list of purchases, and I want to automatically highlight the presence of duplicate orders. Here's what the array looks like. The first two orders are duplicate orders place by mistake. You'll notice that the orderid for each is different, while the email and userid remain the same. So the duplication will need to match ...

Find duplicate content using MySQL and PHP

Hello, I am facing a problem on developing my web app, here is the description: This webapp (still in alpha) is based on user generated content (usually short articles although their length can become quite large, about one quarter of screen), every user submits at least 10 of these articles, so the number should grow pretty fast. By na...

Deduplicating an array of keywords (but not based on EXACT match)

Hi folks! I have a list of a few thousand terms. There is significant overlap in those terms, but in different forms. For example (ruby, a_ruby), (triathlon, triathlete, triathletes), (nonprofit, non_profit, non_profits). Most of these have significant number of character overlap, but not exactly in the same form. For example, (nonprof...

How to delete partial duplicate lines with AWK?

I have files with these kind of duplicate lines, where only the last field is different: OST,0202000070,01-AUG-09,002735,6,0,0202000068,4520688,-1,0,0,0,0,0,55 ONE,0208076826,01-AUG-09,002332,316,3481.055935,0204330827,29150,200,0,0,0,0,0,5 ONE,0208076826,01-AUG-09,002332,316,3481.055935,0204330827,29150,200,0,0,0,0,0,55 OST,0202000068,...