Java File access - Performance considerations

Question

All,

Considering my earlier query Stackoverflow-Post, here is what I am currently doing. This is a kind of flowchart:

1 > Add record to file:
a. Get the details for new record
b. Retrieve all the records from the file.
c. Compare each record retrieved with the new record details. If match found, do not add to the file, else add the new record.

2 > Search for a record in file.
a. Get the details for the record to be searched (Here only the credit card number.)
b. Retrieve all the records from the file.
c. Compare each record retrieved with the record searched for. If match found, display the details.

3 > Delete a record from the file.
a. Get the details for the record to be deleted (Here only the credit card number.)
b. Retrieve all the records from the file.
c. Compare each record retrieved with the record searched for. If match found, delete the entry from the file.

As can be seen, I am doing an IO every time for a operation. Is there any way I can optimize or improve performance for my code?

The API I am using for Add record is FileWriter , for search operation it is BufferedReader and for deleting a record is BufferedReader and PrintWriter

Answer 1

Use a database.

Use an index file for the credit card number. The index file may be structured like a hash or a tree and contains a pointer to the offset of your data file. The lookup in the index file is fast, since you have a single key (credit card number).

Calculate hashes for your credit card numbers and store them in the index. That way, you can quickly identify whether an entry exists or not.

Answer 2

I would also suggest you use a database but if you must use a file then an option i can suggest is Java serialization.Maintain a HashMap with Key as the credit-card number and the rest of the information wrapped in an object as the value.Each time a new records are added add the information to the hashmap and serialize the data and delete the old file.Same you can do for deletion of records and for searching its very easy since the information is in a hashmap , all you have to do is check if the key is present or not and another advantage is you don't have to bother about the format for writing into a file and then parsing it again.

If the number of records are large a problem that can probably occur is that you have to load the large file into memory.A solution is you can maintain a limit for the number of records per file and you can maintain a look-up with a navigable map(check this link for example).A range of credit card number's can be stored in a certain file.The key for the look-up can be the starting range of the credit-card and the the value can the file name.