Create a MySQL table from a list of terms in a text file

Question

Hi,

I have a text file with a long list of terms (approx 800) sorted alphabetically in the format:

aword
bword
cword
...

I would like to use this file to create a new MySQL table where each term is a field, all with the property VARCHAR(5).

Would be best if the fields were inserted in the order in which they appear in the file as I also use this file in scripts for content analysis and it would be easier if the (alphabetical) order was maintained when it came time to insert data into this new table.

For those curious, I am a student doing a thesis project in the history of science that involves content analysis. The fields will be used to hold frequency data. There will be another field that will the primary key and link these fields with other data. I am no programmer, but do have 10 years experience running linux and am usually able to figure things out. However, Google has failed me here.

EDIT

So Damoviso pointed out that what I really needed to do was turn the file into a mySQL command. I used awk and uniq to generate the following:

CREATE TABLE keyterms_frq (filename VARCHAR(20), apperception VARCHAR(5), behaviorism VARCHAR(5), behavioristic VARCHAR(5), behaviorists VARCHAR(5), behaviorist VARCHAR(5), behavior VARCHAR(5), behaviour VARCHAR(5), brain VARCHAR(5), conditioned VARCHAR(5), conditioning VARCHAR(5), condition VARCHAR(5), consciousness VARCHAR(5), conscious VARCHAR(5), experienced VARCHAR(5), experiences VARCHAR(5), experience VARCHAR(5), intellect VARCHAR(5), introspections VARCHAR(5), introspection VARCHAR(5), introspectively VARCHAR(5), introspective VARCHAR(5), intuition VARCHAR(5), memoryimage VARCHAR(5), memory VARCHAR(5), mentality VARCHAR(5), mentally VARCHAR(5), mental VARCHAR(5), mind VARCHAR(5), mirrorscript VARCHAR(5), mirrorwriting VARCHAR(5), unconditioned VARCHAR(5));

Which is unfortunately still not working, as it generates the following:

ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'condition VARCHAR(5), consciousness VARCHAR(5), conscious VARCHAR(5), experience' at line 1

I initially thought there was a limit to how many fields I could generate, but that doesn't seem to be the issue

Thanks for you help, Frank

Answer 1

I'm not sure that I entirely understand what your question is, but I do have a thought:

If you're going to be calculating frequency data, you could write more efficient content analysis algorithms if the words were sorted before they were inserted into the table. This would reduce CPU strain on the database server, and your frequency calculator would also be more efficient.

Maybe you could elaborate on your question a little for us?

Answer 2

So basically what I think you are asking for is this: Read the file into a program.
have the program take each line and generate a string listing all the words.
concatenate a CREATE TABLE statement with your string that holds your field names to create the SQL statement.
send that SQL command to your sql server.
I am still too new at this to give you a good implementation of this idea, but hopefully if this matches what you are looking for, I will give someone an idea.

Answer 3

Personally (and I know this probably won't be popular), but I tend to use Excel and something like Textpad or Notepad++ (or even vanilla Notepad) for once-off database inserts and table creations like this.

Import the text file so you have each word in a new row, sort them as required, and build the SQL commmand around it by inserting columns and a row at the top and bottom.

In this case, I'd suggest:

1. Insert a row at the top with the following contents in cell A1:
   CREATE TABLE MyTable (Id INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
2. In each cell in column B that's next to a word, put the following contents:
   VARCHAR(5),
3. Remove the comma for the last one and put a ");" in the cell below.
4. Save as a tab-delimited file, then using the text editor of your choice, replace tabs with spaces.

You'll be left with a SQL file you can run against the database.

Yes, it's messy and not very 1337, but it's effective.

Answer 4

Hey Frank, I just wrote this on your facebook page, but here it is again:

"condition" is a reserved word in MySQL so you are not allowed to name any column with this word. Check the list here (for MySQL 5.1 at least):

http://dev.mysql.com/doc/refman/5.1/en/reserved-words.html