tags:

views:

604

answers:

6

So, I'm using jdbc to talk to a MySQL DB. For many a table and for many queries/views, I have created one class which encapsulates one row of the table or the query/table result. Accesses to the DB return one object of such a class (when I know exactly that there's only one matching row) or a Vector of such objects.

Each class features a factory method that builds an object from a row of a ResultSet. A lot of ResultSet.getXXX() methods are needed, as is finicky bookkeeping about which value is in which column, especially after changes to the table/query/view layout.

Creating and maintaining these objects is a boring, work-intensive and mind-numbing task. In other words, the sort of task that is done by a tool. It should read SQL (the MySQL variant, alas) and generate Java code. Or, at least, give me a representation (XML? DOM?) of the table/query/view, allowing me to do the java code generation myself.

Can you name this tool?

+5  A: 

I'm a little confused about your questions. Why don't you use an Object-relational-mapping framework like Hibernate?

I used to have the same problem having to read and write a lot of SQL directly. Eventually I started writing newer projects with Hibernate and haven't looked back. The system takes care for me of building the actual tables and running the SQL in the background, and I can mostly work with the java objects.

Uri
Well, I don't want to give up control over how the tables are built. That's what irks me about Hibernate (or Linq, for that matter): I need to be able to taylor the DB schema to the expected load characteristics at run time.
doppelfish
Hibernate also have a place where you can drop SQL directly but it is not the right place to do it.
OscarRyz
Also, with Hb you can control how the tables are created and you can run later a tool against existing schema and the have the object model created automatically.
OscarRyz
What do you need to do to your tables that hibernate doesn't let you do? I've found the configurable tweaks I can achieve with annotations to meet most of my needs (e.g., column naming).
Uri
+1 for Hibernate. In short, ORM respects your time. :-)You can do both forward engineering (generate DDL script from Hibernate entity classes - do this when starting from scratch) and reverse engineering (generate Hibernate entity classes from an existing DB - use this with legacy databases).
Shonzilla
I've never tried reverse engineering with hibernate; how does that work? How does it generate new interfaces?
Uri
It's not Hibernate, but a plugin ( eclipse I think ) you specify the database connection setting, go through some wizards to select the tables, and it generates the hibernate.hbm.cfg.xml ( or what ever name it is I don't remember ) the config for each class and from the contraints infer the relation
OscarRyz
Pretty useful when it comes to a legacy database. You run it, in have some 50+ classes all of them linked and ready to use. :)
OscarRyz
Oh ok, something does generate the classes for you. that's really cool. I love Hibernate. I hate all the time I spent hand coding my DB work, especially before Java supported generics.
Uri
OK, if Hibernate can do this for me, it's an option I'll consider.
doppelfish
A: 

I created a mini-framework like that years ago, but it was for prototyping and not for production.

The idea follows and it is VERY very simple to do. The tradeoff is the cost of using reflection. Although Hibernate and others ORM tools pay this cost also.

The idea is very simple.

  • You have a Dao class where you execute the query.

  • Read the ResultSet Metadata and there you can grab the table name, fields, types etc.

  • Find in the class path a Class that matches the table name and / or have the same number/types of fields.

  • Set the values using reflection.

  • Return this object and cast it in the other side and you're done.

It might seem absurd to find the class at runtime. And may look too risky too, because the query may change or the table structure may change. But think about it. When that happens, you have to update your mappings anyway to match the new structure. So instead you just update the matching class and live happy with that.

I'm not aware on how does ORM tools work to reduce reflection call cost ( because the mapping the only thing it does is help you to find the matching class ) In my version the lookup among about 30,000 classes ( I added jars from other places to test it ) took only .30 ms or something like that. I saved in cache that class and the second time I didn't have to make the lookup.

If you're interested ( and still reading ) I'll try to find the library in my old PC.

At the end my code was something like this:

 Employee e = ( Employee ) MagicDataSource.find( "select * from employee where id = 1 ");

or

Employee[] emps = ( Employee[] ) MagicDataSource.findAll("select * from employee ");

Inside it was like:

Object[] findAll( String query ) {
     ResultSet rs = getConnection().prepareStatemet( query ).executeQuery();
     ResultSetMetaData md = rs.getMetadata();
     String tableName = md.getTableName();

     String clazz = findClass( toCamelCase( tableName ) ); // search in a list where all the class names where loaded.
     Class.forName( clazz );

     while( rs.next() ) {

         for each attribute etc. etc. 
             setter...
         end

         result.append( object );
     }

    return result.toArray();
 }

If anyone knows how ORM tools deal with reflection cost please let me know. The code I have read from open source projects don't event attempt to do anything about it.

At the end it let me create quick small programs for system monitoring or stuff like that. I don't do that job anymore and that lib is now in oblivion.

OscarRyz
Well, I *think* I already have code availlable that goes through the meta data of a ResultSet. I'll spare You the trouble of digging up your code and use my own for starters. Thanks for pointing me that way.
doppelfish
+2  A: 

Hi,

If you are looking for a simple framework to help with the drudge work in writing sql, I would recommend ibatis sql maps. This framework basically does exactly what you want.

Hibernate is also a good option, but it seems a bit oversized for a simple problem like yours.

You might also have a look at the spring framework. This aims to create a simple environment for writing java application and has a very usable sql abstraction as well. But be careful with spring, you might start to like the framework and spend too many happy hours with it 8)

As to your concern with reflection. Java has no major problems anymore with performance overhead of reflection (at least since Version 1.4 and with O/R mapping tools).

In my experience, it is better to care about well written and easily understandable code, than caring about some performance overhead this might perhaps cost, that is only theoretical.

In most cases performance problems will not show up, where you expect them and can only be identified with measurement tools used on your code after it has been written. The most common problems with performance are I/O related or are based on some error in your own code (i.e. massively creating new instances of classes or loops with millions of runs, that are not necessary...) and not in the jdk itself.

bombadil
Totally agree with you. The speed of an application is concerned more with the amount of that in the wire than the operations performed in RAM. So I guess that the mini framework of mine was not that bad after all.
OscarRyz
A short look at the Java tutorial of ibatis makes me think that that's what I'm looking for.
doppelfish
A: 

Apart from the ORMs...

If you're using the rs.getString and rs.getInt routines, then you can certainly ease your maintenance burden if you rely on named columns rather than numbered columns.

Specifically rs.getInt("id") rather than rs.getInt(1), for example.

It's been rare that I've had an actual column change data type, so future SQL maintenance is little more than adding the new columns that were done to the table, and those can be simply tacked on to the end of your monster bind list in each of you little DAO objects.

Next, you then take that idiom of using column names, and you extend it to a plan of using consistent names, and, at the same, time, "unique" names. The intent there is that each column in your database has a unique name associated with it. In theory it can be as simple (albeit verbose) as tablename_columnname, thus if you have a "member" table, the column name is "member_id" for the id column.

What does this buy you?

It buys you being able to use your generic DAOs on any "valid" result set.

A "valid" result set is a result set with the columns named using your unique naming spec.

So, you get "select id member_id, name member_name from member where id = 1".

Why would you want to do that? Why go to that bother?

Because then your joins become trivial.

PreparedStatement = con.prepareStatement("select m.id member_id, m.name member_name, p.id post_id, p.date post_date, p.subject post_subject from member m, post p where m.id = p.member_id and m.id = 123");
ResultSet rs = ps.executeQuery();
Member m = null;
Post p = null;
while(rs.next()) {
    if (m == null) {
        m = MemberDAO.createFromResultSet(rs);
    }
    p = PostDAO.createFromResultSet(rs);
    m.addPost(p);
}

See, here the binding logic doesn't care about the result set contents, since it's only interested in columns it cares about.

In your DAOs, you make them slightly clever about the ResultSet. Turns out if you do 'rs.getInt("member_id")' and member_id doesn't happen to actually BE in the result set, you'll get a SQLException.

But with a little work, using ResultSetMetaData, you can do a quick pre-check (by fetching all of the column names up front), then rather than calling "rs.getInt" you can call "baseDAO.getInt" which handles those details for you so as not to get the exception.

The beauty here is that once you do that, you can fetch incomplete DAOs easily.

PreparedStatement = con.prepareStatement("select m.id member_id from member m where m.id = 123");
ResultSet rs = ps.executeQuery();
Member m = null;
if (rs.next()) {
    m = MemberDAO.createFromResultSet(rs);
}

Finally, it's really (really) a trivial bit of scripting (using, say, AWK) that can take the properties of a bean and convert it into a proper blob of binding code for an initial DAO. A similar script can readily take a SQL table statement and convert it in to a Java Bean (at least the base members) that then your IDE converts in to a flurry of getters/setters.

By centralizing the binding code in to the DAO, maintenance is really hardly anything at all, since it's changed in one place. Using partial binding, you can abuse them mercilessly.

PreparedStatement = con.prepareStatement("select m.name member_name, max(p.date) post_date from member m, post p where post.member_id = m.id and m.id = 123");
ResultSet rs = ps.executeQuery();
Member m = null;
Post p = null;
if (rs.next()) {
    m = MemberDAO.createFromResultSet(rs);
    p = MemberDAO.craateFromResultSet(rs);
}
System.out.println(m.getName() + " latest post was on " + p.getDate());

Your burden moving forward is mostly writing the SQL, but even that's not horrible. There's not much difference between writing SQL and EQL. Mind, is does kind of suck having to write a select statement with a zillion columns in it, since you can't (and shouldn't anyway) use "select * from ..." (select * always (ALWAYS) leads to trouble, IME).

But those are just the reality. I have found,though, that (unless you're doing reporting), that problem simply doesn't happen a lot. It happens at least once for most every table, but it doesn't happen over and over and over. And, naturally, once you have it once, you can either "cut and paste" your way to glory, or refactor it (i.e. sql = "select " + MemberDAO.getAllColumns() + ", " + PostDAO.getAllColumns() + " from member m, post p").

Now, I like JPA and ORMs, I find them useful, but I also find them a PITA. There is a definite love/hate relationship going on there. And when things are going smooth, boy, is it smooth. But when it gets rocky -- hoo boy. Then it can get ugly. As a whole, however, I do recommend them.

But if you're looking for a "lightweight" non-framework, this technique is useful, practical, low overhead, and gives you a lot of control over your queries. There's simply no black magic or dark matter between your queries and your DB, and when things don't work, it's not some arcane misunderstanding of the framework or edge case bug condition in someone elses 100K lines of code, but rather, odds are, a bug in your SQL -- where it belongs.

Will Hartung
A: 

I have been working on this very project. It takes a mysql database and reverse engineeers the java classes. I have been working on this for work, and plan on open sourcing the project. If you are interested, you can contact me for more details.

[email protected]

Milhous
A: 

Edit: Nevermind. While searching for a solution to my own problem, I forgot to check the date on this thing. Sorry. You can ignore the following.

@millermj - Are you doing that for fun, or because there's a need? Just curious, because that sounds exactly like what Java IDEs like Eclipse and NetBeans already provide (using the Java Persistence API) with New->JPA->Entity Classes from Tables functionality.

I could be missing the point, but if someone just needs classes that match their tables and are persistable, the JPA plus some IDE "magic" might be just enough.

TomR