tags:

views:

112

answers:

0
Q: 

ETL framework for loading data into Rails app

I need to load data for my Rails application from multiple providers (REST/SOAP based XML feeds) into the database on a recurring basis. I have written a set of Rake tasks which are kicked off by whenever-generated cron jobs. Each task hits the partner feed endpoint, parses the feed and loads it into the database.

Instead of writing Rake tasks, should I use an ETL framework like ActiveWarehouse (http://activewarehouse.rubyforge.org/etl/) instead? Any suggestions on the best way to do this in Rails?

related questions

What language do you use for Postgresql triggers and stored procedures?
Are Multiple DataContext classes ever appropriate?
Which tools do people use to create Data Dictionaries?
Any experiences with Protocol Buffers?
Mechanisms for tracking DB schema changes
How big can a MySQL database get before performance starts to degrade.
How do I index a database field
How does database indexing work?
How do I connect to a database and loop over a recordset in C#?
Editing database records by multiple users
Object Oriented vs Relational Databases
VFP .NET OLEdb provider does not work in Win 64-Bits. Help
Embedded Database for .net that can run off a network
Connect PHP to an AS/400
Swap unique indexed column values in database.
cx_Oracle - what is the best way to iterate over a result set?
cx_Oracle - How do I access Oracle from Python?
.NET Migrations Engine
Is there a version control system for database structure changes?
SQLite and XSD
How do I version my MS SQL database in SVN?
XSD DataSets and ignoring foreign keys
Flat File Databases in PHP
Throw Error In MySQL Trigger
Binary Data in MySQL