Python Programming - Rules/Advice for developing enterprise-level software in Python?

Question

I'm a somewhat advanced C++/Java Developer who recently became interested in Python and I enjoy its dynamic typing and efficient coding style very much. I currently use it on my small programming needs like solving programming riddles and scripting, but I'm curious if anyone out there has successfully used Python in an enterprise-quality project? (Preferably using modern programming concepts such as OOP and some type of Design Pattern)

If so, would you please explain why you chose Python (specifically) and give us some of the lessons you learned from this project? (Feel free to compare the use of Python in the project vs Java or etc)

Answer 1

I've been using Python as distributed computing framework in one of the worlds largest banks. It was chosen because:

• It had to be extremely fast for developing and deploying new functionalities;
• It had to be easily integrable with C and C++;
• Some parts of the code were to be written by people whose area of expertise was mathematical modeling, not software development.

Answer 2

I'm using Python for developing a complex insurance underwriting application.

Our application software essentially repackages our actuarial model in a form that companies can subscribe to it. This business is based on our actuaries and their deep thinking. We're not packaging a clever algorithm that's relatively fixed. We're renting our actuarial brains to customers via a web service.

1. The actuaries must be free to make changes as they gain deeper insight into the various factors that lead to claims.

   • Static languages (Java, C++, C#) lead to early lock-in to a data model.

   • Python allows us to have a very flexible data model. They're free to add, change or delete factors or information sources without a lot of development cost and complexity. Duck typing allows us to introduce new pieces without a lot rework.

2. Our software is a service (not a package) so we have an endless integration problem.

   • Static languages need complex mapping components. Often some kind of configurable, XML-driven mapping from customer messages to our ever-changing internal structures.

   • Python allows us to have the mappings as a simple Python class definition that we simply tweak, test and put into production. There are no limitations on this module -- it's first-class Python code.

3. We have to do extensive, long-running proof-of-concept. These involve numerous "what-if" scenarios with different data feeds and customized features.

   • Static languages require a lot of careful planning and thinking to create yet another demo, yet another mapping from yet another customer-supplied file to the current version of our actuarial models.

   • Python requires much less planning. Duck typing (and Django) let us knock out a demo without very much pain. The data mappings are simple python class definitions; our actuarial models are in a fairly constant state of flux.

4. Our business model is subject to a certain amount of negotiation. We have rather complex contracts with information providers; these don't change as often as the actuarial model, but changes here require customization.

   • Static languages bind in assumptions about the contracts, and require fairly complex designs (or workarounds) to handle the brain-farts of the business folks negotiating the deals.

   • In Python, we use an extensive test suite and do a lot of refactoring as the various contract terms and conditions trickle down to us.

   Every week we get a question like "Can we handle a provision like X?" Our standard answer is "Absolutely." Followed by an hour of refactoring to be sure we could handle it if the deal was struck in that form.

5. We're mostly a RESTful web service. Django does a lot of this out of the box. We had to write some extensions because our security model is a bit more strict than the one provided by Django.

   • Static languages don't have to ship source. Don't like the security model? Pay the vendor $$$.

   • Dynamic languages must ship as source. In our case, we spend time reading the source of Django carefully to make sure that our security model fits cleanly with the rest of Django. We don't need HIPAA compliance, but we're building it in anyway.

6. We use web services from information providers. urllib2 does this for us nicely. We can prototype an interface rapidly.

   • With a static language, you have API's, you write, you run, and you hope it worked. The development cycle is Edit, Compile, Build, Run, Crash, Look at Logs; and this is just to spike the interface and be sure we have the protocol, credentials and configuration right.

   • We exercise the interface in interactive Python. Since we're executing it interactively, we can examine the responses immediately. The development cycle is reduced to Run, Edit. We can spike a web services API in an afternoon.