Mixed language web dev environments

Question

I have inherited a broad, ill-designed web portfolio at my job. Most pages are written in Perl as most of the data ingested, processed, and displayed on the site comes in the form of flat-files which then have to be meticulously regexed and databased in our MySQL and Oracle databases.

As the first IT-trained manager of this environment, I am taking it upon myself to scrub the websites and lay down some structure to the development process. One of the choices I have been given is to choose whether or not to continue in Perl. There is substantial in-house talent for Java and PHP is rather easy to learn. I have considered taking the reins off the developers and letting them choose whatever language they want to use for their pages but that sounds like it might be trouble if the guy who chose PHP gets hit by a bus and no one else can fix it.

As the years go by hiring Perl programmers becomes more and more difficult and the complexity of maintaining legacy Perl code from previous developers whose main focus may have been just getting a page up and running is becoming very resource-consuming. Another, previous (non-IT) manager was more focused in on quick turnaround and immediate gratification of pages rather than ensuring that it was done right the first time (he has since been promoted outside our branch).

The production server is solaris. MySQL has most of our data but new projects have begun using Oracle more and more (for GIS data). Web servers are universally Apache. We live in an intranet disconnected from the regular internet. Our development is conducted in an Agile, iterative manner.

Whatever language is chosen to push forward into, there are resources to have the existing development staff re-trained. No matter what, the data that comes into our environment will have to be regexed to death so perl isn't going away anytime soon. My question to the community is, what are the pros and cons of the following languages for the above-defined web dev environment: Perl, PHP, Java, Python, and --insert your favorite language here--. If you had it to do all over again, which language set-up would you have chosen?

Edits and Clarifications: Let me clarify a bit on my original post. I'm not throwing everything away. I've been given the opportunity to adjust the course of the ship to what I believe is a better heading. Even if I chose a new language, the perl code will be around for some time to come.

Hypothetically speaking, if I chose Assembly as my new language (haha) I would have to get the old developers up to speed probably by sending them to some basic assembly classes. New pages/projects would be in the new language and the old pages/projects would have to play nice with the new pages/projects. Some might one day be rewritten into the new language, some may never be changed.

What will likely always be in Perl will be the parsing scripts we wrote years ago to sift through and database information from the flat files. But that's okay because they don't interface with the webpages, they interface with the database.

Thank you all for your input, it has been very very helpful thus far.

Answer 1

I think choosing a single language is key and if your database is primarily MySQL, then PHP seems like the obvious choice. It naturally works with your database, it's open source and there is tons of documentation, source code, doesn't require compiling, etc.

People come and go through positions and any website will evolve over time. If you have the ability to set some guidelines and rules, I would choose something that is forgiving, common-place, and easy(er) to learn.

I'd also suggest writing it down so people in the future don't re-invent the wheel.

Answer 2

If regex is important I would choose a language with good support.

If you would use java, you will not be able to just copy paste your regex code from the perl code because the slashes have to be escaped. So I would vote against java.

I'm not familiar with php enough to know its regex features, but given your choices I would go for python. You can create cleaner code in python.

Would ruby also be an option? It also has good perl like regex support and rails supports agile web development out of the box.

Answer 3

first off let me point out that MySQL's spatial extensions work with GIS.

Second, if you have a bunch of Perl programmers that will need tow ork on the new sites then your best bet is to choose something they won't have too much trouble understanding. The obvious "something" there is PHP. When I learnt PHP I'd done some Perl years ago and picked up PHP in no time at all.

Switching to something like Java or .Net (or even Ruby on Rails) would be a far more dramatic shift in design.

Plus with Apache servers you already have your environment set up and you can probably stage any development as a mix of Perl and PHP reasonably easily.

As to the last part of your question: what would you do it in if you could do it over again? To me that's a seprate and basically irrelevant question. The fact is you're not rewinding and starting over, you're just... starting over. So legacy support, transition, skills development and all those other issues are far more important than any hypothetical question of what you'd choose in a perfect world if all other things were equal.

Love it or hate it, PHP is popular and is going nowhere anytime soon. Finding skiled people to do it is not too difficult (well, the dificulty is filtering them out from the self-taught cowboy script jockeys who think they can code but can't) and it's not a far step from Web-based Perl.

Answer 4

It seems that your problem is more legacy code and informal development methodology than the language per se. So if you already have Perl developers on staff, why not start modernizing your methods and your code base, instead of switching to a new language, and creating an heterogeneous code base.

Modern Perl offers a lot in terms of good practices and powerful tools: testing is emphasized, with the Test::* modules and WWW::Mechanize, data base interaction can be done through plain DBI, but also using ORM modules like the excellent DBIx::Class, OO with Moose is now on par with more modern languages, mod_perl gives you access to a lot of power within Apache. There are also quite a few MVC frameworks for Perl. One that's getting a lot of attention is Catalyst.

Invest in a few copies of Perl Best Practices, bring in a proper trainer for a few classes on modern development methods, and start changing the culture of your group.

And if you have trouble finding developers that are already proficient in Perl, you can always hire good PHP people and train them, that shouldn't be too difficult. At least their willingness to learn a new language would be a good sign of their flexibility and will to improve.

It is always tempting to blame the state of your code on the language its written in, but in your case I am not sure that is the case. Lots of big companies seem to have no problem managing huge code bases in Perl, the list is long but the main Web companies are all there, along with a number of financial institutions.

Answer 5

If you're not married to LAMP and are willing or able to jump platforms, I'd suggest you at least investigate ASP.NET and C#. The biggest benefit of the platform (aside from the numerous technical superiorities, but I'm biased on that and YMMV) is the large number of developers available for it. Seriously, it's, like, more than triple the availability of any other language.

Now, granted, a lot of them suck. No offense to those of you that do suck and happen to be reading this, it's just a fact; there are good and bad developers, and the sucky ones tend to gravitate to the hottest language du jour, and that's C#. That said, you tend to get a certain level of suck in every language and the vary nature of some of those languages (particularly PHP and Perl -- it takes a very disciplined programmer to build maintainable code in either of those languages) can even make the likelihood of suck even greater.

Despite the large amount of suck in the talent pool, there is also a huge amount of real talent developing on the platform. I mean absolute coding gods whose feet we aren't worthy to kiss. They're out there, and the large pool means that you have a much greater chance of finding superstars for your team. And I guarantee that most of those guys started with or are at least more than passingly familiar with the LAMP stack, so you'll have a good chance of building a solid migration.

---

Now your situation may not permit this, and I understand fully. In that case, I suggest you 1) move away from Solaris if you can -- easily the worst *nix implementation I've ever dealt with -- and 2) go with anything other than PHP or Perl.

I would personally recommend Python for your processing scripts, but I've never been very happy with the web platforms available for Python programming; admittedly, I've been out of the Python world for several years now, so there may be newer technology out there that you can use. Maybe Twisted has improved, but I'm pretty sure Zope is dead.

So for your web... I dunno. One of the reasons I got into C# and ASP.NET was because of the problems I had with Zope and PHP, and I was never very happy with the way any sort of Java web implementation was organized. I've never used it, but you might look into Ruby on Rails. It's popular these days at least, so that might help your hiring.

---

And that brings me to hiring. Regardless of which platform you end up with, I suggest you consider augmenting your staff.

Given what you've mentioned about your ill-designed system, I would highly suggest that you take a good, hard look at your existing staff and do some pruning. Based on your description you're likely to have some suck there, but you might have a few hidden rockstars. Look for a desire for improvement, interest in theory and system structure, and language neutrality; if they hate all languages but Perl and are unwilling to even investigate anything else, if they are just there for the paycheck, they stay on your legacy team to maintain the existing code-base until you can roll out your new platform. If they still haven't improved their attitudes once the new platform rolls out, lay them off.

If you find a rockstar or one with potential on your staff, they go on the new platform team. They get a voice as to the architecture (and even the platform itself) and training. Hire more to fill out the rest of the new platform team. If you don't already have them (I can't tell from your post) hire a data services team dedicated to your database. Separate your concerns and concentrate your forces. Get Enterprisey -- it's not always a bad thing. :)

Whew! This post has gotten long and devolved into Management, which I'm sure you're already qualified for. I'll shut up about that. To summarize:

Consider C#/ASP.NET. If not, Python / Ruby on Rails, maybe Tomcat. And focus on your talent! That's infinitely more important than platform. You can deliver quality or crap regardless of platform, and your developers are the ones who will determine which you end up with.

Edit: Clarifications after original post edit

Ok, so you can't jump platforms. In that case, re-read my second and third sections. Granted, I've been out of that portion of the industry for a few years now, but I've used both Perl and Python for file processing and I've found Python the better. I stand by that suggestion. Again, I've no preference on the web platform in a LAMP stack, aside from suggesting that you find a platform that has strong templating and separation of concerns -- in other words anything other than PHP. A Rails-like framework if not Ruby on Rails itself.

But again, I suggest that more important than your choice of platforms is your focus on your team. Sift the chaff from the grain. Institute best practices. Group by specialty. Build a solid team.

Answer 6

I would bring in someone who is very good at Perl, to at least look at the current design. They would be able to tell you how bad the Perl code really is, and what needs to be done to get it into good shape.

At that point I would start considering my options. If the Perl code is salvageable, well than great, hire someone proficient in Perl. Also train some of your existing staff to help on the existing code-base. If you don't have someone proficient in Perl in charge of the Perl code, your code-base may become even worse than it already is.

Only if it was in terrible shape, would I consider abandoning it for another language. What that language is, that is something your going to have to think about that yourself.

p.s. I'm a bit biased, I prefer Perl

Answer 7

If your developers are any good, they'll be able to handle anything thrown at them. Deciding what language to use is quite a tricky strategic position, but I recommend you think very carefully before introducing any MORE (i.e. don't).

Unless of course, there is something that you absolutely cannot do (or cannot do sanely) with what you've got.