Your position seems eminently reasonable, that is to say, you are able to explain reasons behind what you are telling people.
The reason they can not do something faster, and perhaps better, with their favorite language, is because you would go crazy managing it. Except, what if you didn't?
A sane management strategy might be (might be!) to generally say no to these things, but every now and then, say yes. And try to learn.
Here is my suggestion: An internal tool that is well written, built in a language you don't normally use, just might open you up to further use of that tool. Python and Django, Ruby and Rails, and the list goes on and on. So you're a Java guy. Good for you.
But your development team, organization can probably outperform expectations when you leave behind arbitrary limits, and leave the limits to what really has a 100% solid business and financial case behind it.
Fear of learning a new programming language is the difference between the mediocre, and the good people in your organization. Artificial limits on their productivity, will in time, cause you to lose those squeaky-wheels. They will go elsewhere. Your question should be "if I tick off these guys and they leave, did I lose something good, something worth keeping, and do the people who I have now, who all write everything in Java, have the capability to build me my next big product, or solve my next big problem, when I need it solved? Or are they all plodding egalitarian members of a groupthink-team, and not one of them capable of an original thought?"
I submit to be beaten-up the following proposal:
i. Say no to Visual Basic, and C# on the grounds that Java and the JVM offer a superset of the benefits of the .net managed code environment. Moving to .net means keeping around a second virtual machine technology, runtime and competing frameworks, while losing Java's portability and open source JVM platform, a net loss to your organization.
ii. Say yes to Ruby and Python, because they are on the list of languages that startups are using to change the world. They are stars in ascendancy.
iii. Say no when you have a clear business case to do so, and a provisional yes when six months or less will give you a better answer on what will provide the optimal results and capabilities to your software team. Is there anything you can't do in Java? No. Is there anything that you can't do faster, and possibly better, in Python and Ruby? Some of your people are saying yes? Give them a chance to prove it.
iv. Also, say no to extensions and improvements to your main Java products using secondary languages. If used, Python and Ruby are for standalone internal tools, not as "first class languages to be used instead of Java to extend your primary existing Java applications", and not as a destination for a total rewrite, and not as a way to abandon Java.