The pros and cons of "Shadow IT" in software development

Question

Recently we’ve seen the emergence of so-called “Shadow IT” within many organisations. If you’re not already familiar with the term, it refers to those who manage to dodge the usual IT governance by means such as using thumb drives to share files or “unapproved” software products to achieve business tasks. Shadow IT can emerge from within technology groups but in many cases is sourced from non-tech areas such as the marketing or sales department.

What I’m really interested in is examples you have of Shadow IT within software development. Products like Excel and Access are often the culprits as their commonality means they’re easily accessible to the broader organisation. In many cases this is driven by someone who has just enough knowledge to make the software perform a business function but not quite enough to be aware of all the usual considerations required when building software for an enterprise.

What sort of cases of Shadow IT have you witnessed in the software development space? What processes have you seen unofficially addressed by this practice and just how important have these tools become? An example would be the use of a single Access database on a folder share becoming common practice for tracking promotions across the marketing department. Remember this cuts both ways; it can be extremely risky (lack of security, disaster recovery, etc) but it can result in innovation from a totally unexpected source.

Answer 1

Well, I suppose these things are everywhere. Not a big deal if it not threatens the company operation in any way.

Answer 2

At the end of the day, the primary driver for most businesses is results i.e. making money. If the business sees that it can achieve the desired outputs necessary for the operation without spending thousands on software but through "shadow IT", then I can only see it being encouraged. I feel that that it is part of our job as developers to point out the pitfalls in operating in this fashion.

The pros of "shadow IT" could be

• cost - less expensive
• whilst the people writing the software may not be software experts, they are likely to be domain experts and have an intrinsic knowledge of how a piece of software should function.
• depending on how the IT is organized, "shadow IT" may be able to respond faster to changes and business needs than the core IT can.

And the cons

• software produced may not be well designed to be extensible, handle errors correctly a d all other aspects that come from experience in software development.
• the software may not be well supported or, due to the way in which it has been produced, there may be no support at all.

Answer 3

Most software developers have "unapproved" software on their computers. Just expect it. I'm not sure how much I have, but I'm sure I have dozens, if not hundreds of utilities that corp. IT has never even heard of on my work laptop.

Answer 4

Over time, the average person is becoming more IT savvy. Younger marketeers and finance people know that Excel and Access make them vastly more efficient. Working without them would make them feel handicapped.

I expect this trend to continue, and Corporate IT becoming more of an enabling organization. Where you make available data, help users troubleshoot their workflow, and limit them to a specific compartment for security.

What was called software development 10 years ago, will be everyman's tool 10 years from now!

Answer 5

There is no such thing. There are dinosaurs, and there are people who need to get work done. If something like 'Shadow IT' happens, it is because 'Official IT' is not doing its job.

Software developers have hundreds of little and not so little applications they need to get their work done. The IT governance organisation should learn how to handle tens of updates a day, and switch to releasing daily (and patching a few times a day). Development has learned how to do that, they are next.

Answer 6

Ya it's a big problem where I work. Architects and DBA's try to make a centralized system but these little "Shadow IT" departments make these small apps that have their own security or duplicated data... Personally, if I was the head of IT I would fire anyone who started such a project without IT support. Kinda harsh but it's important to keep the system healthy.

Answer 7

The advantage is that users get exactly what they want and need, when they want and need it. Getting a request through a largish IT shop is a trying experience for a user. IT rarely has the business knowledge to let them give the business owners exactly what they are asking for, and when requests are denied or requirements amended, an explanation in plain English (or whatever language) is rarely forthcoming.

The disadvantages outweigh the benefits. Societe Generale lost billions due in part to "Shadow IT". It can cause support nightmares when an Access application, for example, becomes essential and outgrows the capabilities of the person who created it, or that person leaves. Even a poorly written Crystal Report can become so popular and widely used that it starts to drag down the database it is accessing when reporting times comes around. And if the person who wrote that report did not fully understand relational databases, it could produce bad data in some situations; data that causes bad business decisions to be made. Using a commercial (outsourced) application guarantees that the users will not get exactly what they want; there will always be compromises, and no explanation of why they were made.

The previous poster was right. Shadow IT exists because IT does not do its job well enough. There is not enough business knowledge, not enough responsiveness, and especially not enough communication. These things are why "Shadow IT" exists. The business owners paid for the machines, the admins, the dbas, and the programmers. It frustrates them when IT loses sight of that.

Answer 8

Why does IT assume they should own and control all technology in the business?

The very fact that we have a name for technology that IT does not control (Shadow IT) suggests that we'd like IT to have control over all technology in an organization.

The only real reason I can think of for IT to have control is security (even then, I'd be very weary of trusting the most sensitive data to IT). Most other reasons given against business user-developed solutions are completely false. Take the reasons above: "software produced may not be well designed...", "the software may not be well supported...". Who are we kidding here? IT's track record on these fronts is simply not good enough to claim the high ground here.

Savvy business users solve their own information problems - they have been doing so long before IT existed. Anyone remember triplicate forms? Fax machines? Photocopiers? These things didn't need IT departments to govern them and they worked very well. If IT cannot solve the problem, or IT's track record has been sufficiently poor that business users have lost faith in IT, then business users will solve their own problems, using whatever means are available to them. Access, Excel, and shared drives are frequently used very successfully by business users. If IT is to stay relevant to an organization, it needs to support it's business users needs and deliver technology that people actually want to use, not just technology people use because they have to.

I have seen an organization where a multimillion dollar portal implementation promised to solve many business technology and information sharing problems. Years later, still not in production, business users gave up, and in despair developed their own solutions by outsourcing the development of a data centric web application. Guess what? It worked brilliantly and other departments are now bypassing IT and doing the same, on their own departmental budgets.

IT is a support organization for business users. This may offend some who believe IT's place to be somewhere alongside executive management in terms of its importance to the business, but IT has to deliver what the business needs, otherwise its just justifying its own existence.

Answer 9

Sometimes I use Amazon EC2 and/or RDS when my company's resources are not enough or would take too long to provision. I pay for this out of my own pocket but do get to achieve my goals faster. All this without having to spend painful hours in meetings, trying to convince superiors or the SA-s that I really do need to do some thing or other.

In my mind, EC2 is the ultimate shadow IT. It's super easy to get going and provides me with the ultimate control.