Hi everyone! I'm Co-Founder and CTO of OnePage (http://myOnePage.com/joel).
I'd be interested to hear your views and answers to this particular problem:
I'm certain Yahoo! or Google don't expose their entire code to their developers! I was just wondering what method you use to restrict people from seeing the full code? In all projects there will obviously be parts of the code which contain important access credentials for database and API keys.
Thanks
They're going to need to see all the code at some stage. For debugging/analysis etc. Depending on the technology you use, you may not be able to hide it. Can they decompile packages (e.g. Java lets you do this).
Besides, it sends the wrong message to say that certain people can't see bits of the codebase (unless you truly have something groundbreaking in the code?).
If you want to maintain some secrecy from the POV of security, then credentials etc. should be configurable. It's ok for the developers to have credentials info for development/testing environments etc. (e.g. the development database).
When deploying to production your 1st-line support (or whoever is trusted) should provide this info, and consequently your development team won't have access to it.
I think the best way to prevent access to the whole system is by creating a system where you have a very loose couppling. That means you can assign the task of creating serviceX to a developer, and this developer only hav to implement a "stabale" version of ServiceX...
Give him the exact requirements, the interface he has to implement and a set of codecontracts, and you are "done"...
Now you only need to implement his code into your system and hope it does not blow up.
The only part i dont understand is why should there be access credentials etc in the code? Those belong into a configuration ;)
I think the real solution is to hire developers you trust, and give them access to what they need. If you feel you have to hide things from your employees, there's a big problem.
That being said, you can split things up into separate modules, then put each module in a different repository and only grant each developer access to the repositories you want.
With subversion, you can setup ACL and Groups. Check:
Not allowing the developers to access the code will definately prove counter productive in the long-term. It would restrict them to be productive/effective in adding new functionality. So IMHO, rethink on this. Its better to hire people whom you trust and let them access the code than trying to restrict it. Secondly, What are you trying to restrict Lines of program code or details of database, login credentials etc ? The credntials to login to db generally resides in a properties file. You can encrypt the content of the properties file and obfuscate the code which reads the properties file and decrypts it. To ensure that programmers dont understand the code, you can obfuscate your code libraries. But obfuscation comes with its own baggage.
Bottom line : you are trying to solve the wrong problem in restrcting developers access to code base
Provide external developers with an assembly or an API of some sorts.
I once wrote a blog application which supported the use of 3rd-party modules. I provided an abstract class and an interface for programmers to implement. If I needed to expose core objects like a user object, I would narrow it--basically providing a read-only object, which I'd then widen for use on "the other side".
It makes sense to limit access to certain production credentials to only those people who administer the production environment (assuming you do have specialized people to do this).
However, in a startup company like yours it doesn't make sense to hide/obfuscate parts of the actual source code from your developers. It seems bureaucratic and unnecessary, and it just sending a message to developers that they can't be trusted or are not important enough to see the whole thing. It also can lead to a culture of secrecy instead of transparency - are these values you want to cultivate in your company?
From what I understand of Google, they are pretty open with the source code within the company. Their algorithms are not a big secret, either; Google regularly publishes academic papers discussing their technology.
Hiding the source code is not the hardest problem to solve. For example you can split it in separate repositories and put a different password on them. Then distribute the compiled code (eg jar files), to the teams that need it. Or expose the functionality of the code as a web-api to avoid de-compilation threats (like google-graphs)
The real challenge though, IMO, is how to allow developers to be productive when using code that exists in your organisation and to whose source code they do not have access.
So, if you have written some code, you want your developers to re-use that code without having access to the source. The important thing in this case is that the code must have a very clean API, and it should be very well documented. Very often in corporate environments access to source code acts as a substitute of documentation.
Finally the way you expose the functionality of the hidden code depends on your deployment environment. For example, a web-app deployed on a server cluster, could be broken to web services.
I'd say it is a poorly modularized system that requires developers to see everything in order to do anything.
A very similar aspect of this problem is the learning curve issue. If you have a large system, you would like new hires to be able to be productive in some area of the system without necessarily understanding the details of every bit.
I worked on a project where my partner and I partitioned the code into UI and communication internals. (This project was an equipment integration simulator, and our code modeled both client and equipment.) He provided me a mock DLL for his internals, and I wrote the UI nearly 95% before we integrated the two. I never saw any of his code.
Hey, let's not be too hard on poor Joel! I don't like being blocked out of things any more than the next guy, and have had my share of frustrations when I was expected to fix something in a hurry but didn't have the proper authorizations to do it. On the other hand, "Hire people you can trust and then you don't have to worry about internal security" is a lot easier said than done. How are you going to insure that all new employees are trustworthy? What are you going to do, in the interview ask them if they are thieves? Do you really expect the thieves to just admit it? It only takes one dishonest person who steals a few million dollars and disappears to destroy a company. Or even one well-meaning but incompetent person who has the keys to the room where the backups are stored to decide that all those disks stuffed in the closet unused should be recycled. I certainly wouldn't want my bank or the people who hold my 401k to hand out authorization to every new person they hire to be able to get into my account and manipulate data without restriction.
While I haven't worked on a public version of this, I have been in a couple of work environments where there were structures around how to divide up the code into various components. Let me describe the examples that may help:
ASP/COM setup: In one company there was VBScript for ASP code on web pages with a object model written in C++ that talked to an Oracle back-end. There were only some developers that could change the object model which limited what some of us could do but also provided a way to separate who did what and different skills for each. The front-end developers had a handful of scripts to pull in the binaries and aside from the APIs didn't often care how things were done.
Back-end/front-end setup: In another company there was the back-end platform that had its own name and a few developers worked on that. A couple of other developers worked on the front-end that connected through an API using .Net remoting of objects to perform various functions. This had its good and bad moments as there would be times where the front-end required new calls to the back-end and we had to wait for those calls to be implemented.
Hopefully that is useful somehow.