- Leadership;
- Community;
- Quality.
In that order.
To clarify that, every successful open source project requires outstanding leadership. Think Larry Wall and Perl, Linus Torvalds and Linux and so on. As much as open source is a community effort, it requires someone pushing the thing forward and doing so without fragmenting is discouraging the community.
It's also worth asking the question: how do you define "success"? You have a closed source project. What are you hoping to achieve by making it open source? Quality? Adoption? Contributors to the code base?