Hi,
I've spent days looking around the internet, to see if there is anyway in to, do a reverse engineer but nought really. I don't there is a .FLA spec floating about. I've spent the last couple of days looking. If their was one, and you were using it, you would receive a cease and desist letter from Adobe, followed by the canadian equivalent of the DMCA take down notice.
According to Wikipedia, the format is a binary format, which is based on a Microsoft Binary compound file format specification, described here on Wikipedia. Compound File Format. But strangely because of, I think, the European anti trust settlements forcing Microsoft to publish their specs means you can download the MS Compoud File Format, here by typing Microsoft compound file format into Google. The first entry is the download link.
Once you have studied it, then you need to get a tool set together. It may be possible, their is a decompiler available for that format from MS. If not, your looking at maybe 3-6 months of work to get a basic spec together.
In the early 90's I reversed engineered the Novell IPX protocol and the NCP stack, which was a case of sending messages to the server, and getting a reply. So it fairly mechanical and tedious, after a few months we had a really good idea what was needed, and we built up a map of the IPX protocol calls, and replies. Same with NCP. We eventually were contacted by Novell who sent us a cease and desist and eventually wanted to sue us. But at the end of the day, they crapped out because we were using Clean Room techniques Wikipedia Clean Rooms explained. We pretty much took it apart from a server product in Binary, using a packet analyser, and rebuilt it using C++. But that was protocol reverse engineering. Much easier that what you are reflecting on.
So the first thing you need, is a disassembler. I can't give you any advice on what one to use, but there is many out their. Once you get the format, and you disassemble the file, you have to try and identify file headers, footers, constants, things like special symbol sequences, so you can start to create a map. For example on page 6 of the Windows Compound Binary File Format, it says in _absig has a signature format of DOCF 11E0 A1B1 1AE1. If you can find that kind of info in a the file, you start the map.
But before you go to that length, see if their is a decompiler about. I think most of the decompilers you find in Google will perhaps be lightweight. Go to hacker groups like these.
CrackZ. That IDA product is a disassembler. This may help. Introduction to Reverse Engineering. Reverse Engineering Community
Hope that help. If you do, do it. It will take months. Persevere!
Bob.