I'm brand new to FreeOrion, so maybe I missed this...
Is the space combat going to be turn based or real time?
As far as turn based goes, the most fun tactics engine I ever played was in the original XCOM games. I know this was ground combat, but a lot of the ideas could carry over.
For those of you not familiar with the XCOM games the combat essentially broke down as follows...
1) Each unit has a given number of "Time units" per turn. Everything they do, from looking around to shooting to changing weapon configuration, takes up a different amount of time units.
2) Only one side moves at a time.
3) What order you move your units in is completely up to you. If you want to do part of A then B then the rest of A that is completely up to you.
4) If you saved time units from your turn that unit might be able to 'react' during your opponents turn. These normally involved shooting (but some times it was panicing) and were not under the control of the player.
5) What you could see depended on where your units were and where they had looked that turn. This doesn't make a whole lot of sense in space, but if we add things like moons, asteroids, nebulas et cetera like have been mentioned in previous posts, it might be useful.
6) "Faster" units would have more time units to spend. This could also be done by everyone having the same number of time units, say 1000, and depending on your skill, technology, health, or many other factors how long you need to do something changes. (The latter would be my personal preference)
7) Your firing accuracy always depended on distance, your weapon, and your firing skill. (Generally in that order)
There were also many complex concepts including mind control, injuries to specific parts of the body would affect you differently, units panicing or going beserk (that always sucked as they would shoot anything near them which was usually your units), and even aliens assimilating you...
The only downside to using a tactics engine similar to XCOM is that for a lot of units it could be a giant pain. I remember some battles would take a very long time, but they were extremely intense so I didn't really care/notice. To combat this I have a couple suggestions...
1) In XCOM there was an auto-complete turn option (you could even auto complete the mission if you wanted) and the computer would take control and try it's best to win for you. You could even move a couple important units and then let the computer auto complete if you didn't care much about some units.
2) Perhaps you could divide units into squadrons and treat different squadrons as unique units. (But you should be able to divide down to individual ships if you need to for some reason)
3) Converserly to number 2, you should be able to create target groups of your enemies fleet. So if you had a large squadron and you wanted to give them an attack order you would be able to specify another large group for them to attack.
All in all, it's a pretty rough idea, but I'd like to know what you think.