have it so system defences only engage when the system is attacked...
For planetary bysed defense I would say this should be true, but for System defense and normal starships I would give the defender the choice to attack the intruder. As for Mines/Satelites/Starbases this is again a seperate issue based on the decision how these are placed.
Or, have a ship class that is small but can slip past systems undetected, designed solely for 'hit and run' raiding
This seems nice. smal ships have a good chance of passing through minefeelds. Cloaking devices should defenetly work this way and smal ships (if there is just one ore a few) should even have a good chance of not beeing detected in a unfriendly system.
Or, when the designers construct the starlane matrix, have it so that starlanes are much more 'connected' to each other, meaning to enter into your cluster, you'd have at least 5 or 6 entry points (maybe includihg a wormhole).
this could be a slider on startup, but if you turn this too high we loose the advantage of frontlines and all the AI bennefits we saw in this.
Or, have a technology that allows players to "blink" past a system.
what do you mean by "blink" ??
Or, have starlanes fluctuate over time. Some new ones are randomly created, others slowly fall apart. so building massive defences on system A won't do much good if suddenly a new entrance opens to your homeworld.
this way neither I nor the AI can actually plan anything. I could go in with a fleet attack a system and after conwering it it may not be connected to my empire any more. This could be a rare event but nothing more.
Building or destroying starlanes artificially (at high costs) on the other hand can be very usefull and are not as unforseeable and luck depending.
Or, finally, have the equivalent of "space artillery". firing bombs through a starlane to explode on the other side, causing small amounts of collateral damage.
This sounds very nice and I would be happy to see something like this in FO