In case of an invasion, the improvements will be destroyed. If you do not need an improvement because you want to place another building in its spot, you can order your builders to remove it. Improvements are constructed by builders and you only have to build them once. Keep in mind that some improvements can give you an income bonus if they are placed on a specific terrain or adjacent to it.
Some of the improvements are available from the very beginning, while others require developing certain technologies. The main purpose of improvements is gaining access to strategic, luxury and bonus resources, and improving production potential of a terrain.