At least at the beginning improvisation is like learning to swim - simply jump into shallow water and start working with your hands and feet. And you begin to see that you little by little start to swim.
When we improvise, at first we can also think similarly - choose a safe playing mode when nobody is listening and start fooling around with a few sounds, listen to what kind of mood they create, and tell a musical story.
Of course, sometime in the future we will need also the skills in music theory, harmony, counterpoint, and many other areas, but first we have to start playing.
Play and fail. Then play some more.
This is how we get rid of the fear of improvisation.