So how can we use this system of 8 or 12 modes in our organ improvisations?
The most obvious application of such modes is in Renaissance style pieces. Since chant-based organ music in the Renaissance period was primarily part of Alternatim practice (when verses of the chant were sung in alternation with the choral polyphony or organ music or both), we too can improvise short versets composed in these modes which would be based on the Gregorian chant melodies.
For example, imagine a hymn, such as well known Veni Creator Spiritus. If the starting note would be G, then its mode is Mixolydian (No. 7). You can improvise a number of verses this way (presented in the increasing number of voices):
1) A Bicinium for 2 voices with the chant in the soprano
2) A Bicinium for 2 voices with the chant in the bass
3) A trio for 3 voices with the chant in the soprano
4) A trio for 3 voices with the chant in the middle voice
5) A trio for 3 voices with the chant in the bass
6) A quartet for 4 voices with the chant in the soprano
7) A quartet for 4 voices with the chant in the alto
8) A quartet for 4 voices with the chant in the tenor
9) A quartet for 4 voices with the chant in the bass
Also you can improvise canons for two voices with one or more additional voices. Canons can be based on any interval but the most common are an octave, a fifth, and a fourth. Most commonly a canonic voice can enter after the distance of 1 note, 2 notes, or 4 notes.
Remember to stay within the same mode throughout the piece. Also your improvisation does not have to be based on the given chant melody at all. We know of versets which are freely composed.
If you are improvising in the polyphonic style, you can have a polyphony of two types - contrasting and imitative. In contrasting polyphony, the rhythm and the melody of the chant is contrasted with the quicker (usually) rhythmical values of other voices. In imitating polyphony every voice imitates the theme or the fragment of the theme in alternation from the same or different pitch level.
Your improvisation can become quite modern and colorful if you include modulations of the same mode and extend your piece with additional episodes. I've done this in my Veni Creator Spiritus (2011).
By the way, do you want to learn my special powerful techniques which help me to master any piece of organ music up to 10 times faster? If so, download my video Organ Practice Guide.