You see, if the musical composition goes for a while in one texture or mode, we are getting used to that texture or mode. After practicing for some time, it becomes less difficult to play.
But when the texture changes we have some problems again because at the time of change we have to get accustomed to the change. And sometimes where the clefs change is also a difficult place.
The time issue with the systematic method is not a big deal - it only takes about 15 minutes a day of regular sight-reading practice to begin to see some tremendous changes in your skills.
But you have to be very methodical about that - increase the difficulty level just one step at a time and practice in ascending numbers of accidentals etc. After you perfect your sight-reading skills, it's fascinating how much faster the entire learning process of new music becomes.
By the way, do you want to learn my special powerful techniques which help me to master any piece of organ music up to 10 times faster? If so, download my video Organ Practice Guide.