If you're not using one of the SPI pin functions, you don't need to assign a pin to the function.
That's the easiest solution (with the next easiest one putting a pull up on it or tying the pin to GND the way you have) otherwise you'll probably see delays and other weirdness.
Hello @lfost ,
Some followers leaves the line floating when he is not handling it. In these cases the recommendation is to configure a pull-up in the MISO line so that it does not float when the follower so not driving the line.