Morde,
This is exactly how my bongos3 setup is done. However, instead of creating a 5x4 bar I've created 4 5v1 bars stacked on eachother that map to the left side of my keyboard. I use both shift and alt to change the set and haven't noticed any issue with it. I'll try re-creating your issue, but you might want to try breaking your single bar into multiple.
D
Edit: try as I might, I can't re-create the issue you describe with my multiple bars.
This is exactly how my bongos3 setup is done. However, instead of creating a 5x4 bar I've created 4 5v1 bars stacked on eachother that map to the left side of my keyboard. I use both shift and alt to change the set and haven't noticed any issue with it. I'll try re-creating your issue, but you might want to try breaking your single bar into multiple.
D
Edit: try as I might, I can't re-create the issue you describe with my multiple bars.