This is pretty much exactly what I was suggesting. I don't see it being an issue, especially under normally driving conditions. Spacing things out does increase leverage but it should be fairly negligible for such a small increase. Even for people who punish their cars I would say it's a non-issue.

Just make sure your caliper bracket bolts and wheel studs are long enough for proper engagement and you're golden. With regards to the wheel studs, the 5mm shouldn't be enough of a change to need longer studs, but you will probably want to buy 5mm caliper bracket bolts like you mentioned.