Seed magazine has a very readable article that provides a 200,000 foot view of the relationship between number theory and quantum mechanics: Seed: Prime Numbers Get Hitched. I do wish Du Sautoy had mentioned whether this had any implications for cryptography; I believe current techniques rely in part on the technical difficulty of factoring large numbers. Naively one might think a breakthrough in understanding prime numbers would not be all that great for the stock market.
He mentions by way of background Riemann's role in anticipating general relativity, and also describing the Zeta functions that play a role in both QM and prime number theory. If we do reconcile general relativity and quantum mechanics, it would not be amazing if Riemann should turn out to be a common source. I have a bit of a personal connection here. As a high school student, back before there were photocopiers, I gave a talk on non-euclidean geometry. I don't believe I've subsequently worked my feeble brain as hard as I did preparing for that presentation. I doubt much of the class got anything from it; I could just as well have delivered it in ancient Greek. All the same, Riemann made a lasting impression on me.
At one point in my brash days I foolishly dismissed the uncanny connection between mathematics and physics with some "clever" phrase that I mercifully don't remember. I apologize to my victim. I've long since joined the ranks of those who find the relationship more than a bit unsettling.