So I'm not sure how familiar with VoIP you are but if you follow the logic that the gigabit phones would be 'better' for audio than the 100M, then that is implying that perhaps you thing the phone audio could exceed (or even come close to) 100M of bandwidth utilization. If that were the case, then you would need at least that much internet connectivity for each phone call, which the majority of us don't have. VoIP isn't demanding as far as bandwidth, it's demanding on the quality of the connection:
https://www.3cx.com/blog/docs/bandwidth-utilised-for-voip/
The port speed on the phones is only relevant if you are putting a device behind the phone and how much bandwidth it needs. You could put a PC behind a phone with a 100M port, but it's going to be limited to 100M. Other than that, there really isn't much to consider. Potentially a device behind the phone can consume so much bandwidth as to affect the audio, but that can also happen with a 1000M port so that really isn't relevant. Generally if you are generating enough traffic that saturating a 1000M port is going to cause audio issues then you have another issue like a network configuration problem, some sort of virus replicating like mad, etc. Otherwise if you regularly push that amount of traffic you wouldn't even be looking at anything 100M.
Another way to think about it, and perhaps where you are getting hung up, is forget that it's a phone and pretend it's just a 2 port switch and base your judgement on that.
Founded in 2005, when VoIP was an emerging technology, 3CX has gone on to establish itself as a global leader in business communications.