While I love using CDNs for my websites and have been doing so for around couple of years, I have always treated them as something that you add-on extra, and not something that is a necessity. And I guess majority of webmasters (especially those who do not run e-commerce sites) feel the same.
However the ongoing debate on net neutrality has made me think otherwise, especially as ISPs try their best to maximize profit, by not only charging their subscribers to access internet, but also trying to charge website and application providers, in order to deliver their content on a preferential basis.
While companies with enough money power and clout eg. Google, Facebook etc. can afford to install their own peering or caching servers with major ISPs worldwide, little guys like you and me cannot.
In such a scenario, the only way out is to piggyback on services of a CDN provider, in hope that they will have better peering agreement with ISPs and will thus be able to transfer your site at similar speed as that of Google or Facebook.
Image courtesy, Wikipedia
Then there is the fact that even though consumer internet speeds are getting faster every year, port speed for some of servers has remained at a paltry 100 Mbps to 1 Gbps (even though, many are many to start upgrading servers). This means if there are 10 users with 30 Mbps internet connection, trying to simultaneously download a file from your server which has a 100 Mbps port speed, then they won’t even be able to utilize half the speed of their internet connection.
Here too CDNs can play a crucial role and distribute the workload on multiples servers and POPs, which will likely allow users to burst to the maximum speed possible by their internet connection and thus removing the bottleneck, which exists due to paltry port speeds of majority of web servers.
While in the next couple of year, we might not reach a stage where these things start becoming all too apparent and start turning users away from your site, it is a possibility worth thinking and planning about.