The main thing is that this isn't just .Net: it's a limitation of the underlying system most every language will use to represent a float in memory. TheThe precision only goes so far.
You can also have some fun with relatively simple numbers, when you take into account that it's not even base ten. 00.1 (1/10th), for example, is a repeating decimal when represented in binary, just as 1/3rd is when represented in decimal.