A longer life also implies a longer reproductive cycle.
A longer reproductive cycle means the species will evolve much slower. That will make it more vulnerable to gradual environmental changes and generally hamper its development.
When there is a new ice age which drastically lowers temperatures for a thousand years, 80% of humanity might get wiped out at first, but the most cold-resistant humans will survive, procreate and pass their cold-resistance genes on to the next generation. When the ice age is over, they will have adapted and get out of it with strong numbers.
But the elves will not be so lucky. Their longer reproduction cycle means that anyone who dies can not be replaced so easily and it will take far longer for the cold-resistance genes to procreate through the population. At the end of the ice age they will have become almost extinct.
When you have a long life without a long reproductive cycle, you will have a different problem: Overpopulation. Too many people get born but not enough old people die and keep consuming resources.
But this affects the survival of the whole species, not so much the individual. When you compare an average middle-aged elf (300 years old) to an average middle-aged human (30 years old), the elf has the advantage that she had far more time to acquire skills and knowledge. You could counter that by making the species which live longer more forgetful. The elf might have been a masterful fletcher 100 years ago and a great warrior 200 years ago, but now barely remembers those times and forgot all of the skills she learned. Now she is no match for a human who acquired these skills just decades ago.