I usually prefer to have my own Webpack configuration instead of create-react-app since it gives me full control over the bundling configuration.
But it really depends on the scale, complexity, and other factors of your project.
If your project is a pretty basic SPA, CRA can work fine.
Otherwise, you should either have your own Webpack configuration, or use libraries like gatsby.js (for static sites) or next.js (for server-rendered sites).
Keep in mind that configuring Webpack goes beyond simply getting it to properly parse and compile your code. Your site should also load quickly and have good performance scores (load critical CSS/JS, prefetching, code splitting etc.).
In my experience, getting Webpack to do all that is possible, but difficult, which is why gatsby.js/next.js are recommended alternatives.
Both gatsby.js/next.js use Webpack under the hood, so technically you can do it all yourself when you use Webpack, but they do all the heavy lifting for you. So unless you need to do something very specific that neither of these libraries supports, I think you should use them.
CRA, then eject it to see how this config is set up.