How to install the Ruby LSP extension in a devcontainer without root user

I'm developing a ruby app in a devcontainer in VScode. In the container I am a non-root user. Now I want to install the Ruby LSP extension in the devcontainer, but I get following errors:

Failed to setup the bundle: Command failed: gem install ruby-lsp ERROR: While executing gem ... (Gem::FilePermissionError) You don't have write permissions for the /usr/local/bundle/gems/sorbet-runtime-0.5.11514 directory.
/usr/local/lib/ruby/site_ruby/3.1.0/rubygems/installer.rb:356:in rescue in install'
/usr/local/lib/ruby/site_ruby/3.1.0/rubygems/installer.rb:305:in install' 
/usr/local/lib/ruby/site_ruby/3.1.0/rubygems/resolver/specification.rb:105:in install' 
/usr/local/lib/ruby/site_ruby/3.1.0/rubygems/request_set.rb:195:in block in install' 
/usr/local/lib/ruby/site_ruby/3.1.0/rubygems/request_set.rb:183:in each' /usr/local/lib/ruby/site_ruby/3.1.0/rubygems/request_set.rb:183:in install' 
/usr/local/lib/ruby/site_ruby/3.1.0/rubygems/commands/install_command.rb:207:in install_gem' /usr/local/lib/ruby/site_ruby/3.1.0/rubygems/commands/install_command.rb:223:in block in install_gems' /usr/local/lib/ruby/site_ruby/3.1.0/rubygems/commands/install_command.rb:216:in each' /usr/local/lib/ruby/site_ruby/3.1.0/rubygems/commands/install_command.rb:216:in install_gems' /usr/local/lib/ruby/site_ruby/3.1.0/rubygems/commands/install_command.rb:162:in execute' /usr/local/lib/ruby/site_ruby/3.1.0/rubygems/command.rb:326:in invoke_with_build_args' /usr/local/lib/ruby/site_ruby/3.1.0/rubygems/command_manager.rb:254:in invoke_command' /usr/local/lib/ruby/site_ruby/3.1.0/rubygems/command_manager.rb:193:in process_args' /usr/local/lib/ruby/site_ruby/3.1.0/rubygems/command_manager.rb:151:in run' /usr/local/lib/ruby/site_ruby/3.1.0/rubygems/gem_runner.rb:56:in run' /usr/local/bin/gem:12:in `<main>' . See Troubleshooting for help

Even if I pre-install the gems locally and then install the extension...

# Gemfile
gem "ruby-lsp"
gem "rubocop"

...I get the same error.

Can someone please tell me what I have to do to get this extension to work? Is there a setting?