Check out scriptcs. Scott Hanselman has a great blog post about it here. It allows you to do scripting with C#, either by running arbitrary scripts as *.csx files, written with with your favorite text editor or by using the REPL. It also has integration with NuGet to allow you to easily pull down dependencies.
The ScriptCs.Core library can be used to host script execution in your own application, like OctopusDeploy has done.
The scripting engine is pluggable. As of now, it uses Roslyn as the C# scripting engine, but there's also work being done to use mono as an alternative. There's nothing stopping you from implementing f.ex. F# support.
To make bootstrapping of popular frameworks even easier, there's script packs that reduces the amount of code you have to write to get something running quickly.
To get you started, check out the samples repository.