How to use .NET Framework dll buit with Entity Framework 6.0 in .NET Core application

If you replace your usages of ConfigurationManager.AppSettings["key"] with using IConfiguration.Item["key"], that should be the least-invasive migration path, in my opinion. There are multiple more "fancy" options in .NET Core, but this should be the path of least resistance, IMHO.

If you need the library to work in both .NET Core and .NET Framework applications, I would recommend creating a small wrapper around the configuration, say, e.g.

public interface IAppSettings {
    public string this[string key] { get; }
}

And, then, creating two implementations of it, one for .NET Core, and one for .NET Framework. Something along the lines of

public class WebConfigAppSettings: IAppSettings {
     public string this[string key] => ConfigurationManager.AppSettings[key];
}

and, one for .NET Core, similar to:

public class NetCoreAppSettings: IAppSettings {

    public NetCoreAppSettings(IConfiguration source) => _source = source;

    public string this[string key] => _source[key];
}

Create the implementations outside of your common EF library.

Then, make your EF library classes needing configuration take a dependency on the IAppSettings interface, and use IoC/Dependency Injection to inject the correct implementation in each application using the library.