If my application is written in C#, could I reference the Microsoft.Deployment.WindowsInstaller.Package.dll, or some other wix dlls, and create a new MSI file for instance?
Yes. Those Microsoft.Deployment.* DLLs are referred to as the "Deployment Tools Foundation", and they are documented in dtf.chm and dtfapi.chm files in the doc folder of your wix installation.
Creating a new MSI from scratch would start like this:
string path = "path/to/myinstaller.msi";
using (var db = new Database(path, DatabaseOpenMode.Create))
{
// lots of SQL queries to put stuff in the database here
}
Alternatively, you could copy an existing template MSI file, and then use SQL queries to modify it.
You would also need to package binary files and add them to the database. You can do that by opening the msi with the InstallPackage class.
In any case, creating an installer in this way will require detailed knowledge of the Windows Installer Database Tables.