Add or update data to existing database with entity framework

Question

I am trying to create a custom migration step to clean up some default values in our database. Basically we have some data that is either invalid and needs to be updated and in other cases it doesn't exist at all. The system is installed on several servers and a lot of this data was added manually at one point so its hard to know what server has what data.

What I want to do is create a migration step to clean it all up. If this value does not exist in the table i need to insert it. However if it does exist then either I updated it or just delete it then insert it. I am having a hard time figuring out how to do this.

Add-Migration DataCleanup

Create migration step

public partial class DataCleanup : Migration
{
    protected override void Up(MigrationBuilder migrationBuilder)
    {
        migrationBuilder.InsertData(
            table: "Blogs",
            columns: new[] { "BlogId", "Url" },
            values: new object[] { 4, "http://sample4.com" });}

    protected override void Down(MigrationBuilder migrationBuilder)
    {
        migrationBuilder.DeleteData(
            table: "Blogs",
            keyColumn: "BlogId",
            keyValue: 4);
    }
}

This will work if the row didn't exist previously, but if the row did exist then I will need to update to be sure the value is the correct value. I am not to worried about the primary keys as this is a reference table they "should" be the same.

The only other option I can think of would be to run a truncate on these tables and then just run the inserts after.

migrationBuilder.Sql("TRUNCATE TABLE [Blogs]", true);

Note on down

I am leaning towards not being able to do the down at all. There will be no way of knowing what state this one server was in before i ran this.

If the values aren't autogenerated, it doesn' matter how you do something as long as the table ends up with what you want. Truncate followed by an INSERT .. VALUES() is OK in this case, both for the 'up' and the 'down' step. Or you could use a MERGE statement that "merges" the new data into the existing table. You can specify that data using row constructor syntax, eg (Values (val11,val12), (val21,val22)) myData(Field1,Field2) defines a "table" called myData with fields Field1 and Field2 which can be used in joins, MERGE statements, subqueries — Panagiotis Kanavos
– Panagiotis Kanavos, Commented Feb 6, 2019 at 13:08
You can add an OUTPUT clause to get inserted/deleted values, just like a trigger, and store them. — Panagiotis Kanavos
– Panagiotis Kanavos, Commented Feb 6, 2019 at 13:09

Linda Lawton - DaImTo · Accepted Answer · 2019-02-07 11:50:09Z

First off i was unable to do a truncate on the tables there are foreign keys set up which meant that that was just not going to work.

I ended up doing a bunch of sql inserts that check first of the row exists and if it doesn then we insert it.

migrationBuilder.Sql("INSERT INTO IdentityResources (Description, DisplayName, Emphasize, Enabled, Name, Required, ShowInDiscoveryDocument) " +
                                 "SELECT 'Your email address', 'User email', 1, 1, 'email', 0, 1 " +
                                 "WHERE NOT EXISTS(SELECT * " +
                                                  "FROM IdentityResources " +
                                                  "WHERE name = 'email'); ", true);

Things got a little more complicated when i needed to add the key from the first insert.

migrationBuilder.Sql("INSERT INTO IdentityClaims (IdentityResourceId, Type) " +
                                         "SELECT id, 'email' " +
                                         "FROM IdentityResources " +
                                         "WHERE Name = 'email' " +
                                         "AND NOT EXISTS(SELECT * " +
                                         "FROM [IdentityClaims] " +
                                         "WHERE type = 'email')");

This all worked out in the end. A new install of the system builds the proper database and everyone gets updated to ensure that they have at least the required data.

Unfortunately removing the unneeded data will have to wait for another day.

Alper Silistre · Accepted Answer · 2021-11-20 18:45:12Z

For those who are wondering if there is any way to solve this problem without dealing with SQL statements, there is one workaround which some might not favor, but it was worked for us.

This works for a scenario where you want to insert some data to the database, but someone already inserted some data previously to the database manually, and there is no record of this operation in the migration history. You want to be a good developer and keep things in record when you insert new data to your tables, but the existing data prevents you to do this properly.

In your mappings, use your EntityTypeBuild builder object in the Entity Framework and take advantage of builder.HasData(IEnumerable data) method. Just add your entities to a list and pass this to the HasData method. When you create a new migration, this data will appear as:

migrationBuilder.InsertData(
                schema: "dbo",
                table: "Country",
                columns: new[] { "Id", "CreatedOnUtc", "ModifiedOnUtc", "Name" },
                values: new object[,]
                {
                    { 1, new DateTime(2020, 04, 17, 13, 31, 19, 188, DateTimeKind.Utc).AddTicks(8570), null, "USA" },
                    { 2, new DateTime(2020, 04, 17, 13, 31, 19, 188, DateTimeKind.Utc).AddTicks(8570), null, "Netherlands" },
                    .
                    .
                });

In this InsertMigration method, there might be existing data already available in your database. Copy and paste the InsertData block to one of your old migrations, where it's already ran against the database. In this code, remove new objects from the values array. So we are moving the old data to a previous migration, only keeping the new data in the new migration. Because otherwise you would get an error during the SQL Insert operation as the previous records were already in the table as you are trying to insert the same data with the same Primary Key.

So when you run this migration against your DB, it will only insert the new rows. The main advantage of this approach is that; if you happen to deploy your application to a new environment along with an empty SQL DB in the future, the previous migration will insert previous seed data and your new migration will insert the remaining part. So this trick will work for new environments as well.

One thing to keep in mind that, ContextModelSnapshot is an auto-generated file when crating migrations. You should put builder.HasData(data) to there before creating the new migration. Because otherwise the Entity Framework does not understand, the seed operation is already applied with the previous migration. This is required for just one time, before creating the new migration, which includes new seed data. After Entity Framework creates the new migration file, this will be overwritten again, so don't worry about touching this auto-generated file.

Collectives™ on Stack Overflow

Add or update data to existing database with entity framework

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related