1

I want to create a product feed using the Merge() function from OrmLite but getting an out of memory exception. Our SQL Server database is quite big and we have multiple nested tables.

Could that be the source of the exception? What would be a good solution for loading a huge amount of data at once? Or what is the best practice in this case?

I've also tried to loop the products and then load the needed data for each product, but that took forever.

var webshops = DbConnection.Select<Webshop>();
var languages = DbConnection.Select<Language>();
var products = DbConnection.Select<Product>();
var productCategories = DbConnection.Select<ProductCategory>();
var productManufacturers = DbConnection.Select<ProductManufacturer>();
var productSpecificationAttributes = DbConnection.Select<ProductSpecificationAttribute>();
var productPictures = DbConnection.Select<ProductPicture>();
var productTranslations = DbConnection.Select<ProductTranslation>();
var productTierPrices = DbConnection.Select<ProductTierPrice>();
var productPurchasePrices = DbConnection.Select<ProductPurchasePrice>();
var stockItems = DbConnection.Select<StockItem>();
var groupedProducts = DbConnection.Select<GroupedProduct>();
var categories = DbConnection.Select<Category>();
var categoryTranslations = DbConnection.Select<CategoryTranslation>();
var manufacturers = DbConnection.Select<Manufacturer>();
var manufacturerTranslations = DbConnection.Select<ManufacturerTranslation>();
var manufacturerTierPrices = DbConnection.Select<ManufacturerTierPrice>();
var specificationAttributes = DbConnection.Select<SpecificationAttribute>();
var specificationAttributeTranslations = DbConnection.Select<SpecificationAttributeTranslation>();
var specificationAttributeOptions = DbConnection.Select<SpecificationAttributeOption>();
var specificationAttributeOptionTranslations = DbConnection.Select<SpecificationAttributeOptionTranslation>();
var stockUnits = DbConnection.Select<StockUnit>(x => x.StockUnitStatus == StockUnitStatus.In);

#region merge

webshops.Merge(languages);
manufacturers.Merge(manufacturerTranslations);
manufacturers.Merge(manufacturerTierPrices);
categories.Merge(categoryTranslations);
specificationAttributes.Merge(specificationAttributeTranslations);
specificationAttributeOptions.Merge(specificationAttributes);
specificationAttributeOptions.Merge(specificationAttributeOptionTranslations);
productCategories.Merge(categories);
productManufacturers.Merge(manufacturers);
productSpecificationAttributes.Merge(specificationAttributeOptions);

products.Merge(productCategories);
products.Merge(productManufacturers);
products.Merge(productSpecificationAttributes);
products.Merge(productTranslations);
products.Merge(productPictures);
products.Merge(productTierPrices);
products.Merge(productPurchasePrices);
products.Merge(stockItems);
products.Merge(groupedProducts);

#endregion
2
  • 1
    Why you use var for everything? It makes your code unreadable for us and even you should get confused about the types. I guess DbConnection.Select returns a System.Data.DataTable, correct? Commented May 22, 2024 at 9:52
  • 3
    Yes, out of memory happens here because you're loading more data than can fit in your memory. Best practice is to only load the data you actually need. This is a shot in the dark (as I can't tell what products is or what you're doing with the data and how you're looping over it) but you could look at chunking the data/ loop, i.e. load 100 products at a time instead of 1/ all Commented May 22, 2024 at 9:53

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.