I have a SQL server C# CLR function that streams results. because of that I open a new connection inside the CLR and because of that I need to create the CLR unsafe. Now I want to move to sql container on linux and there are only safe assemblies allowed.
If there some way to come around that I can stream the data with a context connection? It is no solution to cache all the data as list and return it complete. The amount of data is to much and I will get memory trouble.
Or should I bring my CLR outside of the SQL server as a web service and I call the web service with a CLR? is that possible? sounds strange CLR calls a web service that calls the same sql server and streams the data...
OR is there another idea that I have not focused?
My CLR looks like (shorten pseudo), there are intern more complex classes and code
[SqlFunction
(
DataAccess = DataAccessKind.Read,
IsDeterministic = true,
IsPrecise = true,
SystemDataAccess = SystemDataAccessKind.Read,
FillRowMethodName = "FillMethod",
TableDefinition = ...
)]
public static IEnumerable GetData(DateTimeOffset timeRangeStart, DateTimeOffset timeRangeEnd, int maxRecords, int lcid, string filterClause)
{
// runs a query to get all need parameters to open a new connection.
var connectionString = ContextConnectionGetConnectionString();
using connection = new Connection(connectionString)
connection.Open();
var complexScript = CreateTheScript(...);
var command = connection.CreateCommand(complexScript)
var streamReader = command.ExecuteReader();
try
{
foreach (var items in readData(streamReader))
{
yield return items;
}
}
finally
{
CloseStream(streamReader, streamConnection);
command.Dispose();
}
}
I think there's a lot of context missing. Like, what is in complexScript and why can't that be just a normal (i.e. non-CLR) stored procedure (or function)?
the query string contains a sql like language with virtual table names that is parsed and build to work on the current database. The Tables in the database are also splited like segments and with the timerange only the segment tables in that time range values are will be added to the query that is executed at the end.
CLR function is used in other views and from SQL server reports
complexScriptand why can't that be just a normal (i.e. non-CLR) stored procedure (or function)? How is your current CLR function called and by whom? From the little that you've shown above, I'm not seeing an advantage of using CLR.CreateTheScriptdoes, in the end it creates a SQL query. Instead of returning the result rows directly to the caller, it loops over them before returning them - the results are being streamed by the database already.OPTION( (FAST FIRST N)hint. Otherwise you'll still have to wait for the entire query to finish, including any buffering, before SQL Server starts returning rows even to SQLCLRThe Tables in the database are also splited like...you're describing table partitioning, which is available in all supported versions and editions. Querying partitioned tables, eliminating partitions, is transparent. You don't gain in speed either, partitioning is a data management feature. There are far more important performance features like table compression to reduce IO and columnstore indexes. These provide real 100x improvements