Programming SQL Server "Yukon"

Part I - .NET Integration

Jurgen PostelmansU2U nv/sa

March 3, 2004

Applies to: SQL Server "Yukon" (beta 1) Visual Studio "Whidbey" (PDC preview version)

Summary:

This article will give you an overview how you can write stored procedures, triggers and user-defined functions in SQL Server "Yukon" using C# as a programming language.

Contents: Introduction Starting from a .NET Assembly Registering Class Libraries in SQL Server Registering User-Defined Functions in SQL Server .NET User-Defined Functions and the SqlFunction Attribute Registering .NET Stored Procedures in SQL Server Conclusion About the author

Introduction

One of the major new features of SQL Server "Yukon" is the integration of the .NET Framework Common Language Runtime (CLR) into the SQL Server database engine. This means that you now have the possibility to write your stored procedures, triggers and user-defined functions in managed code. The languages currently supported are C#, Visual Basic .NET, managed C++ and JavaScript .NET. Furthermore, SQL Server "Yukon" can only host the "Whidbey" version of the .NET Framework. Previous versions of the .NET Framework are not supported. For performance reasons, the .NET runtime is lazy loaded by SQL Server. This means that the .NET runtime will only be loaded when it is really necessary, such as when you would execute a managed stored procedure for the first time.

Whether or not you should implement your stored procedures and user-defined functions using .NET code depends largely on what you do in those functions. If they contain a lot of procedural code then writing them in .NET will typically make them faster and easier to implement. If, on the other hand, you do a lot of data access in your functions, implementing them in T-SQL will typically yield the best performance.

Starting from a .NET Assembly

To start we will create a new .NET Class Library in Visual Studio "Whidbey". This Class Library project is called MathTutor and will implement some simple mathematical functions.

Not all functions that you implement in .NET classes can be accessed by T-SQL. The methods you want to expose as stored procedures, triggers or user-defined functions must follow at least 3 conditions:

They must be in public classes. They must be static and public. They cannot be in nested classes.

In the MathTutor project we will implement one class called Math with 3 simple functions. The initial version of this class is shown below.

namespace MathTutor

{

public class Math

{

public static SqlInt32 AddNumbers(SqlInt32 i, SqlInt32 j)

{

return i + j;

}

public static int SubtractNumbers(int i, int j)

{

return i - j;

}

public static SqlInt32 IncrementBy(SqlInt32 by, ref SqlInt32 number)

{

int retValue = 0;

try

{

number += by;

}

catch (Exception ex)

{

retValue = 1;

}

return retValue;

}

}

}

Note that to declare the parameters of the methods, you can either use the standard types provided by the .NET Framework or their corresponding SqlTypes. If you know that your methods will only be used in T-SQL it is preferable to use the SqlTypes which are defined in the System.Data.SqlTypes namespace. The SqlTypes behave in the same way as the built-in SQL Server data types, especially when you're working with NULL values.

Registering Class Libraries in SQL Server

Once the Class Library is compiled we need to perform two steps in SQL Server "Yukon" to expose the methods in our MathTutor class library:

Register the assembly in SQL Server "Yukon".

Register the methods in the assembly as stored procedures, triggers or user-defined functions.

These steps are mandatory because SQL Server "Yukon" cannot execute any arbitrary managed code that you might have in your assemblies.

The first step is accomplished using the CREATE ASSEMBLY statement as shown below.

CREATE ASSEMBLY MathTutor

FROM 'C:\Articles\Yukon\MathTutor\bin\Debug\MathTutor.dll'

The CREATE ASSEMBLY statement registers and loads an assembly in SQL Server. In the FROM clause you specify the location of the assembly you want to register. If this assembly depends on other assemblies you will also need to register these. Once the assembly is registered the original assemblies on the file system are not used anymore. All the registration information is stored in 3 SQL Server system tables. These are called sys.assemblies, sys.assembly_files and sys.assembly_modules.

When you load an assembly in SQL Server you have the possibility to specify a security level for you managed code. This is done using the WITH PERMISSION_SET parameter as shown below.

CREATE ASSEMBLY MathTutor

FROM 'C:\Articles\Yukon\MathTutor\bin\Debug\MathTutor.dll'

WITH PERMISSION_SET = SAFE

When the permission set is set to safe, which is the default, the managed code in our assembly cannot access any external resources like the file system, registry, network...

To un-register an assembly you can use the DROP ASSEMBLY statement.

DROP ASSEMBLY MathTutor

Registering User-Defined Functions in SQL Server

Once the assembly is registered, AddNumbers and SubtractNumbers can be registered as user-defined functions in T-SQL. This is done using the CREATE FUNCTION statement.

CREATE FUNCTION AddNumbers (@I INT, @J INT) RETURNS INT

AS EXTERNAL NAME MathTutor:[MathTutor.Math]::AddNumbers

CREATE FUNCTION SubtractNumbers (@I INT, @J INT) RETURNS INT

AS EXTERNAL NAME MathTutor:[MathTutor.Math]::SubtractNumbers

In the CREATE FUNCTION statement, the clause EXTERNAL NAME is used to indicate that the user-defined function maps to a method in the assembly. The name of the method you want to register is written as AssemblyName:FullyQualifiedClassName::MethodName.

Once the user-defined functions are registered we can call them using the same syntax as you would use for T-SQL user-defined functions.

If you execute the following statement

SELECT dbo.AddNumbers(10,20)

you will see the result shown below:

-----------

30

(1 row(s) affected)

If your user-defined functions do not execute as expected you can debug them by attaching the Visual Studio .NET debugger to the SQL Server process. This process is called SqlServr.exe as show in the picture below.

Figure 1: Attaching the Visual Studio .NET debugger to the SQL Server process

Once the debugger is attached to the SQL Server process, execute your user-defined function in the SQL Workbench. The breakpoint in your Visual Studio .NET project will be hit and you can debug your source code.

As explained previously, you can either use standard types or SqlTypes for the declaration of the parameters or return value of your user-defined functions. The main difference between them lies in the way NULL values are handled. If you pass a NULL value to a standard type parameter an exception is generated. This is due to the fact that standard types like Int32 are not nullable. If you execute the following line of code

SELECT dbo.SubtractNumbers(10,null)

the generated result is

-----------

.Net SqlClient Data Provider: Msg 6569, Level 16, State 1, Line 1

'SubtractNumbers' failed because input parameter 2 is not allowed to be null.

.NET User-Defined Functions and the SqlFunction Attribute

If you know that you will not access any database objects in your user-defined functions you can add the SqlFunction attribute and set the DataAcess and SystemDataAccess properties to None. This way SQL Server can optimize the execution of user-defined functions that do not use the SQL Server

InProc Data Provider. The use of the SqlServer InProc Data Provider will be covered in a follow-up article. The resulting code is show below.

namespace MathTutor

{

public class Math

{

[SqlFunction(DataAccess = DataAccessKind.None,

SystemDataAccess = SystemDataAccessKind.None,

IsDeterministic = true, IsPrecise = true)]

public static SqlInt32 AddNumbers(SqlInt32 i, SqlInt32 j)

{

return i + j;

}

[SqlFunction(DataAccess = DataAccessKind.None,

SystemDataAccess = SystemDataAccessKind.None,

IsDeterministic = true,IsPrecise = true)]

public static int SubtractNumbers(int i, int j)

{

return i - j;

}

public static SqlInt32 IncrementBy(SqlInt32 by, ref SqlInt32 number)

{

int retValue = 0;

try

{

number += by;

}

catch (Exception ex)

{

retValue = 1;

}

return retValue;

}

}

}

Also note that the IsDeterministic property is used on the SqlFunction attribute to mark the function as being deterministic. Deterministic functions always return the same result any time they are called with a specific set of input values.

User-defined functions can not only be used in SELECT statements but also for the definition of computed fields in a table. In the following table there is a field called Addition that contains the result of the execution on the AddNumbers user-defined function.

CREATE TABLE Numbers

(

Number1 INT,

Number2 INT,

Addition AS dbo.AddNumbers(Number1, Number2) PERSISTED

)

The PERSISTED keyword tells SQL Server to physically store the computed values in the field of the table. The value in the computed field will automatically be updated whenever one of the dependent fields change.

Since the Addition field is persisted and since the user-defined function is marked as deterministic you could create an index on this computed field.

CREATE INDEX idx ON Numbers(Addition)

This would not be possible is AddNumbers was not deterministic since the return value of the AddNumbers function would be different every time it's called wilh a specific set of input values.

Registering .NET Stored Procedures in SQL Server

To register a method from an assembly as a stored procedure you use the CREATE PROCEDURE statement together with the EXTERNAL NAME clause.

CREATE PROCEDURE IncrementBy (@by INT, @number INT OUTPUT)

AS EXTERNAL NAME MathTutor:[MathTutor.Math]::IncrementBy

To execute the stored procedure "IncrementBy" the following T-SQL code can be used.

DECLARE @result INT

DECLARE @number INT

SET @number = 111

EXEC @result = IncrementBy 10,@number OUTPUT

PRINT @result

PRINT @number

Conclusion

In this first article we covered the integration between the .NET Framework and SQL Server "Yukon". You saw how stored procedures, user-defined functions and triggers can be created in .NET and used within SQL Server "Yukon".

Programming SQL Server 2005

Part II - .NET Integration and the SqlServer Data Provider

Jurgen PostelmansU2U nv/sa

April 7, 2004

Applies to: SQL Server 2005 Beta 1 (formerly named SQL Server 'Yukon') Visual Studio 2005 (formerly named Visual Studio 'Whidbey')

Summary:

Learn how you can access database objects from within stored procedures or functions using managed code and the SqlServer Data Provider in SQL Server 2005.

Contents: Introduction The SqlContext object Sending data to the client using the SqlPipe object Server-side cursors and the SqlResultSet object .NET User-defined functions and Security Conclusion About the author

Introduction

In the previous article I showed how you can write stored procedures and functions in managed code and how to use them in T-SQL. What I didn't cover was how you can access database objects in managed stored procedures or functions. For this purpose SQL Server 2005 ships with a new managed ADO.NET provider called the SqlServer Data Provider (as opposed to the SqlClient Data Provider, which shipped since the .NET Framework v1.0).

This managed provider is an in-process provider that is used to directly communicate from the Common Language Runtime (CLR) to SQL Server. The SQL Server in-process provider can only connect to the SQL Server that hosts the CLR. It cannot be used to connect to any other SQL Server you might have running on the network. The SqlServer Data Provider is implemented in the System.Data.SqlServer namespace.

The SqlContext object

The main object of the managed SqlServer Data Provider is the SqlContext object. This object represents the current execution context of the managed stored procedure or function that runs inside SQL Server.

A first example of the usage of the SqlContext object is shown in the code below. This code is part of a .NET Class Library called NorthwindDAL.

using System;

using System.Data.SqlTypes;

using System.Data.SqlServer;

using System.Data.Sql;

namespace NorthwindDAL

{

public class Northwind

{

[SqlFunction(DataAccess = DataAccessKind.Read)]

public static SqlString GetProductNameByID(SqlInt32 productdID)

{

SqlCommand cmd = SqlContext.GetCommand();

cmd.CommandText = "select ProductName " +

"from Products where " +

"ProductID = '" + productdID.ToString() + "'";

return (string) cmd.ExecuteScalar();

}

}

}

The static GetCommand function of the SqlContext object is used to create a SqlCommand object. This SqlCommand object is initialized with the select statement we want to execute. On the last line the select statement is executed using the ExecuteScalar function and the result is returned to the caller.

Before the user-defined function can be made available, the assembly in which it resides must be registered in SQL Server. This is done using the CREATE ASSEMBLY statement as explained in the previous article.

CREATE ASSEMBLY NorthwindDAL

FROM 'C:\Articles\Sql2005\NorthwindDAL\bin\Debug\NorthwindDAL.DLL'

Next we need to register the user-defined function using the CREATE FUNCTION statement.

CREATE FUNCTION dbo.GetProductNameByID(@productId INT)

RETURNS NVARCHAR(40)

AS EXTERNAL NAME NorthwindDAL:[NorthwindDAL.Northwind]::GetProductNameByID

Once this is done the user-defined function can be executed

SELECT dbo.GetProductNameByID(1)

----------------------------------------

Chai

(1 row(s) affected)

Sending data to the client using the SqlPipe object

The second most important object of the SqlServer Data Provider is the SqlPipe object. This object is used to send resultsets and error messages back from the server to the client. A first example is shown in the following code snippet:

public static void HelloWorld()

{

SqlPipe pipe = SqlContext.GetPipe();

pipe.Send("Hello world from .NET");

}

Once the stored procedure is registered, it can be executed from the client and the resulting string 'Hello world from .NET' will be returned.

CREATE PROCEDURE dbo.HelloWorld

AS EXTERNAL NAME NorthwindDAL:[NorthwindDAL.Northwind]::HelloWorld

EXEC HelloWorld

Hello world from .NET

To get an SqlPipe object, the static GetPipe method is first called on the SqlContext object. Once the SqlPipe object is obtained, the Send method can be used to transmit data to the calling client. The Send method has 4 overloaded versions that can be used to either send a String, SqlError, SqlDataReader or a SqlResultSet to the client.

public static void GetProductsByCategoryDataReader(SqlInt32 categoryID)

{

SqlCommand cmd = SqlContext.GetCommand();

cmd.CommandText = "select ProductID, ProductName from products";

SqlDataReader reader = cmd.ExecuteReader();

SqlPipe pipe = SqlContext.GetPipe();

pipe.Send(reader);

}

The previous example represents a stored procedure called GetProductsByCategoryDataReader that uses the Send method on the SqlCommand object to transmit a SqlDataReader to the calling client. The SqlDataReader behaves as a forward-only, read-only cursor that is created on top of the result of the query. This is the most lightweight cursor you can have in SQL Server.

Once the stored procedure is registered it can be executed using the T-SQL EXEC statement. The partial result of the executing is shown below.

CREATE PROCEDURE dbo.GetProductsByCategoryDataReader

(@categoryID INT)

AS EXTERNAL NAME

NorthwindDAL:[NorthwindDAL.Northwind]::GetProductsByCategoryDataReader

GO

EXEC GetProductsByCategoryDataReader 1

ProductID ProductName

----------- ----------------------------------------

17 Alice Mutton

3 Aniseed Syrup

40 Boston Crab Meat

60 Camembert Pierrot

...

Server-side cursors and the SqlResultSet object

The result of query cannot only be send to the client by using a SqlDataReader objects. The SqlServer Data Providers also offers a SqlResultSet object which represents a server-side cursor. With a server-side cursor, the server manages the result set using resources provided by the server computer. The big advantage of a server-side cursor is that it returns only the requested data over the network. However, it is important to point out that a server-side cursor is-at least temporarily-consuming precious server resources for every active client. You must plan accordingly to ensure that your server hardware is capable of managing all of the server-side cursors requested by active clients. So using server-side cursors can have a negative impact on performance and scalability. Using a SqlResultSet is sometimes the least desirable way to access databases from the client.

public static void GetProductsByCategoryResultSet(SqlInt32 categoryID)

{

SqlCommand cmd = SqlContext.GetCommand();

cmd.CommandText = "select ProductID, ProductName from products";

SqlResultSet rs = cmd.ExecuteResultSet(

ResultSetOptions.Scrollable | ResultSetOptions.Updatable);

SqlPipe pipe = SqlContext.GetPipe();

pipe.Send(rs);

}

The ExecuteResultSet method returns a fully scrollable and updateable cursor to the client.

In the next example a stored procedure called UpdatePrices is created. This stored procedure will update the prices of certain products with an specific amount that is passed in as a parameter.

public static void UpdateProductPrices(SqlMoney amount)

{

SqlCommand cmd = SqlContext.GetCommand();

cmd.CommandText = "select * from products";

SqlResultSet rs = cmd.ExecuteResultSet(

ResultSetOptions.Scrollable | ResultSetOptions.Updatable);

while(rs.Read())

{

//retrieve the ProductID column and

//if it is 1 update the price

if (rs.GetInt32(3) == 1)

{

//Update the UnitPrice field

rs.SetSqlMoney(5, rs.GetSqlMoney(5) + amount);

//Update the database

rs.Update();

}

}

rs.Close();

}

Again we create a SqlResultSet that represent a scrollable and updatable cursor. We loop over every record that is in the resultset, and if necessary we use the SetSqlMoney method to update the UnitPrice field if necessary.

.NET User-defined functions and Security

All the managed stored procedures and user-defined functions we wrote until now accessed some database objects. But in managed code you can do much more than accessing database objects like tables. The following user-defined function for example will read the content of a file.

[SqlFunction(DataAccess=DataAccessKind.None)]

public static SqlString ReadFromFile(SqlString filename)

{

StreamReader reader = File.OpenText(filename.ToString());

string content = reader.ReadToEnd();

reader.Close();

return (SqlString)content;

}

If the assembly in which this user-defined function resides is registered using the default CREATE ASSEMBLY statement you will receive a security exception upon execution of the user-defined function.

CREATE FUNCTION dbo.ReadFromFile (@filename nvarchar(100))

RETURNS nvarchar(100)

AS EXTERNAL NAME NorthwindDAL:[NorthwindDAL.Northwind]::ReadFromFile

GO

EXEC dbo.ReadFromFile "c:\data.txt"

.Net SqlClient Data Provider: Msg 6522, Level 16, State 1, Procedure ReadFromFile, Line 0

A CLR error occurred during execution of 'ReadFromFile':

System.Security.SecurityException: Request failed.

at NorthwindDAL.Northwind.ReadFromFile(SqlString filename) +0.

This exception is generated because the default permission set under which all .NET code runs inside SQL Server is set to SAFE. This means that no external resources like files can be accessed. In order to correctly register our .NET assembly we need to execute the following T-SQL statement:

CREATE ASSEMBLY NorthwindDAL

FROM 'C:\Articles\Sql2005\NorthwindDAL\bin\Debug\NorthwindDAL.DLL'

WITH PERMISSION_SET = EXTERNAL_ACCESS

The EXTERNAL_ACCESS permission set grants the NorthwindDAL assembly access to external resources like the file system.

CREATE FUNCTION dbo.ReadFromFile(@filename nvarchar(100))

RETURNS nvarchar(100)

AS EXTERNAL NAME NorthwindDAL:[NorthwindDAL.Northwind]::ReadFromFile

GO

SELECT dbo.ReadFromFile('c:\data.txt')

Execution of the user-defined function returns the content of the file specified in the ReadFromFile call.

----------------------------------------------------------------------

Contents of a simple file...

(1 row(s) affected)

Conclusion

In this second article you saw how you can use the SQLServer Data Provider to access database objects in your managed user-defined functions and stored procedures. The SqlServer Data Provider runs inside the SQL Server 2005 database engine and provides direct access to all database objects.