About This Blog

.net & SQL Samples, programming tips and tricks, performance tips, guidelines, and best practices

Sunday, 22 May 2011

Creating Tally Tables using CTE in SQL Server 2005/2008


A Tally table is simply a table with a single column of very well indexed sequential values starting at any value of your requirement and going up to some predefined threshold. The starting value & the ending value should not be arbitrary. It should be as per the requirement in question.

There are n numbers of ways in which a Tally Table could be generated and the most obvious one is by making use of loops.

Here, I would demonstrate how we could generate a Tally Table using CTE. We will make use of the recursive nature of CTEs to get our job done.

Generating a sequence of numbers from 1 to 20

DECLARE @Max AS INT = 20

;WITH CTE AS (
  SELECT 1 Num
  UNION ALL
  SELECT Num + 1 FROM CTE WHERE Num < @Max
)

SELECT * FROM CTE


Generating a sequence of Dates starting with the current date & going till next 20 days

DECLARE @MaxDate AS DATETIME = GETDATE() + 20

;WITH CTE AS (
  SELECT GETDATE() Dates
  UNION ALL
  SELECT Dates + 1 FROM CTE WHERE Dates < @MaxDate
)

SELECT * FROM CTE


I love this way of generating the Tally Tables as it’s just too simple and on the fly.

Tuesday, 10 May 2011

How do we read the SQL Server error log?


Well there is an undocumented system procedure to do it sp_readerrorlog.This procedure reads from the disk using an extended stored procedure. To use it you need to be in the security admin group. Or you will get -
Msg 15003, Level 16, State 1, Line 1
Only members of the securityadmin role can execute this stored procedure.
The parameters for sp_readerrorlog are -
@p1 = Log file to read, this is the number, if you specify 0 you will get the current file ERRORLOG 
@p2 = Type of log to read, SQL Server (1) or SQL Agent (2)
 
@p3 = Some text to find in the log entries
 
@p4 = Some more text to find in the log entries (essentially the result is @p3 AND @p4)
All of the above stated parameters are optional and the default values for each are –
@p1 = 0 (ZERO); will get the current file ERRORLOG 
@p2 = SQL Server (1)
 
@p3 =
 NULL
@p4 = NULL
Using this information we can do the following
EXEC sp_readerrorlog 0, 1, 'SEARCH TEXT 1',' SEARCH TEXT 2'
This only reads one file. If the files have been cycled or the server restarted we will miss our change. We can use a second undocumented procedure to get the list of log files - sp_enumerrorlogs
This takes 1 optional parameter
@p1 = Type of log to read, SQL Server (1) or SQL Agent (2). Same as @p2 for sp_readerrorlog. This defaults on 1 if not specified.
Using these together we can go through the logs looking for our changes.

Prevent recursive CTE from entering an infinite loop


Common Table Expression (CTE) is not a new feature now. It has been around with the launch of SQL Server 2005. Since then, we might have come across many powerful examples of using CTE and have made the most out of it. The best use of CTE I take is through recursion. We can create recursive CTE to get our job done very easily in situations where we need to repeatedly refer a table/view.

The most common situation in which developers land up is an infinite loop for a badly build CTE. So, to counter this situation, we can make use of MAXRECURSION option.
Let’s give it a hands on..



USE AdventureWorks
GO

;WITH EmpCTE
AS
(
  SELECT
    EmployeeID, ManagerID
  FROM
    HumanResources.Employee
  WHERE
    ManagerID IS NULL
  UNION ALL
  SELECT
    e.EmployeeID, e.ManagerID
  FROM
    HumanResources.Employee e
  INNER JOIN
    EmpCTE cte
  ON cte.EmployeeID = e.ManagerID
)
SELECT * FROM EmpCTE OPTION (MAXRECURSION 4)
GO



Now if your CTE goes beyond 4th recursion it will throw an error and stop executing. And if you happen put MAXRECURSION value too low, it might be possible that before your desired result is accomplished it will throw an error.
For example if you change MAXRECURSION to value 3. It will throw following error.
Msg 530, Level 16, State 1, Line 1
The statement terminated. The maximum recursion 3 has been exhausted before statement completion.

In short, MAXRECUSION is good way to prevent recursive CTE to enter an infinite loop.

Friday, 6 May 2011

Find all empty tables in SQL Server Database


Yesterday I was working on one of my Databases and wanted to find out all the empty tables in that. Suddenly, I realized that there is no direct way to get a list of all those tables; even SSMS also doesn’t help much for this.

After googling for a few minutes, I decided to use one of the undocumented stored procedure - sp_Msforeachtable.

To get the list of empty tables, we can use the below tsql –
EXEC sp_MSforeachtable 'IF NOT EXISTS (SELECT 1 FROM ?) PRINT ''?'' '

And, to get a list of tables having at least one row of data, we can use the below tsql –
EXEC sp_MSforeachtable 'IF EXISTS (SELECT 1 FROM ?) PRINT ''?'' '

Please note that the sp_MSforeachtable stored procedure is undocumented, so be careful about using this stored procedure in production code.

Monday, 25 April 2011

Rebuild all the indexes of a table & Indexed views in a Database


Rebuilding an index drops and re-creates the index. This removes fragmentation, reclaims disk space by compacting the pages based on the specified or existing fill factor setting, and reorders the index rows in contiguous pages. When ALL is specified, all indexes on the table/view are dropped and rebuilt in a single transaction. FOREIGN KEY constraints do not have to be dropped in advance.
In general, rebuilding or reorganizing small indexes often does not reduce fragmentation. The pages of small indexes are stored on mixed extents. Mixed extents are shared by up to eight objects, so fragmentation in a small index might not be reduced after reorganizing or rebuilding the index.
To REBUILD all the indexes, the following syntax is sufficient –
ALTER INDEX ALL ON TableName REBUILD WITH (FILLFACTOR = Any number between 0 to 100);

Before attempting to REBUILD, we should first find the level of fragmentation using the below tsql –
SELECT * FROM sys.dm_db_index_physical_stats(db_id(), object_id('tablename'), NULL, NULL , 'DETAILED')
GO

Microsoft Recommends
REBUILD IF > 30% framgmented
REORGANIZE IF >5 % and < 30% framgmented

I have prepared the following TSQL to rebuild all the indexes on the all the tables and all the indexed views of the database in use.

USE DatabaseName
GO

DECLARE @tsql NVARCHAR(MAX) 
DECLARE @fillfactor INT

SET @fillfactor = 90

SELECT @tsql =
  STUFF(( SELECT DISTINCT
           ';' + 'ALTER INDEX ALL ON ' + o.name + ' REBUILD WITH (FILLFACTOR = ' + CONVERT(VARCHAR(3),@fillfactor) + ')'
          FROM
           sysobjects o
          INNER JOIN sysindexes i
           ON o.id = i.id
          WHERE
           o.xtype IN ('U','V')
           AND i.name IS NOT NULL
          FOR XML PATH('')), 1,1,'')

--PRINT @tsql         
EXEC sp_executesql @tsql

Wednesday, 13 April 2011

Pass a collection of object as Table Valued Parameter in SQL Server 2008


Introduction

In my last post on Table Valued Parameters, I discussed about the concept and demonstrated with a small example how to implement them. The example I gave used an object of DataTable as a table valued parameter. However, I have come across many queries on “How to pass a collection as a Table Valued Parameter?” My this post will answer this question.

Many of our first reaction to the question will be – “what is the big issue in that” and to a great extent it’s very much correct. However, when we actually start implementing it, it is then that we realize that we are missing something and it is nothing but IEnumerable<SqlDataRecord> interface. All what we need to do is inherit our collection from IEnumerable<SqlDataRecord> interface. This interface requires our collection class to implement a C# custom iterator method named GetEnumerator which ADO.NET will call for each object contained in the collection when you invoke ExecuteNonQuery or any other execute method. Please note that VB .NET doesn’t support custom iterators.

Implementation
First we’ll define the Order and OrderDetail classes and properties as under:
  public class Order
  {
    public int OrderId { get; set; }
    public string OrderNo { get; set; }
    public int CustomerId { get; set; }
    public DateTime Date { get; set; }
  }

  public class OrderDetail
  {
    public int OrderId { get; set; }
    public int ItemId { get; set; }
    public decimal Quantity { get; set; }
    public decimal Price { get; set; }
    public decimal Discount { get; set; }
  }
Generally, List<Order> and List<OrderDetail> objects serves as collections of Order and OrderDetail objects in our application. But these collections, by default won’t support on their own as input values for TVPs because List<T> doesn’t implement IEnumerable<SqlDataRecord>. Hence, we need to add that ourselves. So we’ll define Orders and OrderDetails classes that inherit List <Order> and List <OrderDetail> respectively, and also implement IEnumerable<SqlDataRecord> to “TVP-enable” them:
  public class Orders : List<Order>, IEnumerable<SqlDataRecord>
  {
    IEnumerator<SqlDataRecord> IEnumerable<SqlDataRecord>.GetEnumerator()
    {
      var sdr = new SqlDataRecord(
      new SqlMetaData("OrderId", SqlDbType.Int),
      new SqlMetaData("OrderNo", SqlDbType.VarChar),
      new SqlMetaData("CustomerId", SqlDbType.Int),
      new SqlMetaData("Date", SqlDbType.Date));

      foreach (Order o in this)
      {
        sdr.SetInt32(0, o.OrderId);
        sdr.SetString(1, o.OrderNo);
        sdr.SetInt32(2, o.CustomerId);
        sdr.SetDateTime(3, o.Date);

        yield return sdr;
      }
    }
  }

  public class OrderDetails : List<OrderDetail>, IEnumerable<SqlDataRecord>
  {
    IEnumerator<SqlDataRecord> IEnumerable<SqlDataRecord>.GetEnumerator()
    {
      var sdr = new SqlDataRecord(
      new SqlMetaData("OrderId", SqlDbType.Int),
      new SqlMetaData("ItemId", SqlDbType.Int),
      new SqlMetaData("Quantity", SqlDbType.Decimal),
      new SqlMetaData("Price", SqlDbType.Decimal),
      new SqlMetaData("Discount", SqlDbType.Decimal)
      );


      foreach (OrderDetail od in this)
      {
        sdr.SetInt32(0, od.OrderId);
        sdr.SetInt32(1, od.ItemId);
        sdr.SetDecimal(2, od.Quantity);
        sdr.SetDecimal(3, od.Price);
        sdr.SetDecimal(4, od.Discount);

        yield return sdr;
      }
    }
  }
I’ll only explain the Orders class; you can then infer any of your own collection classes – implements the custom iterator needed to support TVPs.
First, it inherits List<Order>, so with an Orders object we can do everything that could be done with List<Order>object. It implicitly also implements IEnumerable<Order> and explicitly implements IEnumerable <SqlDataRecord> which means it also has a customer iterator method for ADO.NET to consume when an instance of this collection class is assigned to a SqlDbType.Structured parameter for piping over to SQL Server with a TVP.
Implementing IEnumerable<SqlDataRecord> requires implementing a GetEnumerator method that returns an IEnumerator <SqlDataRecord>. This method first initializes a new SqlDataRecord object with a similar schema of UDTTs that the TVPs are declared as. It then enters a loop that iterates all the elements in the collection. On the first iteration, it sets the column property values of the SqlDataRecord object to the property values of the first Order element, and then issues the yield return statement. Any method which returns IEnumerator<T> and has a yield return statement in it, is a custom iterator method that is expected to return a sequence of objects until the method execution path completes (in this case, when the foreach loop finishes).
When we invoke ExecuteNonQuery to run a stored procedure with a SqlDbType.Structured parameter (that is, a TVP), ADO.NET expects the collection passed for the parameter value to implement IEnumerable <SqlDataRecord> so that IEnumerable<SqlDataRecord>.GetEnumerator can be called internally to fetch each new record for piping over to the server. This method is never called directly.
When the first element is fetched from the collection, GetEnumerator is entered, the SqlDataRecord is initialized and is then populated with values using the Set methods (there’s a SetXXX method for each data type). That SqlDataRecord “row” is then pushed into the pipeline to the server by yield return. When the next element is fetched from the collection, the GetEnumerator method resumes from the point that it yield returned the previous element, rather than entering GetEnumerator again from the top. This means the SqlDataRecord gets initialized with schema information only once, while its population with one element after another is orchestrated by the controlling ADO.NET code for ExecuteNonQuery that actually ships one SqlDataRecord after another to the server.