Home > SSIS Best Practices > SSIS #104 – When Row Count Transformation is invaluable

SSIS #104 – When Row Count Transformation is invaluable

The Row Count transformation is a pretty straightforward transformation. It does the following two things:

1. It counts rows as data passes through a data flow, and

2. it stores the final count in a variable. (Note that the variable is only populated when the data flow is finished, not while the data flow is still in the middle of transferring data.)

image

Here is a data flow design pattern I use very often.

image

At the end of the data flow, my variable @vNewQueueCount is populated with the row count from the data source.

Then I use this variable @vNewQueueCount in a Precedence Constraint expression, as shown below, to decide conditionally whether I need to process my Queue dimension or not.

image

Configuring the Row Count transformation is simply enough. I’ll show you that later in this blog.

What I want to say first is why the Row Count transformation is invaluable in this specific design pattern.

In this design, I’ve decided to use the Recordset Destination, which is a in-memory ADO dataset and it saves data in memory in a recordset that is stored in an Integration Services package variable of the Object data type. In my next blog, I’ll show you how to use a Foreach Loop container with the Foreach ADO enumerator to process one row of the recordset at a time.

The Recordset Destination is a handy component to use when you do not need to (nor want to) persist your data. One drawback of using the in-memory ADO dataset is that Integration Services does not provide any out-of-box methods to get the row count in the dataset. This is when Row Count transformation becomes invaluable to me. Without too much work on my side, I am able to accomplish the following:

1. Retrieve data from the source, and store the data in a, ADO dataset.

2. Also store the row count in a variable.

3. Finally, be able to use the row count variable to conditionally decide whether I need to process my dimension or not.

Here are a few simple steps to configure the Row Count transformation.

1. Define a variable vNewQueueCount.

 image

2. Create a new data flow

image

3. Drag the Row Count transformation to the data flow design surface. On the Advance Editor, for VariableName select User::vNewQueueCount.

image

Finally, you can use the variable in a precedence constraint expression to do conditional processing:

@vNewQueueCount > 0

image

Have fun with the Row Count transformation!

Categories: SSIS Best Practices
  1. No comments yet.
  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: