Is there a WHERE Clause feature in SSIS for a Flat File Source?

Question

I don't know too much about SSIS and I've googled as much as I could on this. I'd really appreciate any help I could get.

I'm building an SSIS package and I only want to insert a specific set of data from a Flat File Source. Is there a way to use a WHERE clause to only pull specific data in?

For example, how can I make this query into a Data Flow Task?

INSERT INTO #TempTable (Column1, Column2)
SELECT Column1, Column2
FROM TEXTFILESOURCE
WHERE Column1 <> 'ABC'

Can this be achieved through Data Flow Task?

Registered

Answer 1

AFAIK there is nothing in the Flat File Source task that can do this, but once you have that set up and the data flowing in, you can use a Conditional Split to filter the rows using the same logic as your Where condition in your query.

Inside

I was under the impression that Conditional Split is mostly just a CASE WHEN statement. You can eliminate data altogether using this filter? So Object Name would be WHERE 1 and Expression would be Column1 <> 'ABC' ? – Registered

Yes, you can send the unwanted rows to another task where you do not have to process them - a Row Count task for example. Then continue processing the rows you are interested in. You only need one condition, as the non-matching rows will be output to the default output flow. – Inside

That did the trick, just had to learn a bit more about the SSIS Expression Language to get the syntax right. Thanks again, I appreciate it! – Registered

Answer 2

Either a conditional split, with your filter conditions as an expression, or, a script task using C# but that would be an overkill if the flat file data is consistent enough you could just use conditional split.

Concordat