Extracting 50 Million records

Hi,

I would like to know whether it is wise to extract data from 50 million data table to get 5000 records through a transformations.
I am worried about the performance related issue. I am also sure that the most of the conditioned column is not indexed.

Please advice,
James

Your question has nothing to do with our software but we will hep anyway.

>>>I am also sure that the most of the conditioned column is not indexed.

Even if some of the conditional fields are indexed it should help to sped it up.

Make sure that you are using select statement as data source

Also I do not recommend using OleDB connections for extraction because you may run out of memory.

When you run it first time set a limit of rows to fetch and write down the the time taken to process it.

It will give you an idea how long it will take to process all records.

Also If I was you I would run it outside normal working hours

Hope that helps,
Peter