Database Table structure change problem

More
11 years 11 months ago #4007 by Allen
Hi,

I have several transformations in which the reader is getting data from a database table. Now I want to add more columns to my database table. If I append columns to the table, it will not affect the transformation. However, if I insert columns, my transformation is ruined because it always recognize columns by column index instead of name. For example, I have column A, B and C in sequence in my database table, after some data validation and transformation, I get column 1, 2 ,3. A to 1, B to 2 and C to 3. Now I inserted a column D between A and B, and my database table has A, D, B, C in sequence. However, in my transformations, D replaced B, B replaced C and C is not connected, which end up with A to 1, D to 2, B to 3 (But I actually still want A to 1, B to 2, C to 3 and dont care about D). I can't understand the logic to recognize columns by index. Can we have the transformation recognize columns by name, not by index?

Please Log in or Create an account to join the conversation.

More
11 years 11 months ago #4010 by admin
Can we have the transformation recognize columns by name, not by index?

People can write sql like

select customer.description,
category.description
from
customer,
category

How it supposed to work in this case?

There are several possible solutions.
1 use views instead
2 use select statement as data source
3 put transformation after data reader and use it for mapping only,
when you add new fields use autmap function.

We will have a look from our site what can be done about it as well.
Peter

Mike
ETL Architect

Please Log in or Create an account to join the conversation.