- Posts: 2
- Thank you received: 0
Data validation using SQL Query
- abuthangam
- Topic Author
- Offline
- New Member
-
Less
More
12 years 9 months ago #1277
by abuthangam
Data validation using SQL Query was created by abuthangam
Hi,
Aplogies if this has been already explained. I am not able to find the info in the forum. Hence, posting it.
I am evaluating Advanced ETL Processor Ent for my Company.
I am loading Customer Master data from Excel file into MySQL database. The source file has got the following fields.
1. EMP_NO
2. COMPANY_NO
3. NAME
4. DOB
...etc
EMP_NO should be checked against EMPLOYEE table to make sure that the Employee does not exist.
COMPANY_NO should be checked against COMPANY table to make sure that the Company exists.
I am trying to implement this check in Validator component. Is this possible? If not, how these validations can be implemented?
Also, is there any option to execute parameterised SQL query, without using Lookup? For example, I would like to execute the below query with the input fields <name> and <company_no> from Excel source, to get the EMP_NO, which will be mapped to an output field
SELECT EMP_NO FROM EMPLOYEE WHERE NAME=<name> AND COMPANY_NO=<company_no>
Thanks & Regards,
Syed.
Aplogies if this has been already explained. I am not able to find the info in the forum. Hence, posting it.
I am evaluating Advanced ETL Processor Ent for my Company.
I am loading Customer Master data from Excel file into MySQL database. The source file has got the following fields.
1. EMP_NO
2. COMPANY_NO
3. NAME
4. DOB
...etc
EMP_NO should be checked against EMPLOYEE table to make sure that the Employee does not exist.
COMPANY_NO should be checked against COMPANY table to make sure that the Company exists.
I am trying to implement this check in Validator component. Is this possible? If not, how these validations can be implemented?
Also, is there any option to execute parameterised SQL query, without using Lookup? For example, I would like to execute the below query with the input fields <name> and <company_no> from Excel source, to get the EMP_NO, which will be mapped to an output field
SELECT EMP_NO FROM EMPLOYEE WHERE NAME=<name> AND COMPANY_NO=<company_no>
Thanks & Regards,
Syed.
Please Log in or Create an account to join the conversation.
12 years 9 months ago - 12 years 9 months ago #1280
by admin
Mike
ETL Architect
Replied by admin on topic Re: Data validation using SQL Query
Hi
1 Use two InList functions
A ) EMP_NO should be checked against EMPLOYEE table to make sure that the Employee does not exist.
use
select distinct EMP_NO from EMPLOYEE
as a source
B ) COMPANY_NO should be checked against COMPANY table to make sure that the Company exists.
use
select distinct COMPANY_NO from COMPANY
as a source
2 It is not possible at the moment but we will add support for it to the futurure release
In most of the cases lookup is faster than running select lookup_value from lookup_table
Imaging doing it for 2M records file
Mike
1 Use two InList functions
A ) EMP_NO should be checked against EMPLOYEE table to make sure that the Employee does not exist.
use
select distinct EMP_NO from EMPLOYEE
as a source
B ) COMPANY_NO should be checked against COMPANY table to make sure that the Company exists.
use
select distinct COMPANY_NO from COMPANY
as a source
2 It is not possible at the moment but we will add support for it to the futurure release
In most of the cases lookup is faster than running select lookup_value from lookup_table
Imaging doing it for 2M records file
Mike
Mike
ETL Architect
Last edit: 12 years 9 months ago by admin.
Please Log in or Create an account to join the conversation.