Common task - loading files into the database not so simple as you might think

When we thought about introducing this functionality we decided to use the file system as a data source or target.
For example, File system as a data source provides the following fields

  • [Directory]
  • [File Name]
  • [Size]
  • [Creation Date]
  • [Read Only]
  • [Hidden]
  • [Archive]
  • [File Data]

The file system as a data target uses just two fields

  • [File Name]
  • [File Data]

The next task actually loading files data into blob fields. We had to perform an incredible amount of testing.
For example, data can be loaded into Oracle Database in 4 ways

  • Via ODBC,
  • Via OLE DB,
  • Via Direct path loading
  • And Conventional path loading

Plus there are different versions of ODBC drivers and OLE DB Providers
And of course different versions of the Oracle database.
And Various data types Raw, Long Raw, CLOB, NCLOB, Blob  
and it has to be repeated for every database type our ETL software supports

Once the file is loaded into the blob field we export it back into a different directory and compare files using MD5

Here is the package we used for testing

Loading Image files into blob fields
And here how we tested the results. We calculated and compared MD5 of files
Loading images into blobs comparing files

We started with small files then we moved to large ones, at some stage we were loading setups of our ETL software into the database.

The largest file we loaded was about 100 Meg.

All I can say forget about programming, just use "Advanced ETL Processor".

Mark Brown
MIS Consultant

Direct link, no registration required.
Star InactiveStar InactiveStar InactiveStar InactiveStar Inactive
 
 
Xerox
Swiss banking
Bank Of Oklahoma
Red Cross
Alta Pacific bank
Copeinca
Gas alberta
NHS
Royal Brunei
First Oklahoma bank
Noresco
Iqvia

Testimonials

What customers say about us

  • swissbanking

    I used Advanced ETL Processor in 2 Enterprises for many business processes and Business automation (outside finance department). I did not find any other tool with so many functions and broad flexibility for that Price! If you need support for bugs or solution design you will get it very fast. Best Support I have ever seen.

    Lionel Albrecht
  • iqvia

    IQVIA and DB Software Laboratory (DBSL) partnered in 2010 and have been working in close cooperation ever since. Over this period of time, DBSL software components formed an integral part of a large number of IQVIA applications currently used by over 20 UK NHS Trusts (Hospitals).

    Dmitry Dorsky,
    Director
  • xerox

    The product is easy to learn and once a developer understands the ETL way for solving the problem at hand, the developer's productivity will increase. Even our DBAs now uses the ETL software to quickly create solutions instead of SSIS or SQL jobs.

    Daniel Fung
    Solutions Architect

Read ETL Software customers feedback

This site uses cookies. By continuing to browse the site, you are agreeing to our use of cookies