Posts

Showing posts with the label Flat-File

SSIS - automatically setting Flat File columns data types

Image
Problem: Have a large number of (TSV) files that need to be imported (periodically) (via SSIS package) into existing MSSQL DB tables. Getting many data type issues from the  OLE DB Destination  tasks eg: [Flat File Source [2]] Error: Data conversion failed. The data conversion for column "PRC_ID" returned status value 2 and status text "The value could not be converted because of a potential loss of data.". and the type suggestions from the connection managers for each table from  Flat File Source  tasks are not accurate enough to prevent errors when running the import package (and the DB types are the correct ones, so don't want to just make them all (wrongly) strings for the sake of loading the TSVs). Is there a way to load the type data for the columns in some single file rather than one by one in the connection manager window for the  Flat File Source  tasks (this would be hugely inconvenient as each table may have many fields)? I have th...

SSIS - Flat File Source Handling NUL (\x00) value

Image
Problem: I am trying to load data from text files to database. My source files contain null character  NUL somehow (Picture1). I just make all the fields as one column (delimited with  {CR}{LF} ). Then I do the preview of the data. The data is just what we need. But then when I run the package, the data changed, not like what I see in data preview. I added a data viewer to see the data. The number 1 disappear in the first row (see the red). It seems that flat file reading ends at  NUL character. But my Row delimiter is  {CR}{LF} , it doesn't make sense the number 1 in the end disappear. Can anyone tell me why is that? Solution: Reproducing the error First of all, I would like to show the steps to reproduce this error using  Notepad++ editor . I created a text file called  TestNUL  that contains data similar to the screenshot posted in the question (commas are placed where  NUL  objects should be): Now, Go To Edit...

SSIS - Fill SQL database from a CSV File

Image
Problem: I need to create a database using a CSV file with SSIS. The CSV file includes four columns: I need to use the information of that table to populate the three tables I created in SQL below. I have realized that what I need is to use one column of the Employee Table,  EmployeeNumber , and Group Table,  GroupID , to populate the EmployeeGroup table. For that, I thought that a Join Merge table is what I needed, but I created the Data Flow Task in SSIS, and the results are the same, no data displayed. The middle table is the one used to relate the other tables. I created the package in SSIS and the Employee and Group Tables are populated, but the EmployeeGroup table is not. EmployeeGroup will only show the EmployeeNumber and Group ID columns with no data. I am new using SSIS, and I really do not know what else to do. I will really appreciate your help. Solution: Overview Solutions using SSIS Using 3 Data Flow Tasks Using 2 Data Flow Tasks Solu...