My Datastage Notes

0 More Next Blog» Create Blog Sign In My Datastage Notes

Pages

Home Surrogate_Key_Stage Different Versions All Datastage Stages Configuration File Surrogate Key Importance: Sequentional_Stage

Dataset SURROGATE KEY IN DATASTAGE Transformer Stage Surrogate Key is a unique identification key. It is alternative to natural key . Sort Stage

Aggregator_Stage And in natural key, it may have alphanumeric composite key but the surrogate is Join Stage Lookup_Stage always single numeric key. Merge_Stage Filter_Stage Surrogate key is used to generate key columns, for which characteristics can be Copy Stage Funnel_Stage specified. The surrogate key generates sequential incremental and unique integers for a Column Generator provided start point. It can have a single input and a single output link. Surrogate_Key_Stage SCD Pivot_Enterprise_Stage Sequence_Activities WHAT IS THE IMPORTANCE OF OF SURROGATE KEY Datastage Study Material/Interview Questions Surrogate Key is a Primary Key for a dimensional table. ( Surrogate key is alternate to Primary Key) The most importance of Datastage Errors and Resolution using Surrogate key is not affected by the changes going on with a database. Datastage Scenarios and solutions Unix Shell Scripting And in Surrogate Key Duplicates are allowed, where it cant be happened in the Primary Key . SQL/Database Datawarehousing Concepts By using Surrogate key we can continue the sequence for any jobs. If any job was aborted at the n records loaded.. By using surrogate key you can continue the sequence from n+1.

Blog Archive

▼ 2014 (53) ► October (7) Surrogate Key Generator: ▼ September (40) The Surrogate Key Generator stage is a processing stage that generates surrogate key columns and maintains the key source. Star schema vs. snowflake schema: Which is better?... A surrogate key is a unique primary key that is not derived from the data that it represents, therefore changes to the data will not change the How to use Aggregate stage to count primary key. In a star schema database, surrogate keys are used to join a fact table to a dimension table. number of reco... surrogate key generator stage uses: Column Export Stage: Create or delete the key source before other jobs run ETL Job Design Standards Update a state file with a range of key values Scenario: Get the max salary from Generate surrogate key columns and pass them to the next stage in the job data file ( Seq ... View the contents of the state file Peek Stage Generated keys are 64 bit integers and the key source can be stat file or database sequence. Scenario: To get the Unique and Duplicates values ... Creating the key source Scenario: Get the next column value Drag the surrogate key stage from palette to parallel job canvas with no input and output links. in current row... Dummy Data Generation using Row Generator Conductor Node in Datastage Sequential File Best Performance Settings/Tips Splitting input files into three different files u... Sequential file with Duplicate Records Scenarios_Unix Double click on the surrogate key stage and click on properties tab. Unix_AWK Unix_SED Unix_Cut UNIX Environmental Variables Other useful UNIX commands UnixFile system security UnixWildcards UnixRedirection UnixSearching the contents of a file UnixDisplaying the contents of a file on the scre... UnixRemoving Files Properties: Unix Move Key Source Action = create

UnixCopy Source Type : FlatFile or Database sequence(in this case we are using FlatFile)

UnixPathnamesListing Directories When you run the job it will create an empty file. Unix Making Directories If you want to the check the content change the View Stat File = YES and check the job log for details. Unix Introduction skey_genstage,0: State file /tmp/skeycutomerdim.stat is empty. RIGHT AND LEFT FUNCTIONS IN TRANSFORMER STAGE WITH... if you try to create the same file again job will abort with the following error. FIELD FUNCTION IN TRANSFORMER STAGE WITH EXAMPLE skey_genstage,0: Unable to create state file /tmp/skeycutomerdim.stat: File exists. SORT STAGE AND TRANSFORMER Deleting the key source: STAGE WITH SAMPLE DATA ... HOW TO CONVERT ROWS INTO THE COLUMNS IN DATASTAGE TRANSFORMER STAGE FOR DEPARTMENT WISE DATA Find Total_Score and Percentage using Transformer ... FIELD FUNCTION IN TRANSFORMER STAGE CONCATENATE DATA USING TRANSFORMER STAGE TRANSFORMER STAGE USING PADSTRING FUNCTION Updating the stat File: TRANSFORMER STAGE USING STRIPWHITESPACES FUNCTION To update the stat file add surrogate key stage to the job with single input link from other stage.

► May (4) We use this process to update the stat file if it is corrupted or deleted. ► February (2) 1)open the surrogate key stage editor and go to the properties tab. ► 2013 (39)

If the stat file exists we can update otherwise we can create and update it.

We are using SkeyValue parameter to update the stat file using transformer stage. Generating Surrogate Keys:

Now we have created stat file and will generate keys using the stat key file.

Click on the surrogate keys stage and go to properties add add type a name for the surrogate key column in the Generated Output Column Name property

Go to ouput and define the mapping like below. Rowgen we are using 10 rows and hence when we run the job we see 10 skey values in the output.

I have updated the stat file with 100 and below is the output.

If you want to generate the key value from begining you can use following property in the surrogate key stage. a. If the key source is a flat file, specify how keys are generated: To generate keys in sequence from the highest value that was last used, set the Generate Key from Last Highest Value property to Yes. Any gaps in the key range are ignored. To specify a value to initialize the key source, add the File Initial Value property to the Options group, and specify the start value for key generation. To control the block size for key ranges, add the File Block Size property to the Options group, set this property toUser specified, and specify a value for the block size.

b. If there is no input link, add the Number of Records property to the Options group, and specify how many records to generate.

No comments: