In most cases parallel jobs and stages look similiar to the Datastage Server objects, however their capababilities are way different. Parallel jobs support a completely new set of stages, which implement the scalable and parallel data processing mechanisms. The major difference between Infosphere Datastage Enterprise and Server edition is that Enterprise Edition (EE) introduces Parallel jobs.The job developer only chooses a method of data partitioning and the Datastage EE engine will execute the partitioned and parallelized processes.ĭifferences between Datastage Enterprise and Server Edition The concept is hidden from a Datastage programmer. The key concept of ETL Pipeline processing is to start the Transformation and Loading tasks while the Extraction phase is still running.ĭatastage Enterprise Edition automatically combines pipelining, partitioning and parallel processing. Pipelining means that each part of an ETL process (Extract, Transform, Load) is executed simultaneously, not sequentially.
This means for instance that once the data is evenly distributed, a 4 CPU server will process the data four times faster than a single CPU machine.
The main outcome of using a partitioning mechanism is getting a linear scalability. Each partition of data is processed by the same operation and transformed in the same way. Partitioning means breaking a dataset into smaller sets and distributing them evenly across the partitions (nodes).
The EE architecture is process-based (rather than thread processing), platform independent and uses the processing node concept.ĭatastage EE is able to execute jobs on multiple CPUs (nodes) in parallel and is fully scalable, which means that a properly designed job can run across resources within a single machine or take advantage of parallel platforms like a cluster, GRID, or MPP architecture (massively parallel processing). Key Datastage Enterprise Edition conceptsĭatastage jobs are highly scalable due to the implementation of parallel processing. Infosphere Datastage 8 tutorial and certification materials now available on ETL-Tools.Info! Infosphere Datastage EE tutorial - Datastage and Qualitystage tutorial based on Information Server 8.1 and Datastage 7.5 EEĭatastage certification - Infosphere Datastage, Qualitystage and Information Analyzer certification materials and study guides (000-415, 000-416, 000-417, 000-418, 000-419 IBM exams)
0 kommentar(er)
