1.5. REDLIN - Live Lineage Concept

1.5.1. Overview

The Live Lineage feature provides an up-to-date view of data dependencies by generating lineage information dynamically. This ensures that the most recent changes in the environment are reflected in the lineage graph.

1.5.2. Hybrid Approach for Performance

To achieve live lineage generation efficiently without sacrificing performance, REDLIN employs a hybrid approach:

1. Current Metadata: Live lineage actively queries and utilizes the most current metadata available in the system. This guarantees that newly created or modified objects are accurately represented.

2. Pre-parsed Base Data: To ensure rapid response times, the live lineage process does not parse the entire data warehouse from scratch. Instead, it leverages the pre-calculated base data generated during the standard REDLIN Batch process for the Business View (BV) release.

By combining the pre-parsed foundational lineage with the current live metadata, REDLIN avoids redundant, computationally expensive parsing operations while delivering an accurate and instantaneous lineage view.

1.5.3. Architecture Diagram

The following diagram illustrates the interaction between the batch process, current metadata, and the Live Lineage generator:

            graph TD
        subgraph Batch Process
            A[Data Warehouse / Source Systems] -->|Parse & Extract| B(REDLIN Batch)
            B --> C[(Base Data / BV Release)]
        end

        subgraph Live Environment
            D[(Current Metadata)]
        end

        C -.->|Fast Retrieval| E{Live Lineage Generator}
        D -->|On-the-fly Updates| E

        E --> F[Live Lineage Output / Diagram]

        style E fill:#f9f,stroke:#333,stroke-width:2px
        style C fill:#ccf,stroke:#333,stroke-width:1px
        style D fill:#cfc,stroke:#333,stroke-width:1px