Static vs Dynamic Data Workflows

Static Workflow

Static workflows hardcode rules into the execution logic. Dynamic workflows query current rules at runtime.

def deduplicate_records():
    # Strategy hardcoded
    return db.query("""
        SELECT DISTINCT ON (email) *
        FROM user_events
        ORDER BY email, created_at DESC
    """)

This runs the same way every time. If the deduplication strategy changes (now use composite key on email + device_id), the code must change. This is why static pipelines break when business logic evolves.

Dynamic Workflow

def deduplicate_records():
    # Query current rules
    rules = context.get("deduplication_strategy")

    # rules.key_fields: ["email", "device_id"]
    # rules.sort_priority: "created_at DESC"

    return db.query(f"""
        SELECT DISTINCT ON ({', '.join(rules.key_fields)}) *
        FROM user_events
        ORDER BY {', '.join(rules.key_fields)}, {rules.sort_priority}
    """)

Same workflow code. Different execution based on current deduplication strategy. When rules update, behavior changes without redeployment. This approach requires separating decision logic from pipeline code.

What Actually Changes