Senior Data engineer

This role involves designing scalable batch and real-time ETL pipelines using PySpark, Structured Streaming, and Delta Lake within a Medallion Architecture. The position requires building autonomous multi-agent workflows, developing RAG pipelines, and integrating vector databases for semantic retrieval. Additionally, the role focuses on managing data governance, security, and operations using Unity Catalog alongside CI/CD workflows.