Senior Software Engineer, Data Engineering
LMArena(1 month ago)
About this role
A Senior Software Engineer in Data Engineering at LMArena will design and build data pipelines and infrastructure crucial for analyzing user vote data to evaluate AI model performance. The role involves collaborating with researchers and product leaders to enhance data quality, scale infrastructure, and generate insightful dashboards, directly impacting AI development. Candidates should have over 5 years of experience in software engineering with a focus on data engineering, proficiency in SQL and Python, and familiarity with big data frameworks like Apache Spark.
Required Skills
- Data Engineering
- Data Pipelines
- SQL
- Python
- Apache Spark
- Data Processing
- Big Data Technologies
- Data Integrity
- Data Quality
- Production Optimization
+5 more
Qualifications
- 5+ years of experience in software engineering, with a dedicated focus on data engineering and big data technologies
- Proficiency in SQL and at least one programming language commonly used for data analysis (Python (preferred), Scala, R)
- Hands-on experience with data processing and pipeline frameworks (Apache Spark, Ray Data, etc.)
- Experience with at least one popular big data analytics platform (Databricks, Snowflake)
- Demonstrated experience in designing, implementing, optimizing, and debugging production data pipelines
About LMArena
www.lmarena.aiLMArena is an innovative platform designed for evaluating AI through human preferences. It allows users to compare various AI models, gather insights, and share their feedback to enhance AI development and research. The platform features a public leaderboard, providing a competitive edge for the best AI models based on user interactions and performance assessments. LMArena serves as a comprehensive resource for those looking to explore and engage with advanced AI technologies.
