Overview
We are seeking an experienced MLOps Engineer to play a pivotal role in scaling an on-site advertising platform for a leading global e-commerce client. This role focuses on transforming daily batch processing into real-time capabilities while establishing best practices and governance within a large engineering team. The ideal candidate will work collaboratively, bringing hands-on expertise to enhance MLOps processes and mentor fellow engineers.
Responsibilities
- Design and deploy end-to-end MLOps processes with an emphasis on governance and automation.
- Architect and implement solutions for transitioning high-volume model serving to real-time performance.
- Lead the integration and application of MLflow for model registry and deployment within the Databricks platform.
- Build and automate robust CI/CD pipelines to ensure dependable model releases.
- Profile and optimize large-scale Spark/Python code for production efficiency.
- Act as the technical lead to instill MLOps standards within the Data Engineering team.
Requirements
- Proven experience in designing and implementing end-to-end MLOps processes in a production environment.
- Expert proficiency with Databricks and MLflow.
- Extensive experience in Apache Spark and Python, particularly with large datasets.
- Strong background in GIT for version control and CI/CD pipeline development.
- Excellent SQL skills for data manipulation and querying.