Overview
We are seeking a knowledgeable Data Engineer for a contract position focused on building a Digital Twin for a law firm, simulating its operations using firm-specific data. The contractor will collaborate closely with a small engineering team, playing a crucial role in designing and maintaining the data architecture that supports simulation and data analysis. This position is fully remote, offering flexibility while contributing to a transformative project.
Responsibilities
- Design, build, and run the data platform to transform raw operational data into usable formats for analysis and modeling.
- Take ownership of data quality from source extraction to final datasets utilized by the modeling team.
- Extract and shape data from new sources, ensuring compatibility with the platform's structures.
- Collaborate with data scientists and software engineers to serve data effectively to applications.
- Ensure that the underlying infrastructure of the data platform is reliable, secure, and cost-effective.
- Contribute to the evolution of the data architecture as the project scales.
Requirements
- Proven experience in building and operating production data pipelines from enterprise operational systems to data warehouses.
- Strong SQL skills with practical experience using Snowflake or a similar cloud warehouse for data modeling and performance optimization.
- Ability to handle and validate messy data from operational databases, including inconsistent data fields and batch timestamps.
- Experience with Python and/or TypeScript for developing pipeline code, with an emphasis on version control and testing practices.
- Familiarity with managing sensitive data, implementing strong access controls and auditability practices.