Overview
We are seeking experienced Storage Engineers to take ownership of the storage layer within a large-scale GPU platform at a fast-growing AI infrastructure company. In this fully remote role, you will set the technical direction for storage while remaining hands-on, focusing on the delivery of datasets, model artefacts, and other key components to GPU clusters for various AI processes. This is a unique opportunity to shape a new function within the organization and work closely with cutting-edge technologies in the storage domain.
Responsibilities
- Design and operate distributed storage systems for high-performance computing (HPC) and large-scale GPU environments.
- Build and implement storage solutions for AI training or inference platforms, addressing checkpointing and dataset distribution.
- Develop and maintain high-performance storage architectures using technologies such as VAST, WEKA, Ceph, or MinIO.
- Integrate and automate storage solutions with Kubernetes and scripting languages like Python and Bash.
- Set architectural standards and guide cross-team collaboration while engaging in hands-on technical work.
Requirements
- Proven background in designing and operating distributed storage systems for HPC or large-scale GPU environments.
- Experience building storage solutions for AI training or inference, with a focus on checkpointing and KV-cache mechanisms.
- Hands-on expertise with high-performance storage systems, including VAST, WEKA, Ceph, or MinIO.
- In-depth knowledge of the Linux storage and I/O stack, including NVMe, RDMA/RoCE, and GPUDirect Storage.
- Experience in integrating storage solutions with Kubernetes (CSI) and automating processes using Python and Bash.
- Staff-level experience in setting technical direction and architecture standards while maintaining a hands-on role.