Shelf is looking for a Senior Backend Software Engineer to build robust backend services for large-scale data processing, ensuring data flows efficiently and remains well-orchestrated. The ideal candidate will have over 4 years of professional software engineering experience, including 1 year specializing in Python, and a deep understanding of distributed systems and ETL-oriented workflows.
Requirements
- Design, implement, and optimize our distributed ETL pipeline, focusing on background processing logic, data transformation, and scalability.
- Develop modular and composable components capable of efficiently processing large-scale data across a diverse range of storage solutions, including S3, RDS/PostgreSQL, Elasticsearch, DynamoDB, data warehouses, and data lakes.
- Implement ML model integrations within the data pipeline, working closely with Data Scientists on model deployment and data flow.
- Develop clean, maintainable code in Python, adhering to best practices in observability, cost-efficiency, and robust error handling.
- Proactively identify and address performance bottlenecks and inefficiencies in current systems, proposing solutions to improve scalability and reliability, while ensuring continuous production stability through thorough testing and monitoring practices.
- Share your knowledge, participate in code reviews, and advocate for best practices to advance our backend development standards.
Benefits
- B2B contract
- Company Stock Options
- Hardware: MacBook Pro
- Modern technical stack
- Develop open-source software
- GitHub Copilot subscription
- LLM credits for other OSS AI code assistants and internal AI tools