Sr Data Engineer
Job Description
Summary
Apple’s Software Engineering group is renowned for its work in user experience, integration, functionality and lifestyle. We are responsible for developing and delivering every state-of-the-art operating system, application and firmware to Apple customers. It\'s a critical component of Apple\'s ecosystem and one that continues to break new ground every day!
The Software Engineering team leverages compute resources for a variety of workloads to develop every piece of software. We are looking for a data engineer to bring better visibility to these workloads, partner with engineering teams to drive improvements in their efficiency, and ensure that our costs remain as low as possible. If you’re interested in driving efficiency that enables Apple to develop software at a global scale, then this is the place for you!
The Software Engineering team leverages compute resources for a variety of workloads to develop every piece of software. We are looking for a data engineer to bring better visibility to these workloads, partner with engineering teams to drive improvements in their efficiency, and ensure that our costs remain as low as possible. If you’re interested in driving efficiency that enables Apple to develop software at a global scale, then this is the place for you!
Description
We are seeking a self-motivated individual with strong technical, communication, and project management skills to handle key responsibilities including but not limited to:
- Develop and maintain Tableau dashboards and other analytical tools to provide visibility into compute resourcing
- Collaborate with engineering teams across Software Engineering to leverage shared compute resources wherever possible
- Monitor service expenditures, alert teams of abnormal or inefficient usage, and advise them on improving efficiency
- Automate the data flow between systems such that a realtime view of utilization is visible at all times
- Develop and maintain Tableau dashboards and other analytical tools to provide visibility into compute resourcing
- Collaborate with engineering teams across Software Engineering to leverage shared compute resources wherever possible
- Monitor service expenditures, alert teams of abnormal or inefficient usage, and advise them on improving efficiency
- Automate the data flow between systems such that a realtime view of utilization is visible at all times
Minimum Qualifications
- Bachelor’s or Master’s degree in Computer Science, Information Systems, Software Engineering, Data Science, or a related field
- 5+ years experience in architecting, designing, and developing large scale data solutions
- Deep understanding and strong development experience with distributed data processing frameworks such as Hadoop, Spark and others
Preferred Qualifications
- Experience with monitoring and alerting (e.g. Honeycomb, Splunk, Grafana, or similar)
- Familiarity with Kubernetes and orchestration/containerization technologies
- Expertise in Python, Swift, Unix Shell scripting and Dependency driven job schedulers
- 5+ years experience in building and maintaining large-scale ETL/ELT pipelines (batching and/or streaming) that are optimized for performance and can handle data from various sources, structured or unstructured
- Experienced in writing and maintaining high-quality code using standard methodologies such as code reviews, unit testing, and continuous integration.
- Stay up-to-date with the latest trends and technologies in data infrastructure, architecture, big data analytics, and apply them to improve the system.
- Familiarity with other related fields, such as data science, machine learning, and artificial intelligence, to design solutions that can accommodate advanced analytics.
- Collaborative mindset to work with various teams, including software engineers, financial analysts, and site reliability engineers
- Excellent design and development experience with SQL and NoSQL database, OLTP and OLAP databases
