Skip to main content

Page loading completed.

Lead Data Engineer

21/10/2024
26/11/2024
Permanent - Full Time
Hyderabad
Data Engineering

Who is Quantium? 

 

Quantium is a world leader in data science and artificial intelligence. Established in Australia in 2002, Quantium is a global team of more than 1,100 people across 14 locations with a unique blend of capabilities across product and consulting services to help businesses unlock value from data and analytics. Quantium partners with the world’s largest corporations to forge a better, more insightful world. 

 

Data Engineering Team

 

As a Lead Data Engineer for India, you will be accountable for leading the technical aspects of product engineering by being hands on, working on the enhancement, maintenance and support of the product on which your team is working, within your technology area. You will be responsible for your own hands-on coding, provide the design thinking and design solutions, ensuring the quality of your team’s output, representing your team in product-level technical forums and ensuring your team provides technical input to and aligns with the overall product road-map.

 

How will you make an impact?  

 

You will work with Engineers in other technology areas to define the overall technical direction for

the product on alignment with Group’s technology roadmap, standards and frameworks, with

product owners and business stakeholders to shape the product's delivery roadmap and with

support teams to ensure its smooth operation. You will be accountable for the overall technical

quality of the work produced by India that is in line with the expectation of the stakeholders, clients

and Group.

You will also be responsible for line management of your team of Engineers, ensuring that they

perform to the expected levels and that their career development is fully supported.

 

Key responsibilities

 

o Produce Quality Code o Code follows team standards, is structured to ensure readability and

maintainability and goes through review smoothly, even for complex changes

o Designs respect best practices and are favourably reviewed by peers

o Critical paths through code are covered by appropriate tests

o High-level designs / architectures align to wider technical strategy, presenting reusable APIs where

possible and minimizing system dependencies

o Data updates are monitored and complete within SLA

o Technical designs follow team and group standards and frameworks, is structured to ensure

reusability, extensibility and maintainability and goes through review smoothly, even for complex

changes

o Designs respect best practices and are favourably reviewed by peers

o High-level designs / architectures align to wider technical strategy, presenting reusable APIs where

possible and minimizing system dependencies

o Estimates are consistently ‘challenging, but realistic’

o Most tasks are delivered within estimate

o Complex or larger tasks are delivered autonomously

o Sprint goals are consistently achieved

o Demonstrate commitment to continuous improvement of squad activities

o The product backlog is consistently well-groomed, with a responsible balance of new features and

technical debt mitigation

o Other Engineers in the Squad feel supported in their development

o Direct reports have meaningful objectives recorded in Quantium's Performance Portal, and

understand how those objectives relate to business strategy

o Direct reports' career aspirations are understood / documented, with action plans in place to move

towards those goals

o Direct reports have regular catch-ups to discuss performance, career development and their ongoing happiness / engagement in their role

o Any performance issues are identified, documented and agreed, with realistic remedial plans in

place

o Squad Collaboration

o People Management

o Produce Quality Technical Design

o Operate at high level of productivity

 

Key activities

 

• Build technical product/application engineering capability in the team by that is in line with

the Group’s technical roadmap, standards and frameworks

• Write polished code, aligned to team standards, including appropriate unit / integration tests

• Review code and test cases produced by others, to ensure changes satisfy the associated

business requirement, follow best practices, and integrate with the existing code-base

• Provide constructive feedback to other team members on quality of code and test cases

• Collaborate with other Lead / Senior Engineers to produce high-level designs for larger pieces

of work

• Validate technical designs and estimates produced by other team members

• Merge reviewed code into release branches, resolving any conflicts that arise, and

periodically deploy updates to production and non-production environments

• Troubleshoot production problems and raise / prioritize bug tickets to resolve any issues

• Proactively monitor system health and act to report / resolve any issues

• Provide out of hours support for periodic ETL processes, ensuring SLAs are met

• Work with business stakeholders and other leads to define and estimate new epics

• Contribute to backlog refinement sessions, helping to break down each epic into a collection

of smaller user stories that will deliver the overall feature Work closely with Product Owners

to ensure the product backlog is prioritized to maximize business value and manage

technical debt

• Lead work breakdown sessions to define the technical tasks required to implement each

user story

• Contribute to sprint planning sessions, ensuring the team takes a ‘realistic but challenging’

amount of work into each sprint and each team member will be productively occupied

• Contribute to the team’s daily stand-up, highlighting any delays or impediments to progress

and proposing mitigation for those issues

• Contribute to sprint review and sprint retro sessions, to maintain a culture of continuous

improvement within the team

• Coach / mentor more junior Engineers to support their continuing development

• Set and periodically review delivery and development objectives for direct reports

• Identify each direct reports’ longer-term career objectives and, as far as possible, factor this

into work assignments

• Hold fortnightly catch-ups with direct reports to review progress against objectives, assess

engagement and give them the opportunity to raise concerns about the product or team

• Work through the annual performance review process for all team members

• Conduct technical interviews as necessary to recruit new Engineers

 

The superpowers you’ll be bringing to the team:  

 

8+ years of experience in design, develop, and implement end-to-end data solutions (storage,

integration, processing, access) in Google Cloud Platform (GCP) or similar cloud platforms.

2. Strong experience with SQL

3. Values delivering high-quality, peer-reviewed, well-tested code

4. Create ETL/ELT pipelines that transform and process terabytes of structured and

unstructured data in real-time

5. Knowledge of DevOps functions and to contribute to CI / CD pipelines

6. Strong knowledge of data warehousing and data modelling and techniques like dimensional

modelling etc

7. Strong hands-on experience with BigQuery/Snowflake, Airflow/Argo, Dataflow, Data catalog,

VertexAI, Pub/Sub etc or equivalent products in other cloud platforms

8. Solid grip over programming languages like Python or Scala

9. Hands on experience in manipulating SPARK at scale with true in-depth knowledge of SPARK

API

10. Experience working with stakeholders and mentoring experience for juniors in the team is

good to have

11. Recognized as a ‘go-to’ person for high-level designs and estimations

12. Experience working with source control tools (GIT preferred) with good understanding of

branching / merging strategies

13. Experience in Kubernetes and Azure will be an advantage

14. Understanding of GNU/Linux systems and Bash/scripting

15. Bachelor’s degree in Computer Science, Information Technology or a related discipline

16. Comfortable working in a fast moving, agile development environment

17. Excellent problem solving / analytical skills

18. Good written / verbal communication skills

19. Commercially aware, with the ability to work with a diverse range of stakeholders

20. Enthusiasm for coaching and mentoring junior engineers

21. Experience in lading teams, including line management responsibilities

 

What could your Quantium Experience look like? 

 

Working at Quantium will allow you to challenge your imagination. You will get to solve complex problems using rigor, precision and by asking great questions – but it also means you can think big, outside the box and push your problem-solving skills to the max. 

 

 

 

By joining the Quantium team, you’ll get to: 

 

 

 

Forge your path: So many of our team have moved around different teams or offices. You’ll be in the driver’s seat, and we empower you to make your career your own. 

Find your kind: Embrace diversity and connect with your tribe (think foodies, dog lovers, readers, or runners). 

Make an impact: Leave your mark. Your contributions resonate, regardless of your role or rank. 

 

 

On top of the Quantium Experience, you will enjoy a range of great benefits that go beyond the ordinary. Some of these include:  

 

Flexible work arrangements: Achieve work life balance at your own pace with hybrid and flexible work arrangements. 

Continuous learning: Our vision is empowering analytics talent to thrive. The Analytics Community fosters the development of individuals, thought leadership and technical excellence at Quantium through building strong connections, fostering collaboration, and co-creation of best practice. 

Remote working: Embrace the opportunity to work outside of your assigned home location for up to 2 months every year.  

 

 

Quantium’s recruitment process involves the collection and use of personal information. Please click on the link “Privacy” for Quantium’s Collection Notice. This provides information on how we collect, use & store your personal information, including potential disclosure to our majority shareholder, Woolworths Group Limited.  

Remember Job