Overview of Job Description:

• This hybrid role requires the skills and responsibilities of a data engineer and a database administrator. This role is responsible for designing, developing, and maintaining data pipelines, data storage solutions, and data integration processes in a cloud-based environment, as well as ensuring the performance, security, and availability of relational databases and database servers. This role requires a high level of technical expertise, problem-solving abilities, and collaboration skills.

Responsibilities:

• Design, develop, and maintain data pipelines, ETL processes, and data storage solutions using various cloud services and tools.
• Administer and manage relational databases and database servers, including configuration, security, backup, restore, performance tuning, and troubleshooting.
• Implement data quality checks, validation rules, and monitoring frameworks to ensure the accuracy and reliability of data assets.
• Work with data architects, data scientists, and business analysts to understand and deliver data solutions that meet business needs and align with data strategy.
• Conduct unit tests, integration tests, and system tests to validate the functionality and performance of data pipelines and databases.
• Create and maintain documentation of data pipelines, databases, and data workflows, including data lineage, data dictionary, and transformation logic.
• Ensure compliance with data governance, data security, and legal requirements.
• Identify and troubleshoot data issues and anomalies using various data quality tools and techniques.
• Collaborate with data analysts, data scientists, and other stakeholders to understand their data needs and provide solutions.
• Keep abreast of the latest technologies and best practices in data engineering and database administration.
• Develop and maintain reports and dashboards that provide insights into data quality, performance, and trends using various reporting tools and platforms.
• Design and implement data validation and testing frameworks to ensure the accuracy and reliability of data products and services.

Qualifications:

• Bachelor’s degree or higher in Computer Science, Information Technology, or a related field.
• 9+ years of experience in data engineering and database administration, with a strong focus on cloud-based services and tools.
• Expertise in ETL tools (e.g., Spark, Airflow, Data Factory) and programming/scripting languages (e.g., SQL, Python).
• Deep understanding of data modeling, database design, and data warehousing principles.
• Experience with administering relational databases and database servers, including performance tuning, security, backup, and restore.
• Familiarity with data integration and migration techniques and tools.
• Excellent problem-solving and analytical skills.
• Strong communication and collaboration abilities.
• Proficiency in report development using various tools and platforms.