Senior Data Engineer

CKCODECONNECT
Full-timeAnywhere, Worldwide

MUST submit a Resume in English, Work hours are in the US PST time zone, California.

Key Responsibilities:

  • Design and architect robust data pipelines for structured, semi-structured, and unstructured data.
  • Develop, manage, and optimize databases, including RDMS (MySQL, PostgreSQL), NoSQL (MongoDB), and data lakes (S3).
  • Implement efficient ETL processes using tools like PySpark and Hadoop to transform and prepare data for analytics and AI use cases.
  • Optimize database performance, including query tuning, indexing, and caching strategies using Azure and GCP caching databases.
  • Build and maintain CI/CD pipelines, manage YML files, and use GitHub for version control and collaboration.
  • Leverage Docker for containerized deployment, with hands-on experience in running Docker commands for database and pipeline management.
  • Ensure solutions adhere to best practices in system design, focusing on trade-offs, security, performance, and efficiency.
  • Monitor, maintain, and troubleshoot database infrastructure to ensure high availability and performance.
  • Collaborate with engineering teams to design scalable solutions for large-scale data processing.
  • Stay updated on the latest database technologies and implement best practices for database design and management.

Qualifications:

  • 4+ years of experience in database architecture and optimization.
  • Expertise in RDMS, NoSQL, and semi-structured databases (MySQL, PostgreSQL, MongoDB).
  • Proficiency in programming languages for database integration and optimization (Python preferred).
  • Strong knowledge of distributed data processing tools like PySpark and Hadoop.
  • Hands-on experience with AWS services for data storage and processing, including S3.
  • Strong familiarity with Redis for caching and query optimization.
  • Proven experience with Docker for containerized deployments and writing CI/CD pipelines using YML files.

CKCODECONNECT is an Equal Opportunity Employer and does not discriminate based on race, age, color, religion, sex, sexual orientation, gender identity, national origin, veteran, disability status or any other characteristic protected by applicable law.

Apply for this job

Resume/CV*

Click or drag file to this area to upload your Resume

Please make sure to upload a PDF

First Name*
Last Name*
Email*
Phone Number*
What is your current location?*