We are seeking a skilled and versatile Data Engineer with a passion for technology and a background in data architecture and pipeline development. As a key member of our team, you will be responsible for designing, implementing, and maintaining data pipelines, as well as analyzing and processing both structured and unstructured data. Your expertise in Scala, Spark, and the Hadoop ecosystem will be crucial as we continue to innovate and deliver cutting-edge solutions.
Key Responsibilities:
- Design and develop robust data pipelines and architectures, including ETL/ELT processes.
- Analyze data to identify issues, inconsistencies, and gaps, and implement effective solutions.
- Utilize both relational databases (RDBMSs) and NoSQL databases for data storage and processing.
- Write high-performance Scala code and SQL statements, ensuring the scalability and efficiency of our solutions.
- Collaborate with cross-functional teams to understand and clarify customer requirements.
- Demonstrate an agile mindset, being adaptable and willing to take on tasks outside of your core strengths to support team objectives.
- Maintain autonomy in working on Unix-based systems, ensuring smooth operations and troubleshooting when necessary.
- Exhibit strong communication skills, both verbal and written, to effectively interact with team members and stakeholders.
Nice-to-Have Skills:
- Experience with open-source technologies such as Spark, Hive, HBase, and Kafka.
- Familiarity with Cloudera and IBM mainframe technologies.
- Fluency in English, with proficiency in additional languages considered a plus.
Qualifications:
- Bachelor's degree or equivalent experience in computer science, engineering, or a related field.
- Proven experience in data engineering
- Strong knowledge of Scala, Spark, and the Hadoop ecosystem, including Parquet and ORC file formats.
- Ability to work in a dynamic environment and adapt to evolving requirements and priorities.
- Passion for technology and a willingness to learn and grow within the organization.
If you are ready to embark on an exciting adventure in the world of data engineering, apply now and join our team! We are committed to fostering a culture of continuous learning and growth, so even if you don't check all the boxes, we encourage you to apply if you have a strong work ethic and a passion for technology.
