Launch Recite Me assistive technology
Back to job search

Azure Data Engineer - Wealth Management - InsideIR35

  • Location:

    City of London

  • Sector:

    Investments

  • Job type:

    Permanent

  • Salary:

    £500 - £700 per day

  • Contact:

    Matthew Thresher

  • Contact email:

    Matthew.Thresher@oliverjames.com

  • Job ref:

    JOB-032023-203350_1680016777

  • Published:

    ongeveer 5 uur geleden

  • Duration:

    6 Months

  • Expiry date:

    2023-04-12

Clients Azure Data Platform utilises Azure Data Factory, Databricks and Delta lake along with a meta data repository to accelerate ingestion and processing of data. We call this the "Data Platform Framework".

Develop and maintain data ingestion, storage, processing and integration solutions. This includes the development and management of the Data Framework for metadata driven ingestion into the Azure Data Platform.

Responsibilities:

  • Define, update and deliver the DevOps model including CI/CD pipelines for the Azure Data Platform in line with wider GTS standards
  • Maintain and update the Data Platform Framework ensuring effective ingestion, storage and transformation of data through the stages of the data lake
  • Design and implement data integrations between the Azure Data Platform and operational systems both on premises and in the cloud
  • Build data pipelines for data throughout the Azure Data Platform stages, integrating data management best practices at all stage transitions suitable for their use case, e.g. reporting or archival of data from operational systems or unstructured data stores, Azure Data Platform housekeeping
  • Develop data sets which are optimised for their given use case, e.g. self-service analytics, paginated reports or machine learning and advanced analytics
  • Reverse engineer existing data solutions and business developed application to redevelop into the Azure Data Platform
  • Migrate data from legacy reporting solutions on the Azure Data Platform to ensuring that data integrity and service levels are maintained
  • Responsible for ensuring work products are optimised for run-cost minimisation.
  • Ensure initiatives reduce complexity, risk and cost of solution delivery and enable scaling and repeatable use.
  • Recognise opportunities to re-use existing data flows/data pipelines, data sets etc
  • Ensure industry 'data' best practice is leveraged in defining, delivering and managing data and data technology, including Data Management and Data Governance.
  • Define and contribute to the development and implementation of appropriate practices, standards, governance and risk management processes for Data Engineering, in line with policies and technology principles.
  • Engineers for data quality and adoption requirements across the enterprise for the Azure Data Platform.
  • Define, contribute to and deliver the Software Development Lifecycle for data including test and quality assurance.

Skills & Experience:

  • Azure Data Services (including Data Factory, Databricks, Synapse Analytics, Data Lake)
  • Microsoft BI experience (one or more of SSAS, SSRS, Power BI and other Power Platform)
  • SQL including T-SQL (Queries, views, stored procedures, functions)
  • Integration and ETL design and delivery Including (SSIS, Azure Data Factory)
  • Data visualisation (one or more of Tableau, QlikView, Power BI, Business Objects)
  • Developing analytical use cases using Spark programming in Python and one other language (e.g. R, and Scala)
  • Relational and multi-dimensional data modelling
  • Operating within an IT function, including systems analysis and data analysis.
  • Designing and delivering Testing including Regression, Unit, integration, and system for data solutions
  • Data Quality management and root cause analysis
  • Developing designs for data pipelines and data flows and data sets
  • Multiple migration of code and data from legacy solutions to alternative technology
  • Coaching and mentoring developer community
  • Working with both Agile and Waterfall delivery methodologies.
  • DevOps and associated technology (GITHub, Azure DevOps)

Desirable:

  • NoSQL databases (e.g. Cosmos)
  • Streaming analytics and real-time use cases
  • Operating metadata driven data platform frameworks

Image 2020 11 03 T18 19 45

The latest OJ Webinar

Are you looking to relocate?