Posted Aug 22, 2024 805 views
The primary focus will be in collecting, storing, processing and analysing of huge sets of data with choosing optimal solution to use and creating data pipeline architecture in TugstenPlus platform as well as optimizing data flows. The Candidate will support our dev team on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects. The ideal Candidate must have strong experience using a variety of data analysis methods, a variety of data tools, building and implementing data pipeline:
- Work with business cases to identify datasets and design data pipeline architecture
- Analyse data from sources to choose optimal structure and storage architecture
- Assess the consistency and accuracy of new data sources
- Monitor performance of data pipeline execution and fix any issues
- Define technical requirements for third-party applications to pull data.
Core Accountabilities
- Analyse business cases and identify data sources (internal/external) with solution to collect them
- Create and maintain optimal data pipeline architecture
- Assemble large, complex data sets that meet functional/non-functional business requirements
- Design/Build an ETL pipeline
- Keep data secure and control role-based access to collected data
- Work with stakeholders to assist with data-related technical issues and needs
Skills Requirement
- Experience building and optimizing data pipelines, architectures and data sets.
- Knowledge of various ETL techniques and frameworks
- Experience with integration of data from multiple data sources
- Proficient understanding of distributed computing principles
- Proficiency in using query languages such as SQL and experience working with relational/non-relational databases
- Experience working with and creating data architectures, data models, data warehouses/data lakes
- Minimum of 3 years’ experience in area of data engineering/analysis
Technical Skills
- Experience with Databricks (Azure, AWS) and lakehouse approach
- Experience with relational SQL and NoSQL databases, including Microsoft SQL Server, Mongo DB
- Strong knowledge and experience using object-oriented languages (Python, etc.) and SQL language (Spark SQL) to manipulate and transform data
Skills
Job Details
Rendlen Solutions
We are providing IT solutions
What is your Competitive Advantage?
Similar Job Titles
Network Design / Optimization Engineer L2
Zubair Anjum
Business Excellence Specialist
Nestle Pakistan
Zohaib Ali
Marketing Manager
Digital Online Devices
Ayesha Shaukat
Social Media Marketing Manager
Ivoke
Aqib Irshad
Head of HR, Training & OD
FINCA
Babar Ahmed Khan
Territory Sales Manager
Engro Foods
Ali Aziz
CEO
Shopistan
Sehrish Azhar
Motivational Speaker & Blogger
Living Upbeat
Usman Habib
Recruitment & Training Executive
ACS Synergy (Pvt) Ltd.
Muzammil Baig
Digital Marketing Manager
IT Services Group
Mona Sharif
Senior Team Lead
Konext PVT. LTD
