Senior Azure Data Engineering Consultant

ProCogia is a data consulting firm based in Vancouver, BC with employees and clients throughout the United States and Canada. Our expertise lies in Data Operations, Data Engineering, BI & Analytics, Data Science & Bioinformatics across a multitude of industries including Telecom, Pharma, Biotechnology, Retail, Logistics, Technology, Financial Services, Media & non-profit sector. We apply a technology-neutral approach which gives us the ability to put our clients first when offering our Data solutions. We are a diverse and cohesive team dedicated to delivering top-tier, comprehensive data solutions for our clients. In exchange for your skills and technical know-how, we offer a competitive salary, generous benefits and plenty of opportunity for personal growth. A 'growth mindset' is a quality we look for in all our new hires and has helped drive much of our recent expansion across North America. Our unique approach is to maximize the value derived from data. Working within ProCogia’s dynamic environment will allow you to realize your full career potential. Over the past two years, ProCogia has experienced significant growth, and we are committed to maintaining a high level of gender equality within the company. Our diversity and differences empower us to develop innovative and effective data solutions for our clients. Our Mission: We empower our customers to make smart, agile decisions through strategically custom-made data solutions. We attract, nurture and retain diverse, motivated, and cooperative team players who love what they do. Our Values: TRUST: We build trust in all our relationships, both internal and external. GROWTH: We encourage life-long learning, both personally and professionally. INNOVATION: Our technology-neutral & innovative approach enables ground-breaking data solutions. EXCELLENCE: We strive to excel in everything we do. At ProCogia, we are passionate about creating data-driven solutions that provide sound answers to our clients' most pressing challenges. Our projects range from Data Warehouse builds, deploying Cloud Data Solutions, Dashboarding, & building predictive models. We work alongside leading clients in various industries including Pharmaceuticals, Telecommunications, Technology, Financial Services & Retail. Our work environment presents opportunities to acquire valuable experience in diverse industries, boosting your personal & professional development. In the last two years, ProCogia has doubled in size and we are dedicated to maintaining a balanced gender ratio. Our diversity and differences equip us to create innovative and effective solutions for our clients. Job Description The Senior Data Engineering Consultant has the following responsibilities and duties: - Develop and maintain optimal data architecture pipeline. - Conduct root cause analysis on internal and external data and processes and share insights with various stakeholders using reporting and visualizing tools. - Design the infrastructure required for optimal extraction, transformation, and loading of data from a wide range of data sources into Snowflake and other platforms. - Understand existing implementation and enhance it as needed. - Collaborate with other data engineers, data ingestion specialists, and experts across cross-functional teams. - Handle large datasets to meet functional and business requirements. Requirements The Senior Data Engineering Consultant should have the following skills, education, and experience: Skills Required: - Proficient experience with Snowflake Data Warehouse. - Exceptional SQL and Python, PySpark skills using Spark framework. - Extensive knowledge of Microsoft Azure cloud services to build scalable data pipelines and workflow management using ADF, Dataflow, and the use of Databricks with PySpark. - Experience in ETL/ELT and understanding of Data Warehouse concepts. - Expertise in designing and managing efficient Data Models in big data ecosystems. - Familiar with Docker/containerization. - Knowledge of API-based integration and security (Restful and SOAP). - Familiar with Data Visualization tools such as Power BI etc. - Ability to learn quickly and translate data into actionable results. - Excellent communication skills with onsite team and client stakeholders. Preferred: - Experience with JSON Programming. - Solid Linux programming skills. - Cosmos DB, Teradata, SQL Server (all nice to have). - Stream-processing systems: Streaming-Analytics, IoT Hub, Event Hub, Kusto Queries (all nice to have). Education Required: - Bachelor’s degree in a quantitative field such as computer science, computer engineering, or other related disciplines. Preferred: - Master's degree or PhD in a quantitative field such as computer science, computer engineering, or other related disciplines. Experience: - A professional with at least 5+ years of experience in a Data Engineering role with a master’s degree or 7+ years of experience in a Data Engineering role with a bachelor’s degree. Location: - Pune, or willing to relocate. Shift: - EST shift (with some flexibility). ProCogia is proud to be an equal opportunity employer. We are committed to nurturing a diverse and inclusive workspace. All qualified candidates will receive consideration for employment regardless of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status.