Our client is a leading media enterprise with a rich history and a commitment to excellence. There digital brands are dedicated to delivering high-quality content and engaging experiences to their audiences. This person is responsible for the information needs of the organization. Primarily responsible for building complex data pipelines and designing systems for use by the business. Also responsible for monitoring the health status of the data ecosystem and digging into the vast repository of raw data to create insights on the data, its use and growth. This position plays an integral role in designing and building new data products/systems, leading projects, and supporting other engineers. This is a remote position.
CANDIDATES MUST BE US CITIZEN OR GC HOLDER TO BE CONSIDERED. NO SPONSORSHIP PROVIDED.
WHAT YOU'LL DO:
- Work with / mentor data engineers in programming best practices
- Provide management on statistics around data usage
- Identify and create strategies to address data quality concerns and enforce standards.
- Implement data management repositories based on multiple internal and external data sources
- Manage logical and physical data models and maintain detailed design documents
- Troubleshoot critical ETL workflow and data centric problems and recommend solutions
- Work with analytics business partners to analyze business needs, data sources and develop technical data pipeline solutions to ingest raw data into data warehouse tables
- Write new or modify existing code and conduct complete end to end unit tests on data and data pipelines
- Collaborate with the business analytics team to analyze, resolve, and put in place corrective measures to maintain data accuracy and integrity in support of strategic analytics applications
- Work with SQL databases, APIs, flat files, web scrape and other methods to obtain and ingest raw data leveraging appropriate coding languages such as SQL or Python
- Write transformation logic on raw data and subsequently create semantic layers to publish data in a form suitable for consumption by business users and BI visualization tools
- Performance tune processes to ensure data processing is meeting all business and technical expectations
- Develop API’s to enable business and/or other system’s consumption of published data
- Create and maintain pipeline process documentation and recovery procedures on how to resume failed pipeline processing
- Consult with and assists other programmers to analyze, schedule and implement new or modified workflows
- Provide training to engineers and other IT personnel to grow knowledge in techniques and processes
WHAT YOU'LL NEED:
- Bachelor’s degree in computer science, Management information systems (MIS) or related degree / experience commensurate to a degree preferred
- 4+ years of experience designing, building and deploying solutions with Azure Data Factory
- 4+ years of Experience with Data lake technologies including ADLS Gen. 2, AWS S3, AWS Glue preferred
- 5+ years of hands-on experience implementing, maintaining, and supporting data management solutions including program/project delivery
- 7+ years of experience with SQL & T-SQL code development - experience with Snowflake preferred.
- 5+ years of experience with Python
- 4+ years of experience with Databricks
WHAT YOU'LL BRING:
- Superior technical analysis, critical thinking, troubleshooting and problem-solving skills
- Extensive SQL & general data analysis expertise
- Outstanding Experience with ETL tools and techniques
- Experience building high volume pipelines
- Expert in data discovery and data analysis
- Experience with data virtualization techniques
- Full understanding of data base concepts, data typing and database cardinality principles
- Predictive analytics experience is a plus
- Experience with PySpark is a plus