Data Engineer (Web Scraping)

Location: Singapore
Job Type: Permanent
Discipline:
Salary: Negotiable
Contact: Chelsea Phan
Email: email Chelsea
Reference: BBBH6613_1632994963
Posted: over 2 years ago

About NextWave

NextWave Partners is the Recruitment Partner of choice within the Clean Energy, Sustainable Infrastructure, ESG, Impact Investment, Climate-Tech & Technology sectors. We are committed to supporting industries battling climate change towards a net-zero future and a sustainable economy.

About the Role

We are currently working with a client to build an AI and data team to drive sustainability impacts. They are looking for high performing data engineers to join their team to build scalable and reliable data infrastructures for customers from various industries.

Roles and responsibilities

  • Design and build scalable infrastructures to crawl high volume of data from various sources

  • Use Optical Character Recognition technology to scrape structured and unstructured data

  • Extract, process and store large sets of data from open-standard file format and data interchange format such as XML and JSON

  • Use relevant tools such as Apache Beam and Apache Spark to build data pipelines that enable batch and stream data processing or concurrency

  • Build data tools to assist data scientists in building and optimizing AI initiatives

  • Prepare and maintain detailed documentation on data pipeline and infrastructure

Requirements

  • At least 2 years of relevant working experience in building data pipelines and performing web scraping

  • Highly skilled in data crawling tools and approach such as BeautifulSoup, CasperJS, PhantomJS, Selenium and Nodejs

  • Experienced with data orchestration tools such as Apache Airflow, and rest APIs and relevant web requests

  • Excellent coding skills in one or many different languages ( eg: C++, Javascript, C#, .Net, Python)

  • Strong understanding of NoSQL databases

  • Great to have knowledge in cloud platforms such as GCP, AWS and Microsoft Azure

  • Great to have some experience in tools such as ElasticSearch, Outsystems, Graph Database and Snowflake

  • Ideally a self-motivated individual who is passionate about driving sustainable impacts

Application

If you are interested in this position, please apply directly on the platform with your latest CV. We will review your application and revert back promptly.

Keep in touch

If you would wishto keep up to date with the latest NextWave opportunitiesand industry updates, please follow us on LinkedIn and create your profile on our website to receive a weekly newsletter in your inbox!

Our commitment

Diversity is a core value at NextWave Partners, and we are proud to be partnering with equal opportunities employers. All qualified applicants will receive consideration for employment without regard to race, colour, religion, gender, gender identity or expression, sexual orientation, national origin, disability or age.

EA Registration No: R2199999

NextWave Partners Ltd. (EA License No: 16S8303 - UEN: 201602833E)
Web: www.next-wavepartners.com