Data Engineer (Web Scraping)
Location: | Singapore |
Job Type: | Permanent |
Discipline: | |
Salary: | Negotiable |
Contact: | Chelsea Phan |
Email: | email Chelsea |
Reference: | BBBH6613_1632994963 |
Posted: | over 2 years ago |
About NextWave
NextWave Partners is the Recruitment Partner of choice within the Clean Energy, Sustainable Infrastructure, ESG, Impact Investment, Climate-Tech & Technology sectors. We are committed to supporting industries battling climate change towards a net-zero future and a sustainable economy.
About the Role
We are currently working with a client to build an AI and data team to drive sustainability impacts. They are looking for high performing data engineers to join their team to build scalable and reliable data infrastructures for customers from various industries.
Roles and responsibilities
Design and build scalable infrastructures to crawl high volume of data from various sources
Use Optical Character Recognition technology to scrape structured and unstructured data
Extract, process and store large sets of data from open-standard file format and data interchange format such as XML and JSON
Use relevant tools such as Apache Beam and Apache Spark to build data pipelines that enable batch and stream data processing or concurrency
Build data tools to assist data scientists in building and optimizing AI initiatives
Prepare and maintain detailed documentation on data pipeline and infrastructure
Requirements
At least 2 years of relevant working experience in building data pipelines and performing web scraping
Highly skilled in data crawling tools and approach such as BeautifulSoup, CasperJS, PhantomJS, Selenium and Nodejs
Experienced with data orchestration tools such as Apache Airflow, and rest APIs and relevant web requests
Excellent coding skills in one or many different languages ( eg: C++, Javascript, C#, .Net, Python)
Strong understanding of NoSQL databases
Great to have knowledge in cloud platforms such as GCP, AWS and Microsoft Azure
Great to have some experience in tools such as ElasticSearch, Outsystems, Graph Database and Snowflake
Ideally a self-motivated individual who is passionate about driving sustainable impacts
Application
If you are interested in this position, please apply directly on the platform with your latest CV. We will review your application and revert back promptly.
Keep in touch
If you would wishto keep up to date with the latest NextWave opportunitiesand industry updates, please follow us on LinkedIn and create your profile on our website to receive a weekly newsletter in your inbox!
Our commitment
Diversity is a core value at NextWave Partners, and we are proud to be partnering with equal opportunities employers. All qualified applicants will receive consideration for employment without regard to race, colour, religion, gender, gender identity or expression, sexual orientation, national origin, disability or age.
EA Registration No: R2199999
NextWave Partners Ltd. (EA License No: 16S8303 - UEN: 201602833E)
Web: www.next-wavepartners.com
Job has Expired