Data Engineer (Web Scraping & OSINT)

About the Role

Everstar is establishing a specialized Data Engineer position dedicated to Web Scraping and Open-Source Intelligence (OSINT). This role exists to construct a robust digital investigation infrastructure capable of transforming vast quantities of public and semi-public data into actionable strategic knowledge. The position is integral to the Research and Development division's mission of building systems that track supply chains, monitor market dynamics, and uncover hidden relationships within global networks. Success in this role means moving beyond simple data retrieval to create an automated, scalable foundation for intelligence gathering and analysis.

Key Responsibilities

Your primary mission is to architect and implement the end-to-end data lifecycle for OSINT operations. You will be responsible for designing the initial ingestion frameworks that collect information from diverse online sources while respecting legal and ethical boundaries. This involves developing and maintaining automated extraction mechanisms, often referred to as scrapers or crawlers, which convert unstructured web pages into organized, structured records.

A core focus will be the creation of time-series based storage architectures. These systems are designed to preserve historical data, allowing for the tracking of changes, trends, and anomalies over extended periods. You will also conduct in-depth market analysis, performing what can be described as autopsies on industry sectors to understand production ecosystems, supply chain flows, and manufacturing landscapes.

Continuous monitoring of the operational environment is essential. This includes tracking signals related to component availability, identifying potential shortages, and observing the emergence of new competitors or suppliers. To support these efforts, you will build and maintain comprehensive knowledge bases, including contact directories and organizational maps, to identify key decision-makers and map the relationships between different entities.

Collaboration with internal stakeholders is a critical component of the role. You will work closely with analysts and strategists to interpret high-level investigative questions and convert them into precise data collection and validation workflows. This requires a careful selection of tools and cloud service configurations to ensure system stability, security, and adherence to compliance standards.

Required Qualifications and Experience

To succeed in this position, you must possess demonstrable experience in Data Engineering and OSINT, with a strong emphasis on automated data collection rather than manual techniques. You are expected to write and maintain reliable scraping and crawling solutions, primarily using Python or other relevant programming languages.

Practical experience with database management is essential. You should be comfortable working with both relational databases utilizing SQL and non-relational databases (NoSQL). A strong understanding of time-series data management is considered a significant advantage for versioning and tracking information over time.

You must have a solid grasp of ETL and ELT processes, including data normalization, enrichment, deduplication, logging, and preparation for downstream analytical use. The ideal candidate is a systematic thinker who can independently design processes that span the entire data journey, from initial collection through normalization, storage, change tracking, and final retrieval.

Strong analytical skills are non-negotiable. You must exhibit extreme attention to detail, rigorously verify the reliability of sources, filter out informational noise, and translate raw data into concrete, strategic recommendations. The role also demands excellent soft skills, including the ability to thrive in a dynamic environment, articulate conclusions logically, maintain transparent documentation, and collaborate effectively within a team.

Position Details and Benefits

This is a full-time, remote position offering a competitive monthly compensation package in the range of 1500-2000 USD. The engagement is structured to provide a balanced work-life integration, featuring 24 days of annual leave, a flexible working schedule, and the option for up to 10 days of remote work annually.

Everstar provides comprehensive support to ensure your well-being and professional growth. This includes medical insurance and access to psychological consultation services. also, the company invests in employee development by offering compensation for relevant courses and providing access to educational materials. You will benefit from mentorship, internal support, and a dedicated team environment.

The corporate culture is designed to recognize achievements, provide gifts for significant life events, offer an additional day off for birthdays, and foster a strong internal community. The physical workspace is located in the Kyiv office on the right bank, equipped with all necessary resources and staffed by colleagues who are easy to work with.

What you'll do

• Meet the bar Practical notes