Overview
The eCOS Data Extraction & Verification Platform was developed to streamline repetitive searches and structured data collection from the eCOS portal. It replaced a time-consuming manual workflow with a centralized Flask dashboard capable of managing extraction jobs, monitoring progress and organizing results.
The platform combined browser automation with HTML parsing to navigate authenticated pages, process large certificate-number ranges and convert retrieved information into structured CSV datasets. The project has since been discontinued following the shutdown or unavailability of the source website.
Challenge
The source portal relied heavily on dynamic page elements, authenticated sessions and JavaScript-driven navigation. Record searches had to be performed individually, while inconsistent page structures and interactive login requirements made conventional HTTP-based extraction unreliable.
The solution also needed to handle long-running bulk operations, preserve job progress and provide a practical way for administrators to review and download collected results.
Solution
A hybrid extraction workflow was implemented using Selenium for browser navigation and authenticated interactions, while BeautifulSoup handled structured HTML parsing and field extraction.
A responsive Flask dashboard provided secure user access, job configuration, real-time progress monitoring and result management. The platform supported session handling, resumable workflows, CSV generation, individual file downloads and bulk ZIP exports. SQLite was used for application and user-management data, while GitHub Actions automated deployment to the Linux server.
Result
The platform consolidated a multi-step manual process into a managed automation workflow. Administrators could initiate bulk searches, monitor extraction progress and retrieve standardized datasets from one interface.
Although the project is no longer operational because the source portal has been discontinued, it demonstrates practical experience in browser automation, authenticated data workflows, HTML parsing, backend development and server deployment.
