- Chi tiết công việc
- Giới thiệu công ty
Vị trí công việc này hiện tại đã hết hạn nộp hồ sơ, bạn có thể tham khảo thêm một số công việc liên quan phía dưới
Mô tả công việc
Mô tả Công việc
Crawler Development and maintenance: Mentor junior developers in crawler development techniques and best practices, Provide guidance in architectural decisions for data crawling and management.
Technical Leadership: Analyze and improve the performance of crawlers, reducing latency and increasing data retrieval efficiency, Address challenges related to rate limiting and blocking mechanisms, Performance Optimization: Implement security measures to protect sensitive data and ensure compliance with data privacy laws, Monitor and adapt to changes in social media platforms's APIs, rules, and regulations
Security and Compliance: Integrate crawlers with existing systems and third-party tools to streamline data access and usage., Work closely with other engineering teams to incorporate new features and maintain compatibility across services.
Integration and collaboration: Build data pipelines to clean, process and store data effectively, Collaborate with data scientists and analysts to ensure data quality and availability for analysis.
Data Management: Design, develop and maintain robust web crawlers that can effectively collect data from Facebook, Tiktok, Threads, Linkedin, and other platforms, Optimize crawling algorithms to ensure data is gathered quickly and accurately while adhering to ethical standards and platforms policies
Crawler Development and maintenance: Mentor junior developers in crawler development techniques and best practices, Provide guidance in architectural decisions for data crawling and management.
Technical Leadership: Analyze and improve the performance of crawlers, reducing latency and increasing data retrieval efficiency, Address challenges related to rate limiting and blocking mechanisms, Performance Optimization: Implement security measures to protect sensitive data and ensure compliance with data privacy laws, Monitor and adapt to changes in social media platforms's APIs, rules, and regulations
Security and Compliance: Integrate crawlers with existing systems and third-party tools to streamline data access and usage., Work closely with other engineering teams to incorporate new features and maintain compatibility across services.
Integration and collaboration: Build data pipelines to clean, process and store data effectively, Collaborate with data scientists and analysts to ensure data quality and availability for analysis.
Data Management: Design, develop and maintain robust web crawlers that can effectively collect data from Facebook, Tiktok, Threads, Linkedin, and other platforms, Optimize crawling algorithms to ensure data is gathered quickly and accurately while adhering to ethical standards and platforms policies
Yêu cầu
Yêu Cầu Công Việc
Technical skills:
Strong programming skills in python, javascript, or the other relevant languages.
Hands-on experience with web scraping libraries and tools like beautiful soup, scrapy, selenium, or puppeteer.
Familiarity with API development and RESTful services.
Experience
Proven track record in building web crawlers for social media platforms
Understanding of platform-specific challenges (e.g., rate limiting, data structures,)
Data skills:
Experience working with big data tools (e.g., Hadoop, Spark).
Proficiency with SQL/NoSQL databases.
Technical skills:
Strong programming skills in python, javascript, or the other relevant languages.
Hands-on experience with web scraping libraries and tools like beautiful soup, scrapy, selenium, or puppeteer.
Familiarity with API development and RESTful services.
Experience
Proven track record in building web crawlers for social media platforms
Understanding of platform-specific challenges (e.g., rate limiting, data structures,)
Data skills:
Experience working with big data tools (e.g., Hadoop, Spark).
Proficiency with SQL/NoSQL databases.
Quyền lợi
Chế độ bảo hiểm
Thông tin khác
Độ tuổi:
Không giới hạn tuổi
Lương:
40 Tr - 50 Tr
VND
Không giới hạn tuổi
Lương:
40 Tr - 50 Tr
VND
Thông tin chung
- Ngày hết hạn: 30/06/2024
- Thu nhập: 40 Tr - 50 Tr VND
Việc làm tương tự
CÔNG TY CỔ PHẦN TRUYỀN THÔNG DU LỊCH VIỆT
Thỏa thuận
04/01/2025
Hồ Chí Minh
Công Ty TNHH Phát Triển Hệ Thống Nichietsu
Thoả thuận
10/01/2025
Hồ Chí Minh
Công ty Cổ phần Phần mềm MOR
15 - 60 triệu
19/01/2025
Hồ Chí Minh
Công ty Cổ phần Viễn thông FPT - FPT Telecom
Thương lượng
31/01/2025
Hồ Chí Minh
hỗ trợ doanh nghiệp
Giải thưởng
của chúng tôi
Top 3
Nền tảng số tiêu biểu của Bộ
TT&TT 2022.
Top 15
Startup Việt xuất sắc 2019 do VNExpress tổ chức.
Top 10
Doanh nghiệp khởi nghiệp sáng tạo Việt Nam - Hội đồng tư vấn kinh doanh ASEAN bình chọn.
Giải Đồng
Sản phẩm công nghệ số Make In Viet Nam 2023.