ShareShift Indexing Policy & Your Privacy

Welcome to the ShareShift F2 Indexing Policy page! We're delighted you're here to learn more about how we operate and our commitment to responsible web scanning.

At ShareShift, our mission is to build a valuable dataset of technology and domain name insights, helping businesses and individuals understand the digital landscape better. To achieve this, we responsibly scan public web pages, focusing solely on identifying technologies and assing domain value.

Our Commitment to You:

• Positive Contribution: We aim to be a positive contributor to the internet ecosystem by providing useful, anonymized data.
• Lightweight & Efficient: Our custom crawler is designed to be exceptionally light. It fetches a maximum of 8KB of content, deliberately avoiding full page downloads, images, scripts, or any resource-intensive operations. This minimizes our footprint on your server.
• Privacy-First: Your privacy is paramount. We explicitly do not crawl or store any personally identifiable information (PII). We employ robust filtering mechanisms to ensure compliance with GDPR and all applicable data privacy laws, removing any potential PII before it's ever processed.
• Respect for robots.txt: We fully respect and adhere to the robots.txt protocol. This file is your directive to web crawlers, and we follow it diligently.

How We Identify Ourselves:

You'll find our identifier in your server logs via our User-Agent:

compatible; f/2.0 bit.ly/3FLv7xo

This link (which brought you here!) is our way of being transparent about our activities.

Managing Our Access to Your Site:

We understand that you may have specific preferences regarding web crawlers. You have complete control over our access to your website through your robots.txt file.

• Simply add the following to your robots.txt file:

  User-agent: *
  Disallow: /

• To block only the ShareShift index fleet from your entire site, add this specific directive to your robots.txt file:

  User-agent: ShareShift
  Disallow: /

What we store

We extract and store anonymized information about the technologies identified on public websites, such as web server types, CMS platforms, analytics tools, and certain domain metadata. This data is aggregated and used to build our technology and domain name insights dataset. We focus primarily on technology detection and security compliance across the European domain name landscape.

Questions or Concerns?

We value open communication. If you have any questions, concerns, or would like to discuss our indexing practices further, please don't hesitate to reach out to us at https://shareshift.io/contact

Thank you for your understanding and for helping us contribute to a more informed digital world!