Should you block AI from scanning your website?

The case for blocking

Some organisations are choosing to restrict access for AI crawlers (such as those used by ChatGPT, Copilot, Google’s Gemini or other large models). Their reasoning usually falls into three areas:

1. Control over content use

If your content is being used to train AI systems, you may not have visibility or consent over how it is reused or represented. Taking measures to safeguard against your website being scrapped by AI crawlers may feel like solution*. 

2. Protecting originality and competitive advantage

Organisations that have invested heavily in research or specialist insight might not want that knowledge folded into the collective intelligence of an AI model that anyone can query. For organisations in the arts, culture and heritage sectors, websites often showcase original imagery, video or written work created by artists and contributors. Allowing AI systems to scrape that material raises important questions about copyright, moral rights and artistic control. 

3. Reducing risk

Sectors handling sensitive or proprietary information have legitimate reasons to limit how much of their content sits in the public domain or is processed by automated systems.

These are all valid concerns. But there is another side to the story.

The case against blocking

1. Visibility and reach

AI is increasingly part of how people discover information. Blocking crawlers could mean your content is missing from future search experiences or summarised results. In other words, you may become invisible in emerging digital ecosystems.

2. Trust and authority

If your content is credible and publicly available, allowing it to be referenced by AI systems can help reinforce your authority in your field. By opting out, you may lose opportunities to be represented as a trusted source.

If your goal to inform the public and answer their questions without them needing to visit your website, then you may want your content to appear in AI search results as a way of fulfilling your remit of making important, complex information available.

3. The web’s collaborative nature

The internet has always evolved through shared learning. While AI introduces new dynamics, it still depends on open, high-quality information to deliver meaningful outcomes for users. Contributing to that ecosystem, carefully, can align with the spirit of accessibility and openness that good digital experiences are built on.

Our view

At Path, we believe decisions like this should be guided by clear thinking, not fear.

Blocking AI crawlers outright might protect your content, but it could also isolate your organisation's voice from the evolving ways people access information online.

The more constructive approach is to understand what you want to protect and why. For most organisations, a selective approach, allowing discovery but managing sensitive content responsibly, strikes the right balance.

What matters most is having a considered policy rather than a knee-jerk reaction. AI will continue to shape how the web works. The opportunity lies not in hiding from it, but in understanding it, and ensuring that your digital presence remains visible, valuable and aligned with your principles.

If you would like to discuss how AI might affect your organisation’s website, from visibility to content protection, we would be happy to help. Path can advise on the right approach for your context, combining technical expertise with strategic insight. 

Get in touch to arrange a consultation.

*Note, it is not possible currently to completely block all AI bots from scraping your site, however, there are steps you can take that will highly restrict AI bots from all or some of your content.