Make every page findable.

Crawling, rendering, indexing and structure decide whether your best pages can rank at all. We find the problems in your logs and crawl data, then fix them in your platform.

  • LogsWe read what bots crawl, straight from your server logs.
  • INPCore Web Vitals fixes handed to our speed engineers.
  • hreflangInternational sites mapped and validated.
  • MigrationsRedirect maps and launch checks so rankings survive.
Rendering check

What Google gets before your JavaScript runs

Google can render JavaScript, but it queues the work and plenty of crawlers never do. Compare the HTML your server sends with the page a browser builds.

Server HTML

Product image
In the HTML
Product name
Gentle 10% Vitamin C Serum
Missing: Price and stock
Filled in later by JavaScript
Missing: Description
Filled in later by JavaScript
Missing: Reviews
Filled in later by JavaScript
Missing: Related products links
Filled in later by JavaScript
Missing: Product structured data
Filled in later by JavaScript
Breadcrumbs
Home / Serums / Vitamin C

Rendered page

Product image
Shown
Product name
Gentle 10% Vitamin C Serum
Price and stock
£24.00 · In stock
Description
Fragrance-free 10% vitamin C for sensitive skin. Patch tested.
Reviews
4.6 from 212 reviews
Related products links
Pairs with: SPF 50 fluid · Barrier cream
Product structured data
Product, Offer, AggregateRating
Breadcrumbs
Home / Serums / Vitamin C

5 of 7 key elements missing from the server HTML. With server-side rendering on, all seven are in the server HTML.

Platform by platform

The technical SEO issues we check first on each platform

Every platform has its own habits. These are the ones we run into most often, and roughly what the fix looks like on each.

  1. ShopifyHosted

    Shopify and Shopify Plus

    Shopify handles a lot for you: sitemaps, a sensible robots.txt, canonical tags. The problems usually come from themes and apps layered on top.

    • Duplicate product pathsProducts reachable under /collections/x/products/y. The canonical points to /products/y, but themes still link the long path. We fix the theme links.
    • Tag filters/collections/x/tag URLs multiply fast. We decide which deserve indexing and keep the rest out with robots.txt.liquid rules or noindex.
    • App-injected contentReviews, search and filter apps that add content with JavaScript. We check what’s in the server HTML.
    • Markets and hreflangShopify Markets adds hreflang for subfolders or domains. We check the return tags, the x-default and that each market’s prices match.
  2. BigCommerceHosted

    BigCommerce

    BigCommerce gives you real control over URL structure, which is useful, and also an easy way to make a mess.

    • Faceted searchProduct filtering adds parameters to category URLs. We decide which facets earn an indexable landing page.
    • URL changesChanging a product or category URL can create a redirect automatically. We check that setting is on and the redirect list stays clean.
    • Theme renderingSome Stencil widgets render in the browser. We confirm prices, reviews and related products are in the HTML.
    • Multi-storefrontSeveral storefronts need canonical and hreflang rules per storefront, so they don’t compete with each other.
  3. WooCommerceSelf-hosted

    WooCommerce

    WordPress gives you total control, plus every plugin’s opinion about how SEO should work.

    • Parameter URLs?orderby, ?filter_ and add-to-cart links crawled by the thousand. We block or canonicalise them.
    • Tag and attribute archivesProduct tag and attribute archives that duplicate your categories. We noindex or remove them.
    • Plugin overlapTwo SEO plugins both writing canonicals or schema. We pick one source of truth and switch the other off.
    • Server responseSlow, uncached pages hurt crawl rate as well as visitors. We check response times in the Crawl Stats report.
  4. Adobe CommerceEnterprise

    Adobe Commerce (Magento)

    Powerful and flexible, with a lot of settings that create duplicate URLs if nobody checks them.

    • Layered navigationFilter combinations create crawlable URLs in every store view. We set rules and check canonical settings per scope.
    • Category pathsThe “use categories path for product URLs” setting creates several URLs per product. We choose one path.
    • Store views and hreflangSeveral languages or regions in one install need hreflang built per store view, with return tags.
    • Indexers and cacheStale indexers or full-page cache can show bots out-of-date prices and stock. We check cron and indexer health.
  5. HeadlessCustom

    Headless and JavaScript builds

    Headless can be brilliant for speed and terrible for SEO. It mostly comes down to how pages are rendered.

    • RenderingClient-side rendering hands bots an empty shell. We push for server-side or static rendering on every template that should rank.
    • Status codesApps that return 200 for missing products create soft 404s. The server has to send real 404 and 410 codes.
    • MetadataTitles, canonicals and structured data generated per route on the server, not injected after load.
    • SitemapsHeadless builds often launch without one. We generate it from the product catalogue and keep it current.
Log-file analysis

Where crawlers spend their time

Your server logs record every bot request. Read over two weeks they show whether Google is busy with products, or lost in filter and search URLs that should never have been crawlable.

Bot hits by section, last 14 days
CrawlerOn filter and search URLsOn products and collectionsHits to redirects
Googlebot43% 8% after clean-up37% 72% after clean-up1,990 223 after clean-up
Bingbot47% 9% after clean-up34% 69% after clean-up1,327 188 after clean-up
AI crawlers21% 5% after clean-up48% 59% after clean-up442 174 after clean-up

Swipe the table sideways to see every column.

Illustrative log sample.

Indexability funnel
StageShopify storeWooCommerceHeadless JS
Discovered: URLs Google found18,40026,9002,400
Crawled15,200 −2,900 filter and tag URLs21,300 −5,600 ?filter and ?orderby URLs2,380
Rendered15,20021,300700 −1,680 empty until JavaScript runs
Indexable1,480 −13,700 duplicates via /collections/x/products2,200 −19,100 parameters and tag archives640 −60 canonical points elsewhere
Indexed: pages indexed1,180 −300 thin or soft 4041,650 −550 thin tag and author pages520 −120 rendered too late
After fixes1,460 found, 1,360 indexed1,720 found, 1,540 indexed1,520 found, 1,410 indexed
Crawl on useful pages22% 88% after fixes18% 86% after fixes54% 90% after fixes

Swipe the table sideways to see every column.

Illustrative counts for a 1,000-product store.

Common mistakes

Seven technical SEO mistakes we see every month

These are everyday breakages that cost traffic without anyone noticing, listed roughly by how often we find them. Each one shows what happens and the fix.

A noindex left over from staging

Pages vanish

What happensThe site launches with the staging meta robots tag or an X-Robots-Tag header still set. Search Console starts showing pages dropping out within days.

The fixCheck the tag and the header on every template at launch. We add it to the go-live checklist and to a weekly automated check.

robots.txt blocking files Google needs

Broken rendering

What happensBlocking /assets or /scripts stops Google rendering the page properly, so it sees a different page from your visitors.

The fixAllow CSS and JavaScript. Block only URL patterns that waste crawl, such as internal search and sort parameters.

Canonicals pointing at the wrong page

Wrong URL ranks

What happensPaginated pages canonicalised to page one, or every variant pointed at a URL that redirects. Google may ignore the hint and pick its own.

The fixSelf-referencing canonicals on indexable pages, and canonicals only ever pointing at live, indexable URLs.

Redirect chains after a redesign

Lost link value

What happensOld URL to new URL to newer URL, sometimes four hops deep. Each hop slows crawling, and some bots give up following.

The fixPoint every old URL straight at its final destination and update internal links so they skip redirects entirely.

Sitemaps full of junk

Mixed signals

What happensRedirects, 404s, noindexed pages and parameter URLs in the sitemap tell Google you aren’t sure which pages matter.

The fixOnly canonical, indexable pages that return 200, regenerated automatically whenever the catalogue changes.

Soft 404s on deleted products

Crawl waste

What happensA “product not found” page that returns 200 OK. Google treats it as a soft 404 and keeps coming back to check.

The fixA real 404 or 410 for products gone for good, and a 301 when there’s a genuine successor.

Structured data that disagrees with the page

Rich results lost

What happensThe markup says £24 while the page says £29 because a sale ended. Merchant Center and rich results can drop the product.

The fixGenerate markup from the same data that renders the page, so the two can’t drift apart.

International

Every language version vouches for the others

hreflang only works when each version links back to every other version, with valid language and region codes. One missing return tag and Google may ignore the set, showing the US page to buyers in London.

hreflang return tags: Three problems in this set.
PageAlternate it points toWhat is wrong
en-USde-DEThe US page lists the German page, but the German page does not link back. Google may drop the pair.
fr-FRen-GBThe French page points to “en-UK”. UK is not a valid region code; it must be en-GB.
de-DEx-defaultThe German page has no x-default, so buyers outside your markets get no fallback.

Swipe the table sideways to see every column.

Self reference
Each page lists itself as well as its alternates. That is expected.
After fixes
Every pair now links both ways with valid codes and an x-default.

Illustrative set of four pages: en-GB, en-US, de-DE and fr-FR, plus x-default.

Worked example

The order we’d fix a bloated store in

Illustrative, not a real client. Picture a WooCommerce store with about 3,000 products, where Search Console knows about 40,000-odd URLs and most of them aren’t indexed. This is the order we’d work in.

  1. Measure first

    Crawl the site, pull the Page indexing report and two weeks of server logs. Usually the culprit jumps out: tens of thousands of ?filter_ and ?orderby URLs.

    Week 1

  2. Stop the leak

    Block crawling of sort and add-to-cart parameters and remove internal links to them. No point fixing pages Google keeps getting distracted from.

    Week 2

  3. Clear the duplicates

    Noindex tag and attribute archives nobody searches for, and merge near-identical categories with 301s.

    Weeks 2 to 3

  4. Fix what should rank

    Self-referencing canonicals, a clean sitemap, Product markup from the catalogue and faster server response on category pages.

    Weeks 3 to 5

  5. Watch and confirm

    Crawl stats, indexed pages and impressions, weekly. Expect “discovered, not indexed” to fall over a couple of months, not days.

    Months 2 to 3

Glossary

Technical SEO terms you’ll see

You’ll see these in your audit. Here’s what each one means.

Crawl budget
Roughly how many URLs Google will fetch from your site in a given period. Only a real limit on big sites, but stores with filters often are big sites.
Rendering
Running a page’s JavaScript to build what visitors see. Google does it later, in a queue; many other bots don’t do it at all.
Canonical tag
A hint telling search engines which of several similar URLs is the main one. A strong hint, but not an order.
noindex
A meta tag or HTTP header asking search engines to leave a page out of results. The page has to stay crawlable for them to see it.
hreflang
Tags linking the language or regional versions of a page, so the right one shows in each market. Every version must link back.
x-default
The hreflang fallback for visitors who don’t match any of the languages or regions you list.
Soft 404
A missing page that returns 200 OK instead of 404. Google spots it and treats it as an error anyway.
INP
Interaction to Next Paint, the Core Web Vital for responsiveness. It replaced First Input Delay in March 2024. See speed optimisation.
Questions

About technical SEO

Anything else, ask on the audit call.

Ask an engineer

Not answered here? Ask on the free 30-minute audit call. An engineer replies within one business day.

Quick facts

  • Based in Delhi, India Working across time zones, with written weekly updates.
  • Fixed quote first Nothing is billed before you agree it.
  • You own it The code, workflows and accounts are yours.
  • Stores and ERPs Shopify, BigCommerce, WooCommerce and Adobe Commerce, with NetSuite, SAP Business One, Odoo and Dynamics 365.
See all questions
What does a technical SEO audit include?

A full crawl, a rendering check on key templates, a review of robots rules, sitemaps, canonicals and structured data, and, where you can share them, your server logs. You get a ranked fix list with the platform-specific change for each item.

Can Google index JavaScript sites?

Google can render JavaScript, but it defers the work and other crawlers, including most AI crawlers, often do not run it at all. Content and links that matter should be in the server HTML. For headless builds that usually means server-side or static rendering.

Do Core Web Vitals affect rankings?

They are one signal among many, measured on real visits through the Chrome UX Report. The current metrics are LCP, CLS and INP, which replaced FID in 2024. We hand failing templates to our speed engineers rather than chase scores in isolation.

Can you protect rankings during a platform migration?

Yes. We crawl the old site, map every URL with traffic or links to its new home, test the redirects on staging and watch Search Console daily after launch. See our migration service for the full process.

Do you need access to our servers?

Not always. Read access to Search Console, analytics and your platform admin covers most audits. Log files help on large sites; your host or CDN can export them.

How often should we run a technical SEO audit?

A full audit once a year, and before and after any migration or redesign. In between, automated monitoring catches the common breakages, like a stray noindex or a sitemap that stops updating.

Is crawl budget something we need to worry about?

For most sites under a few thousand pages, no. Stores with filters, faceted navigation or lots of parameters often generate far more URLs than they have products, and that’s when crawl efficiency starts to matter.

Do you implement the fixes or just report them?

Either, whichever suits you. We can make changes in your theme or codebase through a reviewed pull request, or hand your developers a ranked list with the exact change for each item.

Does structured data improve rankings?

Not directly. It helps search engines understand the page and makes it eligible for rich results such as prices and review stars. FAQ rich results are now limited to well-known government and health sites and HowTo rich results have been retired, so we focus on Product, Organization, BreadcrumbList and LocalBusiness markup.

Which pages is Google
not seeing?

Send your domain. We will run a crawl and a rendering check on your main templates and show you the biggest leak on a 30-minute call.

We save what you type here so we can follow up if you get interrupted.
One file, up to 10 MB: PDF, Word, Excel, CSV, text, PNG or JPG.

An engineer replies within one business day.

What happens next

  1. An engineer replies Within one business day, by email.
  2. A free 30-minute audit On a call with an engineer.
  3. A fixed quote It follows the audit. Nothing is billed before you agree it.
All services

Get a quote. One form, one real reply.

Tell us what you are building or fixing. An engineer reads every request and replies within one business day, with a fixed scope, price and dates once we have talked.

Direct
[email protected]
Response time
Within one business day
Cost of asking
Nothing. The first 30-minute audit is free.

An engineer replies, not a form queue.