Skip to content
All posts
seoprogrammatic-seonextjsmarketplacecase-study

Programmatic SEO Without Thin Content: Lessons from Qooty

November 12, 2026 12 min read· Mahmoud Makhlouf
اقرأ هذا المقال بالعربية

A shopper in Shaqra searches for detergent offers and lands on a page that says "Detergent deals in Shaqra" above six items from one store. The Riyadh version of the same page shows the same flyer. Google sees two URLs that are mostly the same, and multiplies that by every city and category on the site.

Short answer: generating thousands of pages from your data only helps when each page has something a searcher wants. The work that matters is a gate that decides which pages earn index, sitemaps that list only those pages, real 404s for empty ones, and one stable URL per real product. On Qooty, that meant fewer submitted URLs, deliberately.

The business problem: more pages can mean less traffic

Programmatic SEO is attractive because you already have the data (stores, cities, brands, categories), and a template can turn it into tens of thousands of landing pages overnight, each targeting a search like "Almarai milk offers in Jeddah".

The trouble starts when the data behind a page is thin. A city and category pair with zero live offers still renders a heading, an intro and an empty grid. Multiply that across every combination and most of your site is empty template. Four costs follow:

  • Crawl budget. Google's own guidance for large sites says duplicate and low-value URLs pull crawling away from the pages you care about. On a deals site where offers change weekly, slow recrawling means stale prices in search results.
  • Quality signals. Google's spam policies describe "scaled content abuse": many pages generated mainly to rank, with little value to users. Its helpful-content guidance asks directly whether you use "extensive automation to produce content on many topics." Empty combinatorial pages fit that description even when nobody meant them to.
  • Soft 404s. A page that returns 200 OK but shows nothing is reported in Search Console as a soft 404. Those URLs keep getting crawled and add nothing.
  • Monetization. Ad networks review sites for content value before approving them. Mostly empty shells make that a hard case.

So the founder's question is "how many of these pages deserve to be in Google today, and does our system know the difference?"

Your options

ApproachWhat it doesTrade-off
Index everythingEvery combination is indexable and in the sitemapFastest to ship; most exposure to thin-content and crawl-waste problems
Hand-pick pagesAn SEO person chooses which pages existSafe, but does not scale and goes stale as data changes
Data-driven gateEach page decides index or noindex from live counts; sitemaps apply the same ruleMore engineering; the page set grows and shrinks with real inventory
Fewer, deeper pagesMerge thin variants into stronger hub pagesGood for quality; you give up some long-tail queries

For a catalog that changes every week, the data-driven gate is the only option that stays correct without someone babysitting it. In practice you pair it with the last row.

How we built it in Qooty

Qooty is a Saudi offers platform: weekly supermarket and retail flyers, broken into individual products, searchable by city, store, brand and category, on web and mobile. The web app is Next.js with Arabic and English routes, and it generates its landing pages from the same data the app uses. We covered the shared-backend side of that setup in one backend for Flutter and Next.js.

The page families are what you would expect: city hubs, store pages, brand hubs, about 70 subcategory landing pages under /ksa/deals/, weekly flyers, offers and products. Then come the combinations. City by subcategory alone is 9,288 URLs, brand by subcategory 10,000, city by brand 5,160. Roughly 24,000 combinatorial URLs before you count a single product.

a city deals page (city by subcategory) showing products from several different storesa city deals page (city by subcategory) showing products from several different stores

An external technical SEO audit and our own pass over Search Console and analytics data pointed at the same issues: empty pages returning 200 and getting indexed, duplicate and non-canonical pages, a structured-data warning on flyer pages, and broken redirects. We wrote the recovery plan on November 7, 2026, and shipped it in phases.

What we changed

  1. A thin-content gate on every generated listing. A page earns index only once it renders real rows. Below the threshold it still works for a visitor who follows a link, but it is noindex, follow.
  2. Stricter rules for city pages. The same chain's flyer repeats across cities. One store's six items in Shaqra made a page about 43% identical to the Riyadh page. City by subcategory pages now need at least six products from at least two different stores.
  3. Sitemaps that apply the same rule as the pages. Before this, roughly 30% of the URLs we submitted in sitemaps were pages that marked themselves noindex. A shared indexability endpoint in the backend now feeds both the pages and the sitemaps.
  4. Real 404s. Unknown paths return a true 404 instead of an empty 200. Store comparison pages exist only for two stores that share a category and both have offers; anything else is a 404 and stays out of the sitemap.
  5. Honest structured data. We removed shipping and return-policy fields that described policies Qooty does not have (it lists offers; it does not sell), and removed Event markup from flyer pages. Google's event guidelines say not to mark sales or promotions as events.
  6. Expired content that stays useful. Expired flyers are archived for 90 days instead of being deleted. An expired product page returns 200 with "offer ended", the last price, and similar live offers.
  7. Canonical price pages. One permanent /price/[slug] page per real-world product, built from price history across weeks and stores. More on this below.

After deployment, the sitemaps shrank to the pages worth crawling:

SitemapBeforeAfter
City deals9,6321,102
Stores1,880698
Brands2,9942,102
Store hubs15254
Store comparisons902

The comparison sitemap going from 90 to 2 looks dramatic. It is the honest number: most of those pairs compared stores that do not sell the same things.

What the numbers say

We took over Qooty in March 2026 and replaced the existing website with a new one built on this architecture. Google Search Console for the last 16 months tells the story better than any adjective: before March the site was close to invisible in search, with clicks and impressions near zero. After the rebuild, impressions climbed to roughly 12,000 a day at the June peak, and over the period the site collected 1.53 million impressions and 22.3 thousand clicks from Google, at an average position of 7.7.

Google Search Console for Qooty over 16 months: near zero before the March 2026 rebuild, then 22.3k clicks and 1.53M impressions at an average position of 7.7Google Search Console for Qooty over 16 months: near zero before the March 2026 rebuild, then 22.3k clicks and 1.53M impressions at an average position of 7.7

The same chart shows why the work below matters. From late summer the daily numbers slid back, which is the pattern a large site shows when Google re-evaluates thin and duplicate pages. That is exactly what the gate, the cleaned-up sitemaps and the canonical price pages target. The recovery plan shipped in November 2026, so its effect is not in this chart yet.

Under the hood

The gate

The rule is deliberately boring, and every caller uses the same function, so a page and its sitemap entry can never disagree.

// lib/thin-content.ts (simplified)
export const MIN_ITEMS_TO_INDEX = 3;

export function isIndexableListing(itemCount: number): boolean {
  return itemCount >= MIN_ITEMS_TO_INDEX;
}

// City pages repeat the same chain's flyer across cities,
// so they must genuinely compare stores to earn an index.
export const MIN_CITY_DEALS_ITEMS = 6;
export const MIN_CITY_DEALS_STORES = 2;

export function isIndexableCityDeals(products: number, stores: number) {
  return products >= MIN_CITY_DEALS_ITEMS && stores >= MIN_CITY_DEALS_STORES;
}

The counts come from one backend endpoint that computes products per city and subcategory, offers per city and store, and products per brand, cached for 30 minutes. A sitemap route is then a loop with one continue.

Partitioned sitemaps

The sitemap index lists 16 sub-sitemaps, one per page family. Google allows up to 50,000 URLs or 50 MB per sitemap file, so partitioning is less about the limit and more about diagnosis: when Search Console reports a problem in sitemap-city-deals.xml, you know which template to open.

sitemap.xml (index)
 ├─ sitemap-products.xml     capped at 5,000 URLs, cities fetched 8 at a time
 ├─ sitemap-price-pages.xml  only canonical products that pass the gate
 ├─ sitemap-city-deals.xml   only city pages that pass the stricter rule
 └─ ... 13 more
 (not listed: brand x subcategory, city x brand, price ranges)

Three families stay out of the index entirely. Most of their URLs are noindex shells, and the ones with real rows are still reachable through internal links. Products with a canonical price page are skipped in the products sitemap, since their canonical URL is the /price/ page.

Canonical price pages

Each weekly flyer creates product listings that live for seven days and then expire, the worst possible shape for search. The fix groups them into one permanent page per real product, with price history across weeks and stores, and the weekly listings point to it with rel=canonical.

Matching is the hard part. Barcodes were almost absent (21 out of about 27,000 rows), so identity is a normalized Arabic name plus a parsed size, combined with the brand when there is one. Unbranded piece goods such as "3-piece trolley set" merged different products, so they get no canonical page at all.

a /price/(slug) page showing one product's price history across stores and weeksa /price/(slug) page showing one product's price history across stores and weeks

The backfill on production turned 27,225 listing rows into 18,179 canonical products. Only 582 of them passed the index gate:

// services/canonical (simplified)
const MIN_WEEKS_TO_INDEX = 3;
const MIN_STORES_TO_INDEX = 2;
// A 2.5x price spread usually means two products were merged
// (a single pack and a carton). Keep those out until the data is fixed.
const MAX_PRICE_SPREAD_TO_INDEX = 2.5;

s.priceSpread = s.minPrice && s.maxPrice ? s.maxPrice / s.minPrice : 1;
s.suspect = s.priceSpread >= MAX_PRICE_SPREAD_TO_INDEX;
s.indexable = !s.suspect &&
  (s.weeks >= MIN_WEEKS_TO_INDEX || s.stores >= MIN_STORES_TO_INDEX);

582 out of 18,179 is the point. Price history had started recording only a week earlier, and a page with one price from one store is no better than the flyer listing it replaces. As weeks accumulate, more products cross the gate on their own.

Previews and fast discovery

Every main page family (city, store, brand, product, offer, weekly flyers, featured deals) gets a generated 1200×630 social image from one shared template, mirrored for Arabic. One lesson: an Arabic font that rendered fine in the browser crashed the edge image renderer, so the template uses another Arabic font that subsets cleanly.

When content changes, the admin side notifies the site, which revalidates the affected pages and submits their URLs to IndexNow in batches of up to 10,000, the protocol's per-request limit. Bing, Yandex and others use IndexNow; Google does not, so for Google the sitemaps and their lastmod values do the work.

Core Web Vitals: still in progress

Search Console's mobile Core Web Vitals report tells a less tidy story, and we would rather show it as it is. Through July and August between roughly 700 and 2,100 mobile URLs sat in "needs improvement." On August 23 the report moved them to "poor" (1,514 URLs). By September 28 that was down to 186 poor URLs, all for the same reason: LCP over four seconds on mobile. At that export date, no URL group was rated "good" yet.

The report groups similar URLs and only counts pages with enough real-user data, so its totals move for reasons beyond your code. What we control is the foundation. Flyer and store images used to ship at full resolution (about 2 MB on the homepage alone); a same-origin image route now resizes them, converts to WebP, and serves them with a one-year immutable cache behind Cloudflare. Fonts are self-hosted. Real-user LCP, INP, CLS, FCP and TTFB from every page view roll up on the dashboard's performance page, next to Vercel Speed Insights, so we see LCP per route without waiting weeks for Search Console.

admin dashboard performance page showing real-user LCP, INP and CLS by platformadmin dashboard performance page showing real-user LCP, INP and CLS by platform

What we learned

  • The gate has to live in one place. The ~30% of sitemap URLs that were noindex came from pages and sitemaps applying different rules.
  • Duplicate content in programmatic SEO usually comes from your data, not your template. The same flyer in 40 cities is the real source of near-duplicates.
  • Data quality limits what you can index. The price-spread check catches bad merges before Google sees them.

Checklist: before you launch thousands of generated pages

  1. Count how many of your combinations have at least a few real items today. That number, not the total, is your real page count.
  2. Put the index rule in one function and call it from both the page metadata and the sitemap.
  3. Make empty or impossible pages return a real 404, not a friendly empty state with 200 OK.
  4. Partition sitemaps by page family so Search Console errors point at a template.
  5. Decide what happens when content expires. Archive with a useful page, then remove, instead of deleting the same day.
  6. Give each real-world product one permanent URL, and point weekly listings at it with rel=canonical.
  7. Mark up only what is true. No shipping policy you do not have, no Event for a sale.
  8. Measure Core Web Vitals from real users per route, not only from lab tests.

Frequently asked questions

Is programmatic SEO against Google's guidelines?

No. Generating pages from structured data is fine when each page is useful to the person who lands on it. Google's spam policies target pages generated mainly to manipulate rankings with little value to users. The practical test is whether a page has real, distinct content right now.

Should thin pages be noindex or 404?

It depends on whether the page can ever be useful. A city and category page that is empty this week but may have offers next week can stay live as noindex, follow. A page that can never have content, such as a comparison between two stores that sell different things, should return a real 404.

Does Google support IndexNow?

At the time of writing (November 2026), no. IndexNow is used by Bing, Yandex, Naver, Seznam and others. For Google, keep sitemaps accurate, with honest lastmod dates, and link new pages from pages that already get crawled.

How many URLs can one sitemap hold?

Google accepts up to 50,000 URLs or 50 MB uncompressed per sitemap file. Large sites use a sitemap index pointing to several smaller sitemaps, split by page type so problems are easy to trace.

What should a founder ask their SEO or development team?

Ask how many generated URLs exist, how many are indexable, and what rule decides it. Ask whether the sitemap applies the same rule as the page, what an empty page returns, and what happens when content expires. Vague answers usually mean the problems above.

Next step

If you are planning a catalog, marketplace or directory with generated pages, start by measuring how many combinations have real content today. If you want a team to review your page model, sitemaps and index rules before you scale, send us a short brief and we will tell you what we would gate first. For the wider build, our marketplace development guide covers the rest of the platform.

// Let's build

Have a product in mind? Get a clear plan and a free 15-minute intro call.

Tell us what you're building. We'll come back with an approach, a rough timeline, and a ballpark — usually within 24 hours.

Book a free discovery call WhatsApp