<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[Carlos Calatayud's Blog]]></title><description><![CDATA[Carlos Calatayud's Blog]]></description><link>https://carloscalatayud.hashnode.dev</link><generator>RSS for Node</generator><lastBuildDate>Mon, 02 Nov 2026 19:51:35 GMT</lastBuildDate><atom:link href="https://carloscalatayud.hashnode.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[Google Trends API in Python (2026): pytrends, 429 errors, and comparing keywords at scale]]></title><description><![CDATA[Google Trends is still one of the few free signals for "what are people searching for, relatively". But if you try to use it from code, you quickly run…]]></description><link>https://carloscalatayud.hashnode.dev/google-trends-api-in-python-2026-pytrends-429-errors-and-comparing-keywords-at-scale</link><guid isPermaLink="true">https://carloscalatayud.hashnode.dev/google-trends-api-in-python-2026-pytrends-429-errors-and-comparing-keywords-at-scale</guid><category><![CDATA[Python]]></category><dc:creator><![CDATA[Carlos Calatayud]]></dc:creator><pubDate>Sun, 25 Oct 2026 00:11:27 GMT</pubDate><content:encoded><![CDATA[<p>Google Trends is still one of the few free signals for "what are people searching for, relatively". But if you try to use it from code, you quickly run into three problems:</p>
<ol>
<li>There is no generally available official API.</li>
<li>The unofficial routes (pytrends and friends) break or return <code>429 Too Many Requests</code>.</li>
<li>Even when you get data, numbers from different requests are <strong>not comparable</strong>, because each request is rescaled to its own 0–100.</li>
</ol>
<p>This post is a practical guide to all three, with Python you can copy. It is a different angle from my earlier post on <a href="https://dev.to/carloscalatayud/how-to-compare-100-google-trends-keywords-on-one-0-100-scale-beyond-the-5-term-limit-1fii">comparing 100 keywords on one scale</a>: here the focus is getting data reliably into Python and pandas, and knowing which option to pick.</p>
<h2>Option 1: the official Google Trends API (alpha)</h2>
<p>Google <a href="https://developers.google.com/search/blog/2025/07/trends-api">announced an official Trends API</a> in July 2025 as an alpha with limited access by application. If you get in, use it — it's the cleanest route (consistently scaled data across requests). As of writing, access still isn't open to everyone, so most developers need one of the options below. Check Google's current status before you build on it; I'm not going to guess at its quotas or terms.</p>
<h2>Option 2: pytrends</h2>
<p><a href="https://github.com/GeneralMills/pytrends">pytrends</a> wraps the same internal endpoints the Trends website uses. The basic call:</p>
<pre><code class="language-python">from pytrends.request import TrendReq

pt = TrendReq(hl="en-US", tz=360)
pt.build_payload(["python", "javascript", "rust"], timeframe="today 12-m", geo="US")
df = pt.interest_over_time()          # index = date, one column per keyword, + isPartial
print(df.tail())
</code></pre>
<p>Things that bite people:</p>
<ul>
<li><strong>Max 5 keywords per payload.</strong> (The website now accepts more terms in Explore, but pytrends payloads are still built around 5.)</li>
<li><strong>The repo is archived</strong> and no longer maintained, so when Google changes something, fixes come from forks or not at all.</li>
<li><strong>429 errors.</strong> Google rate-limits per IP and session. A loop of a few dozen requests from a laptop is often enough to get blocked for a while, and from cloud/datacenter IPs it can happen almost immediately.</li>
<li><code>tz</code> <strong>and</strong> <code>hl</code> <strong>matter.</strong> Weekly buckets shift with timezone; mismatched settings are a common reason why "pytrends doesn't match the UI".</li>
</ul>
<h3>Surviving 429s</h3>
<p>There's no magic header. What works is pacing, backoff, resuming and — for volume — more IPs:</p>
<pre><code class="language-python">import json, os, random, time
from pytrends.request import TrendReq
from pytrends.exceptions import TooManyRequestsError

CACHE = "trends_cache.json"
cache = json.load(open(CACHE)) if os.path.exists(CACHE) else {}
pt = TrendReq(hl="en-US", tz=360, retries=0)

def fetch(batch, timeframe="today 12-m", geo="US", tries=6):
    key = "|".join(batch) + f"|{timeframe}|{geo}"
    if key in cache:
        return cache[key]
    for i in range(tries):
        try:
            pt.build_payload(batch, timeframe=timeframe, geo=geo)
            df = pt.interest_over_time().drop(columns="isPartial", errors="ignore")
            cache[key] = df.reset_index().astype({"date": str}).to_dict("list")
            json.dump(cache, open(CACHE, "w"))
            time.sleep(random.uniform(8, 20))          # be polite between calls
            return cache[key]
        except TooManyRequestsError:
            wait = min(900, 30 * 2 ** i) + random.uniform(0, 10)
            print(f"429 on {batch}, sleeping {wait:.0f}s")
            time.sleep(wait)
    raise RuntimeError(f"Gave up on {batch}")
</code></pre>
<p>The cache is the important part: when (not if) you get blocked, you restart and don't lose the work already done.</p>
<h2>Option 3: paid SERP-style APIs and hosted scrapers</h2>
<p>Services such as SerpApi, DataForSEO and many Apify Actors expose Google Trends behind a normal REST API with proxies and retries handled for you. You trade money for not maintaining scraping code. When comparing them, look at:</p>
<ul>
<li><strong>Price per what?</strong> Per request, per keyword, per result row — these differ a lot for the same job.</li>
<li><strong>How many keywords per request</strong> and whether the output is already merged.</li>
<li><strong>Whether multi-batch results are put on one scale</strong> (most aren't — see next section).</li>
</ul>
<h2>The real trap: numbers from different requests don't compare</h2>
<p>Say you fetch <code>["python", "javascript", "rust", "go", "kotlin"]</code> and then <code>["swift", "ruby", "php", "scala", "elixir"]</code>. Both batches come back with something at 100. That doesn't mean the top term in each batch is equally popular — each batch is divided by <strong>its own</strong> maximum.</p>
<p>Trends values are: share of searches for the term in that place and time, divided by the maximum share among the terms in the request, times 100, rounded to an integer. So concatenating batches gives you numbers on different, unknown scales.</p>
<h3>Fix: an anchor term in every batch</h3>
<p>Put the same anchor term in every request, then rescale each batch so the anchor lines up:</p>
<pre><code class="language-python">import pandas as pd

def compare(keywords, anchor, **kw):
    others = [k for k in keywords if k != anchor]
    frames, ref_sum = [], None
    for i in range(0, len(others), 4):                 # anchor + 4 = 5 terms
        batch = [anchor] + others[i:i + 4]
        df = pd.DataFrame(fetch(batch, **kw)).set_index("date").astype(float)
        s = df[anchor].sum()
        if s == 0:
            raise ValueError("Anchor is zero in this batch; pick a bigger anchor")
        ref_sum = ref_sum or s
        df = df * (ref_sum / s)                        # align to batch 1
        frames.append(df if not frames else df.drop(columns=anchor))
    out = pd.concat(frames, axis=1)
    return out * 100 / out.max().max()                 # back to 0-100 overall

data = compare(["python", "javascript", "rust", "go", "kotlin",
                "swift", "ruby", "php", "scala", "elixir"], anchor="javascript")
print(data.mean().sort_values(ascending=False).round(1))
</code></pre>
<p>Why the <strong>sum</strong> of the anchor instead of week-by-week ratios? Because every value is a rounded integer. A week where the anchor is 2 vs 3 is a 50% "difference" from rounding alone. Summing over the whole period smooths that out.</p>
<h3>Choosing the anchor</h3>
<ul>
<li>It should be <strong>in the middle</strong> of your keyword set. If it's huge, small terms round to 0 next to it. If it's tiny, big terms flatten its signal to 0–1.</li>
<li>If your list spans several orders of magnitude, chain through a <strong>bridge</strong> term: batch A = big anchor + medium term; batch B = medium term + small terms. Multiply factors along the chain. Each hop adds some error, so keep chains short.</li>
<li>Anything whose best raw value is under ~5 in its batch is low precision. Flag it rather than treating it as a solid number.</li>
</ul>
<h3>Other sources of noise</h3>
<ul>
<li><strong>Sampling.</strong> Google computes Trends on a sample, so the same request on different days can differ by a few points. Fetch all batches in one session, and for important decisions average a couple of runs.</li>
<li><strong>Identical parameters.</strong> <code>geo</code>, <code>timeframe</code>, category and property (web / YouTube / News / Shopping / Images) must be the same in every batch, or the anchor trick is meaningless.</li>
<li><strong>Partial last period.</strong> Drop rows with <code>isPartial == True</code> before computing averages.</li>
</ul>
<p>For academic-grade calibration across thousands of queries, look at G-TAB (Google Trends Anchor Bank) from EPFL, which builds a whole ladder of anchors.</p>
<h2>Exporting to CSV / Google Sheets</h2>
<p>Once everything is on one scale, a wide CSV is the most convenient format for spreadsheets:</p>
<pre><code class="language-python">data.drop(columns="isPartial", errors="ignore").to_csv("trends_wide.csv")
summary = pd.DataFrame({
    "average": data.mean(), "peak": data.max(), "peak_date": data.idxmax(),
    "latest": data.iloc[-1],
}).sort_values("average", ascending=False)
summary.to_csv("trends_summary.csv")
</code></pre>
<p>Import <code>trends_wide.csv</code> into Google Sheets (File → Import) and you can chart all series together, which you can't do with batches of 5 on separate scales.</p>
<h2>Which option should you use?</h2>
<table>
<thead>
<tr>
<th>Situation</th>
<th>Reasonable choice</th>
</tr>
</thead>
<tbody><tr>
<td>A few comparisons, occasionally</td>
<td>Google Trends website or pytrends</td>
</tr>
<tr>
<td>You have official API access</td>
<td>Official API</td>
</tr>
<tr>
<td>Dozens of keywords, one-off analysis</td>
<td>pytrends + anchor code above + patience</td>
</tr>
<tr>
<td>Recurring jobs, many keywords, no time to babysit 429s</td>
<td>A hosted API / scraper</td>
</tr>
</tbody></table>
<h2>Honest limits (of every option)</h2>
<ul>
<li>Trends is <strong>relative</strong>, not search volume. "Twice as high" means twice the share of searches, not a number of searches.</li>
<li>Very small terms are often reported as 0 relative to bigger ones. No method recovers information Google doesn't return.</li>
<li>Results vary slightly between fetches because of sampling.</li>
<li>Unofficial methods can break whenever Google changes its internal endpoints.</li>
</ul>
<h2>If you'd rather not run this yourself</h2>
<p>Disclosure: I built a hosted version of exactly this workflow as an Apify Actor — <a href="https://apify.com/satin_armeria/google-trends-advanced-comparator">Google Trends Compare – Bulk 100 Keywords API</a>. You send up to 100 keywords; it handles batching, anchor/bridge chaining, retries and proxies, and returns one 0–100 scale as CSV/JSON plus a summary (average, peak, latest) and an optional self-check against a direct Google query. It's pay-per-use: about $2 per 1,000 keywords compared. The free <a href="https://github.com/carlos-calatayud/google-trends-compare-100-keywords">MIT Python code</a> does the same chaining if you prefer to run it yourself.</p>
<p>Questions or a case where the anchor method gives odd results? Leave a comment — those edge cases are the interesting part.</p>
]]></content:encoded></item><item><title><![CDATA[How to Compare 100 Google Trends Keywords on One 0-100 Scale (Beyond the 5-Term Limit)]]></title><description><![CDATA[Originally published on DEV. Google Trends only compares 5 terms per request (8 groups in the new Explore UI), and every request is rescaled so its own…]]></description><link>https://carloscalatayud.hashnode.dev/how-to-compare-100-google-trends-keywords-on-one-0-100-scale-beyond-the-5-term-limit</link><guid isPermaLink="true">https://carloscalatayud.hashnode.dev/how-to-compare-100-google-trends-keywords-on-one-0-100-scale-beyond-the-5-term-limit</guid><category><![CDATA[data analysis]]></category><category><![CDATA[SEO]]></category><category><![CDATA[web scraping]]></category><category><![CDATA[google trends]]></category><category><![CDATA[Python]]></category><dc:creator><![CDATA[Carlos Calatayud]]></dc:creator><pubDate>Sat, 24 Oct 2026 18:37:22 GMT</pubDate><content:encoded><![CDATA[<p><em>Originally published on <a href="https://dev.to/carloscalatayud/how-to-compare-100-google-trends-keywords-on-one-0-100-scale-beyond-the-5-term-limit-1fii">DEV</a>.</em></p>
<p>Google Trends only compares 5 terms per request (8 groups in the new Explore UI), and every request is rescaled so its own peak is 100. A "40" in one chart and a "40" in another are not the same number. Here is how to put 100 keywords on one 0-100 scale.</p>
<h2>Why you can't just paste batches together</h2>
<p>Each request is normalized independently. If batch A contains a giant term, everything in A gets squashed toward zero; if batch B only has small terms, one of them gets stretched to 100. Concatenating them is merging two charts with different y-axes.</p>
<h2>The fix: anchor chaining</h2>
<ol>
<li>Pick an anchor: a steady, mid-to-high volume term (avoid spiky or ambiguous ones).</li>
<li>Split your keywords into groups of 4 and add the anchor to each group (5 per request).</li>
<li>Fetch every group with identical geo, timeframe, category and property.</li>
<li>For each group g: <code>factor_g = sum(anchor in reference group) / sum(anchor in group g)</code>. Use sums over the whole series, not per-point ratios: values are rounded integers, so a single point can be off by +/-25%.</li>
<li>Multiply every series in group g by <code>factor_g</code>, keep the anchor once, then divide everything by the global max and multiply by 100.</li>
</ol>
<h2>Pitfalls</h2>
<ul>
<li><strong>Rounding:</strong> a keyword 200x smaller than the anchor reads 0. Use a smaller "bridge" keyword already placed on the common scale to chain down.</li>
<li><strong>Sampling:</strong> Google samples searches, so the same request can differ by a few percent. Fetch all batches in one session.</li>
<li><strong>Check yourself:</strong> re-query a few keywords directly against each other and compare with your stitched numbers.</li>
</ul>
<h2>Free code</h2>
<p>The method is open source (MIT) here: <a href="https://github.com/carlos-calatayud/google-trends-compare-100-keywords">https://github.com/carlos-calatayud/google-trends-compare-100-keywords</a></p>
<h2>Hosted version</h2>
<p>If you don't want to manage batches, rate limits and 429s, I built an Apify Actor that does this for up to 100 keywords per run and self-checks the result against a direct Google query: <a href="https://apify.com/satin_armeria/google-trends-advanced-comparator">https://apify.com/satin_armeria/google-trends-advanced-comparator</a> . It costs $2 per 1,000 keywords compared (no-data keywords aren't charged). Disclosure: I'm the author.</p>
]]></content:encoded></item></channel></rss>