--- title: "Research providers | SEO Skill" description: "Connect optional research providers locally, control paid requests and combine keyword, domain, competitor, link and AI answer evidence with first-party data." canonical: "https://seoskill.dev/docs/research-providers" language: "en" --- # Research providers Connect optional research providers locally, control paid requests and combine keyword, domain, competitor, link and AI answer evidence with first-party data. Research a market without replacing the evidence from your own site. Optional providers add keyword estimates, result snapshots, domain footprints, ranking pages and competitor rows. Search Console still supplies owner-verified search performance. Crawls still supply current page and technical evidence. DataForSEO supports the broadest set of current research reports. Semrush Version 3 supports bounded keyword, domain, ranking-page and search-competitor research. Ahrefs API v3 supports those shared research reports plus attributed Domain Rating and bounded backlink evidence. The report contracts are provider neutral, so the adapters return the same evidence shape where their capabilities overlap. A report fails clearly when a selected provider does not support its operation or market. Use the [Semrush guide](https://seoskill.dev/docs/semrush) for the exact key, connection commands, supported reports and API-unit behavior. Use the [Ahrefs guide](https://seoskill.dev/docs/ahrefs) for connection, supported reports, Domain Rating attribution, backlink evidence and API-unit limits. ## Connect DataForSEO on this computer Create an API login and API password in DataForSEO, then run: ```sh seo providers dataforseo connect seo providers dataforseo status --check ``` The first command validates the credentials with the free account endpoint. The second command can repeat that check and show the connection state. The password is saved in the system keychain when available, with a private local file fallback. It is not stored in a project profile, report, cache entry or spend record. Agents and CI can use environment variables without saving credentials: ```sh export SEO_DATAFORSEO_LOGIN='your-dataforseo-login' export SEO_DATAFORSEO_PASSWORD='api-password' seo providers dataforseo status --check --json ``` Do not put these values in a repository, command history, report parameters or an issue. Remove locally saved credentials with: ```sh seo providers dataforseo disconnect ``` ## Set local cost and work limits first Provider calls can be charged. Inspect the current local limits and account context before running broad research: ```sh seo providers dataforseo limits seo providers dataforseo spend ``` Set hard daily and monthly limits in US dollars, plus maximum requests and requested rows per report: ```sh seo providers dataforseo limits \ --daily-notice 1 \ --daily-limit 5 \ --monthly-limit 25 \ --requests 20 \ --rows 10000 ``` The CLI reads current endpoint prices from the free account response, reserves estimated spend before a paid request, and records the actual provider cost afterward. Unknown or unsafe price evidence blocks paid work instead of assuming it is free. Local request and row limits also stop accidental fanout inside one report. Cached results avoid repeating paid work during their retention window. A report includes cache hit, miss or bypass state, observation time, estimated cost, actual cost and provider task ids. Use `refresh: true` only when the decision needs newer provider evidence. ## Know what a provider request sends The software stores provider credentials, caches, histories and spend records locally. A live provider request still sends the inputs needed for that operation directly to the provider. | Operation | Inputs sent to DataForSEO | | --- | --- | | Keyword, result and domain research | Keywords or seeds, domains or URLs, market, location, device, filters and limits required by the report. | | Link research | The target domain or URL, provider filters and result limits. | | Indexed AI mention research | Target names and aliases, surface, market and result limits. | | Fixed AI prompt observation | The full prompt, selected model, country, web search setting and output limit. | | First-party enrichment | Selected Search Console query text or a derived research seed, only when `includeExternal` or `includeSerps` is explicitly enabled. | First-party enrichment does not send Google OAuth tokens, Google account identity, Search Console property IDs, clicks, impressions, click-through rate, positions, or Google Analytics rows. The report keeps provider estimates and first-party metrics separate. DataForSEO processes live inputs and responses under its [privacy policy](https://dataforseo.com/privacy-policy). As of 23 July 2026, that policy says API task data is stored for 365 days. Local cache removal and `seo providers dataforseo disconnect` do not delete provider-side task data. Use `seo reset --yes` to remove saved credentials, local configuration, caches, histories, spend records, logs and saved reports. Environment credentials remain under the control of your shell or runtime. Local DataForSEO, Semrush and Ahrefs file imports are different. They are read from disk and are not uploaded to a provider or the project maintainer. ## Discover the research reports The normal CLI and MCP discovery flow exposes the same reports: ```sh seo reports list --category opportunities --json seo reports describe competitive-opportunities --json seo reports describe domain-overview --json seo reports describe competitor-keyword-gap --json seo reports describe ai-mention-research --json seo reports describe ai-prompt-observations --json ``` Run a report through the existing catalog command. There are no separate provider-named report commands. ```sh seo reports run domain-overview \ --params '{"domain":"example.com","countryCode":"GB","languageCode":"en"}' \ --json seo reports run competitive-opportunities \ --params '{"target":"example.com","seeds":["seo reporting software"],"countryCode":"GB","languageCode":"en","keywordProvider":"semrush","serpProvider":"dataforseo"}' \ --json seo reports run serp-competitors \ --params '{"keywords":["blue widget","red widget"],"targetDomain":"example.com","countryCode":"GB","languageCode":"en"}' \ --json ``` The existing `link-evidence` report accepts a live Ahrefs or DataForSEO target through the direct `seo links` command. It does not add another report or MCP tool. The current domain research endpoints use country and language. They do not accept a city, postcode or device. Use `serp-results` when you need a current location and device-specific result snapshot. ## Use ranked-keyword exports without API access You can run four domain research reports from local DataForSEO, Semrush or Ahrefs exports. No provider connection or paid request is needed. The report still uses the same typed rows and evidence rules as the live API path. Put the report parameters in a JSON file when the file paths make the command hard to read: ```json { "target": "example.com", "countryCode": "GB", "languageCode": "en", "researchFiles": [ { "dataset": "ranked-keywords", "file": "./exports/example-keywords.csv", "provider": "semrush", "exportedAt": "2026-07-20T12:00:00Z", "columns": { "keyword": "Search Term", "url": "Destination", "position": "Current Rank", "searchVolume": "Monthly Demand" } } ] } ``` Run it through the normal report catalog: ```sh seo reports run ranked-keywords \ --params-file ./ranked-keywords-run.json \ --json ``` The importer accepts CSV, TSV, JSON arrays, JSONL and NDJSON. It detects UTF-8, UTF-16, comma, semicolon and tab-separated files automatically. This includes Ahrefs downloads that use UTF-16 and tabs despite having a `.csv` name. You do not need to convert the file first. Each current ranking needs a keyword, an absolute HTTP or HTTPS ranking URL, and a positive position. The importer also recognises common export names such as `Query`, `Phrase`, `Current URL`, `Landing Page`, `Current Position`, `Search Volume`, `Volume`, `Difficulty`, `KD`, `CPC`, `Competition`, `Keyword Intents`, `Current organic traffic` and `Traffic`. Leave `columns` out when those common headings match. Add it when an export uses different names. A named mapping overrides automatic matching for that field. The accepted canonical fields are `keyword`, `url`, `position`, `absolutePosition`, `searchVolume`, `keywordDifficulty`, `cpc`, `paidCompetition`, `intent`, `resultCount`, `estimatedTraffic`, `resultType` and `searchVolumeUpdatedAt`. Copy source headings exactly. The importer rejects missing headings and stops one source column from being assigned two meanings. This avoids treating a percentage as a visit count or a previous position as the current rank. An export can contain separate organic and search-feature rows for the same keyword and page. `ranking-pages` counts that as one ranked keyword. Its ranking buckets and estimated traffic keep both result rows because each one describes a different search appearance. Those totals can be higher than the unique keyword count without being duplicates. Set `countryCode` and `languageCode` to the market used for the export. The file does not carry enough consistent metadata for the importer to verify that choice. Keep every file in one run on the same market. Ambiguous fields stay missing. For example, `Traffic %` is not treated as estimated monthly visits. Use a `YYYY-MM-DD` date or an ISO timestamp with a timezone for `exportedAt`. The same input works with these reports: | Report | What the file supplies | | --- | --- | | `ranked-keywords` | Ranked terms, URLs, positions and any optional metrics present in the export. | | `ranking-pages` | Page groups, retained keyword counts and repeated URL patterns calculated from the imported rows. | | `serp-competitors` | Repeated domains across the requested terms when the files contain more than one domain. | | `competitor-keyword-gap` | Site and competitor rows from up to four exports from the same provider. | The gap report calls a term relevant only when at least two meaningful words also appear together in one retained Search Console query. Common words such as `new`, `search`, `page` and `tool` do not count. A one-word match stays weak, words found across separate queries are not combined into stronger evidence, and search operators such as `site:` and `filetype:` are ignored. `domain-overview` still requires a live provider. A ranked-keyword export cannot safely recreate the provider’s full domain totals, traffic estimate or movement history. Pass one to four files from the same provider. Separate files are useful for a site and up to three competitors. A single run can normalize at most 100,000 rows across all files. CSV, TSV, JSONL and NDJSON files are capped at 50 MB each. Regular JSON arrays are capped at 10 MB because they are loaded as one value. Every file gets its own entry under `evidence.imports`. That entry records the provider, export time, detected encoding and delimiter, SHA-256 file hash, included fields, explicit column mapping, bytes read, total file rows, supplied rows, valid rows, filtered historical rows, rejected rows, duplicates and row cap. Ahrefs comparison rows with no current URL or position are filtered historical rows, not broken records. The report scans a bounded file to finish its hash and file-row count even when the row cap stops normalization. Missing rows remain unknown because a local export does not prove full provider coverage. ## Research indexed AI mentions and citations `ai-mention-research` checks one named target in a provider-indexed set of AI answers. It can compare up to five named competitors in the same dataset, return cited domains and bounded question samples, mark citations from an owned domain, and compare sample wording with retained Search Console queries. ```sh seo reports run ai-mention-research \ --params '{"target":{"label":"Example Analytics","aliases":["Example"]},"competitors":[{"label":"Competitor Cloud"}],"domain":"example.com","surface":"google-ai-overview","countryCode":"GB","languageCode":"en","location":{"code":2826},"site":"sc-domain:example.com","sampleLimit":10}' \ --json ``` The default path makes two paid live API requests: one for metrics and one for question samples. Set `includeSamples` to `false` for a metrics-only run with one paid request. The report accepts one location and language per run, at most five competitors, at most five aliases per target, and at most 25 retained samples. Cached results use the local provider cache and do not repeat paid work until their retention window expires unless `refresh` is true. DataForSEO currently supports Google AI Overview and ChatGPT mention records. Its ChatGPT data is limited to United States English, so use country code `US`, language code `en`, and US location code `2840` for that surface. Other market combinations fail before a paid request. Read the result in this order: 1. `dataStatus`, exact surface and market, source status, coverage, cache, cost, task ids, and warnings. 2. Provider metrics for each target, then `targetComparisonShare`. That share describes only the supplied comparison set. 3. Question samples, observation dates, model, answer truncation, cited URLs, and `ownedSources`. 4. Optional Search Console source limits and each sample’s bounded lexical matches. 5. Repeated question patterns, data-source briefs, findings, caveats, and next steps. Provider-indexed records are not live prompt observations or a complete census. Mention count and AI search volume do not prove traffic, sentiment, citation quality, authority, future visibility, or demand for a new page. Search Console overlap is a lexical heuristic over retained rows. Run `ai-prompt-observations` when the current answer would change the decision. For programmatic SEO, repeated question terms can reveal a useful research direction. They do not approve a template. Confirm shared intent, existing page coverage, data rights, stable identifiers, fields, geographic coverage, update frequency, missing values, duplicates, useful page variation, and representative output before increasing page count. ## Record current AI answers for fixed prompts `ai-prompt-observations` runs one to five fixed prompts across one to four explicit current models from ChatGPT, Claude, Gemini, or Perplexity. The full set of prompt and model combinations is capped at 20 paid live requests. Start with one prompt and one model, then add only observations that can change a decision. ```sh seo reports run ai-prompt-observations \ --params '{"prompts":[{"id":"analytics-tools","group":"commercial","prompt":"Which privacy-friendly analytics tools suit a small publisher?"}],"models":[{"surface":"chatgpt","model":"current-model-name"}],"target":{"label":"Example Analytics","aliases":["Example"],"domains":["example.com"]},"competitors":[{"label":"Competitor Cloud"}],"countryCode":"GB","languageCode":"en","site":"sc-domain:example.com"}' \ --json ``` The requested model is the exact model you chose. The effective model is the model the provider says actually ran. Fan-out queries are extra supporting searches the provider reports using to form the answer. Keeping these fields separate prevents a provider-side model change from looking like a visibility change. Model names are exact. Before paid work, the report checks the provider’s free current model catalog. A retired or misspelled model fails with current choices and confirms that no paid request started. This avoids silently moving a saved prompt set to a new model. Perplexity always uses web search. Other models must advertise web-search support when `webSearch` is true. Read the result in this order: 1. `dataStatus`, warnings, request count, provider, market label, and optional Search Console status. 2. Each observation’s coverage, cache state, requested and effective model, checked time, answer, citations, token use, task id, and exact returned cost. 3. Named-target matches and cited domains. These describe only the retained answer and the aliases or domains you supplied. 4. Comparison status. Only a fresh, complete, untruncated observation with the same provider, prompt, requested and effective model, market label, web-search setting, and output-token limit supports a change. 5. Supporting-search themes in `fanOutThemes`, optional Search Console overlap, findings, caveats, and next steps. The preflight estimate contains provider base fees only. The response records the exact task cost after token and web-search charges are known. If any request fails, actual total cost stays partial or unknown instead of pretending the missing request cost zero. Cached results do not support a new change claim. Answers and bounded history stay in the local SQLite store. The history keeps at most 90 observations per exact configuration, 10,000 observations in total, and 128 MB of logical payload. Answers are capped at 24 KB, citations at 25, fan-out queries at 20, and structured agent output at the shared 96 KB budget. The country and language fields label and separate a fixed prompt set. AI response endpoints do not expose identical location or language controls across every surface, so prompt wording remains the direct language instruction. One answer is not a universal result, visibility score, assistant ranking, or proof that a target never appears. Supporting-search themes and Search Console word matches are bounded research leads. Before planning a programmatic template, validate keyword demand, current search-result intent, existing page coverage, source rights, stable identifiers and fields, geographic coverage, update frequency, missing values, duplicates, useful page variation, and representative output. No new page remains a valid result. ## Start a local search investigation with first-party evidence `local-search-demand` finds retained Search Console queries whose wording contains an explicit place name, nearby phrase, UK postcode or a US ZIP code with supporting local wording. Add the real place names, aliases and known ZIP codes for the market because automatic detection is deliberately conservative. ```sh seo reports run local-search-demand \ --params '{"site":"sc-domain:example.com","locationTerms":["london","manchester"],"days":90}' \ --json ``` The default path makes one bounded Search Console request and no paid provider request. It returns the local query and landing-page evidence, filtered-row counts, action labels and repeated URL patterns. The patterns are prompts to inspect working local page families, not proof that more pages should exist. Request live context only for a short decision-critical set: ```sh seo reports run local-search-demand \ --params '{"site":"sc-domain:example.com","locationTerms":["london"],"includeSerps":true,"countryCode":"GB","languageCode":"en","location":{"name":"London,England,United Kingdom"},"device":"mobile","serpLimit":3}' \ --json ``` That path adds at most three current organic snapshots through the same cache, spend limits and provider-neutral result contract as `serp-results`. Read the recorded location, device, observation time, coverage, cache and cost for every snapshot. The report also groups recurring organic domains as unclassified search competitors and retains bounded local-pack listing rows with their observed rank, title, URL, phone, rating and provider identifier when returned. These rows do not prove listing ownership, complete Maps coverage or Google Business Profile performance. Add optional visitor geography only when it would change a landing-page or template decision: ```sh seo reports run local-search-demand \ --params '{"site":"sc-domain:example.com","locationTerms":["london","manchester"],"googleAnalyticsPropertyId":"123456789","analyticsLimit":5000}' \ --json ``` This makes one bounded Google Analytics request for landing page, country, region, city and sessions. Query strings are removed from the landing-page dimension before an exact path join. Unmatched, missing, malformed, duplicate, capped and quality-warning states stay visible. The returned geography describes measured sessions for retained landing pages. It does not establish which Search Console query produced a session. A place in the query does not prove the searcher’s physical location. Search Console average position is not an exact local rank. Keep query wording, landing-page geography and exact live results separate when deciding whether to improve a current page, review overlapping pages or investigate a new local template. ## Review live referring-link evidence Run the bounded provider path for a domain: ```sh seo links --provider dataforseo --target example.com --json ``` Add a Search Console property when you own the target and want first-party page evidence in the same result: ```sh seo links --provider dataforseo --target example.com \ --search-site sc-domain:example.com --json ``` The default run makes one summary request, then requests up to 100 live representative backlinks with one row per referring domain. `--row-limit` can raise that bounded request to 500. It does not fetch every backlink or fan out across every target page. Both requests run sequentially, use the local provider cache, and pass through the same report spend limits as keyword and domain research. Read these result areas in order: 1. `dataStatus`, warnings, caveats and source provenance. 2. `providerSummary` for current provider counts and named metrics. 3. `providerEvidence` for coverage, filters, cache, cost and observation time. 4. `findings` and `targetContext` for saved-crawl and Search Console evidence. 5. `targetCounts` and the bounded representative `links` list. A matching saved crawl can flag linked targets observed with an error response, redirect, conflicting canonical or non-indexable state. Search Console can add retained clicks and impressions for the same target. Those sources stay separate: a provider row does not prove link quality, a missing Search Console row is not zero, and a saved crawl does not prove the page still has the same state. Open the referring URL and check the target live before acting. Provider rank and spam metrics retain their DataForSEO names and 0 to 100 scales. They can help order a review, but they are not Google ranking factors, universal authority scores or interchangeable with another provider’s metric. ## Start with first-party opportunities Use `keyword-opportunities` when the question starts with your own Search Console data. It combines quick wins, second-page evidence, query clusters, repeated first-party URL patterns and optional provider metrics in one bounded review queue. Use the provider-led reports for different questions: - `competitive-opportunities` turns a topic into a bounded keyword and competitor investigation order with current results and optional link context. - `domain-overview` gives a quick country-level estimated footprint. - `ranked-keywords` opens bounded keyword and ranking-page rows. - `ranking-pages` finds pages behind the footprint and repeated path patterns. - `serp-competitors` finds domains that recur across an explicit keyword set. - `competitor-keyword-gap` compares up to three relevant competitors with retained Search Console themes and the provider footprint for your site. Pass a Search Console property only when it matches a site you own. Competitor domains stay provider-only. The report never pretends to have first-party evidence for somebody else’s site. ## Classify competitors before comparing gaps A recurring result domain can be a publisher, directory, community, marketplace or business. `serp-competitors` leaves undeclared site types as `unknown`. Review those domains, then pass a small relevant set to `competitor-keyword-gap`. The gap report does not label every competitor term an opportunity. It keeps four states separate: 1. Already observed in retained Search Console rows. 2. Already ranked in the provider footprint for the site. 3. Relevant gap candidate based on bounded first-party theme overlap and competitor evidence. 4. Unverified competitor term. An absent Search Console row means only that the bounded retained data did not contain it. Search Console can omit anonymised queries, and row caps or the date range can hide evidence. ## Review programmatic SEO patterns carefully Ranking pages and competitor gaps inspect repeated URL structures. That helps an agent find possible page families without downloading a whole domain. A path pattern does not prove shared intent, quality, useful data or that more pages should be created. For a plausible new template, the gap report returns a bounded data-source brief. Check: - stable identifiers, fields, units and geographic coverage; - update frequency, missing values, duplicates and malformed records; - access rights, attribution, rate limits and local cache retention; - the useful information that makes each generated page distinct; - representative sparse, stale and boundary records before scaling. For an existing first-party template, run `pseo-audit` before adding variants. For a new family, inspect current results and representative competitor pages, then complete the data-source research before implementation. ## Read each source by what it can establish Provider volume, difficulty, traffic, intent and visibility are estimates. Search Console clicks and impressions are first-party measurements for the selected property and dates. A crawl observes the current site version. A live result report observes one query, market, device and time. Useful analysis can combine these sources, but it should not blend their numbers into one unexplained score. Read source coverage, request filters, observation dates, cost, warnings and caveats before the findings. Verify a shortlisted keyword with current results before changing content or a programmatic template.