Scraping a Website

  1. Click Scrape website.
  2. Step 1 Configure — enter Website URL (required), optional Tags, Category (defaults to website). userMetadata is not available for website sources in v1.
  3. Click Continue.
  4. Step 2 Verify DNS — follow Record type, Host/name, Value, TTL, and Expires guidance, then Verify DNS when required for your page credits.
  5. Step 3 Crawl progress — watch Status and Pages ingested.
  6. Use View knowledge library or Start new scrape when finished.
Scraping a website

Page credits messaging explains pages remaining without DNS verification vs verified websites.