Search

Only Google Gemini and Microsoft Copilot run on a search index their parent company crawls itself. Anthropic Claude rents Brave Search , Mistral Le Chat rents Brave too, OpenAI ChatGPT rents Bing plus its own crawler, and Meta AI rents both. The key clue: Claude’s web_search tool exposes a literal BraveSearchParams field, and citation overlap with Brave runs about 86.7%.

Only Five Companies Actually Crawl the Open Web

Before mapping each AI lab to its backend, the key constraint is simple: only five operators crawl the open web at scale. Everything else sold as a “search engine” resells one of those indexes. The five are Google, Microsoft Bing, Yandex, Baidu, and Brave Search, with Mojeek as a much smaller niche sixth.

You can build a personal AI research assistant that ingests PDFs, web bookmarks, and notes into a local ChromaDB vector store. It answers questions with cited sources using Ollama and a local LLM like Llama 4 Scout. The system uses sentence-transformers to embed your documents into a searchable index. When you ask a question, it pulls relevant passages and writes an answer that cites the exact source and page. The whole stack runs offline on consumer hardware, so your research data stays private.

You can build a private AI search engine modeled on Perplexity . You combine SearXNG with a local language model running through Ollama . Here is the stack. SearXNG pulls results from many search engines at once. A Python scraper fetches and cleans the actual page content. The LLM then turns everything into a cited answer with inline references like [1], [2]. No API keys, no telemetry, no query logging to third-party AI services. A machine with 12 GB VRAM runs the whole pipeline, and most queries come back in 5-15 seconds.

Pair Meilisearch v1.12’s fast REST API with HTMX 2.0’s hx-get and hx-trigger attributes, and you get a real-time, typo-tolerant search box that returns results in under 50ms. You write no custom JavaScript and pull in no React or Vue. The server renders HTML fragments that HTMX swaps into the DOM, so the whole search box stays under 15 KB of total JS. This post covers the full setup, from Docker Compose to a working search UI with faceted filtering.

Need to search hundreds of thousands of short strings, like tags, usernames, or SKU codes? Heavyweights like Elasticsearch are often overkill. For a dataset of 200k tags, each about 10 characters long, what you want is low latency, a small memory footprint, and easy deployment.

This guide sorts the best modern tools into two groups: standalone servers and in-process libraries. That split helps you pick the right fit for your next project. These engines also work well for adding search to static site generators.