Photon: Building a retrieval and ranking engine from scratch

Introduction

Perplexity’s AI-native search stack retrieves and ranks candidate pages, surfacing the most relevant content for the model to ground its answers. Originally, this stage ran on an open-source engine that we adapted to our system. However, as our index and workloads grew, we hit hard limits on costs, tail latency, and node recovery time that we couldn't resolve within the existing third-party platform.

To overcome these challenges, we built Photon, a new in-house retrieval and ranking engine. This continues our approach of building AI-first search infrastructure from scratch, as we did with CobbleDB, our distributed key-value store for serving page records, and our Search API.

Photon uses compact data formats to read and decode only what each query needs, with batch-aware caching and asynchronous reads to reduce disk access and overlap waits. Index building runs separately from query serving, updating one serving group at a time and warming its caches before routing live queries to it.

Photon now powers our search pipeline and is also the foundation of a new fast preset for our Search API, with single-search-call latency of 160 ms at p50 and 230 ms at p95. Across six public benchmarks, the fast preset reduced estimated model-plus-search cost per task by 68% while maintaining aggregate task quality comparable to the default preset. We are releasing it today so developers can access these improvements.

Expand

Expand

What the retrieval and ranking engine requires

For each query, the retrieval and ranking engine searches a large, distributed index to select and order a small set of relevant pages within the search stack’s latency budget. The system must also keep the index up to date as new pages arrive and existing pages change, without letting that work slow down live queries.

We evaluated several open-source options and chose a mature, general-purpose retrieval and ranking engine that met our initial needs. We then adapted it to our architecture and workloads, maintaining stable latency and throughput at production scale.

Where our previous system fell short

We maintained our own fork to adapt the engine to our growing index, but addressing key limitations increasingly required redesigning large parts of the platform. Several constraints proved difficult to resolve without fundamental architectural changes:

1. Scaling costs. Both the number of documents we store and the amount of per-document data we need to retrieve and rank have been growing over time. This scaling put pressure on our hardware, especially the amount of RAM available. To control costs, we needed more compact data representations and finer control over what stays in memory and what is read from disk on demand.

2. Poor tail latency. On production traffic, the old system’s p99 response time was about 800 ms. The dataset exceeded available RAM, so we could not use mlock to keep it entirely in memory. Accessing nonresident data could trigger major page faults, stalling query execution while the operating system read from disk.

The engine also periodically merged its main disk index with smaller indexes flushed from memory. During this disk index fusion process, the number of page faults rose even higher, with p99 latency reaching approximately 1.2 s for 10–15 minutes. As the caches warmed up, the frequency of page faults and latency both decreased, suggesting that cold reads were a major contributor to the spikes. Preloading data and warming the index reduced the spikes but did not adequately address high tail latency.

3. Lengthy recovery and synchronization. In the previous system, deploying an additional cluster and synchronizing its data could take more than a week. Synchronization was also required when adding groups of nodes to hold additional data copies or recovering from node failures. During recovery, a larger share of search responses was partial, which lowered final-answer quality scores.

Ultimately, we concluded that it would be simpler and cheaper to build from scratch rather than continuing to adapt and maintain our fork. Drawing on our experience operating the previous system, a small engineering team worked alongside a swarm of persistent coding agents to build Photon.

Where Photon fits into the search stack

Photon is designed for two primary workflows: serving search requests and ingesting updates from the processing pipeline.

A request from the higher-level search service reaches the Photon broker through a load balancer. The broker selects a shard group and forwards the request to each shard, monitoring timeouts and issuing additional requests when needed.

Each shard performs retrieval, initial ranking, and second-stage ranking, then returns its best candidates to the broker. The broker combines these candidates into a single ranked list and fetches key document fields for the selected pages from the relevant shards. It then returns these results to the higher-level search service, which gathers the page content needed for the agent to generate a grounded answer.

Index updates follow a separate path. Pillar, our publishing layer, prepares the data and stores it in YTsaurus tables. Indexers convert these tables into index structures that shards can load for search. The controller coordinates data preparation, index building, and deployment of new index versions to serving groups. Before handling live traffic, each shard replays real search queries to warm its caches.

Expand

Designing Photon for our workflows

Three design choices make Photon well suited to our workload: compact data representations, caching and asynchronous disk reads, and separating index updates from query serving.

Data representations for retrieval and ranking

Retrieval and ranking access document data differently. Lexical retrieval starts with query terms and finds documents that contain them. Ranking then needs detailed information about each selected document, including term frequencies and positions.

Photon uses two complementary representations to support these access patterns. An inverted index stores a list of matching documents for each term, called a posting list, to support candidate selection. Separate per-document records provide the data needed for ranking. We also optimize how posting lists are stored and traversed to improve selection within a fixed work budget.

Specialized data representations enable us to store key data more efficiently. A more compact inverted index reduces the amount of data read or moved per query, improving latency by eliminating unnecessary I/O operations. These formats also make it possible to check whether a document appears in a list without fully decoding it, a property our traversal logic relies on to control how much work a query does under a fixed budget. Optimizing per-document records lets us store more data per document for ranking, while also reducing the scaling costs associated with our previous system.

The inverted index

For each term, the inverted index stores a posting list. These lists can range from a handful of documents to millions, so no single format works well for every list. Photon chooses the format based on the number of documents containing the term.

Short lists use an inline representation that fits within a single page and requires no block table. Longer lists that don’t fit the inline format are divided into blocks covering fixed ranges of document identifiers. Each block’s representation depends on its density: sparse blocks store sorted arrays of document-ID offsets, while dense blocks use bitmaps. Sparse blocks use galloping search, which advances in increasingly large steps to skip regions of the list. Dense blocks allow membership checks by looking up the bit corresponding to a document.

A term’s query frequency determines where its posting list is placed on disk. Frequently queried lists are stored near one another, concentrating hot data in a smaller storage region. Within each list, documents are ordered by a separate usefulness signal so that more promising candidates are encountered earlier.

Expand

Traversing lists within a work budget

Candidate selection requires traversing posting lists to find potentially relevant documents. Exhaustively scanning every list is expensive, so Photon uses a WAND-like algorithm that traverses only some lists while still accounting for matches in the others.

The algorithm divides lists into two groups. Driving lists advance the traversal and propose candidate documents. Probe lists contribute to the scores of those candidates but do not generate candidates themselves.

Candidate scoring combines weighted query-term coverage and a frequency contribution, with document-length normalization and a cap on gains from repeated terms. Photon first checks term presence in the arrays and bitmaps to bound each candidate’s maximum possible score. It reads exact term frequencies only for candidates that could still meet the selection threshold.

Selection then proceeds in three steps:

• Traversal: Start with a strict query-coverage requirement. If too few candidates are found, repeat with a relaxed requirement.
• Retention: Candidates are held in a bounded set using a score that combines coverage and frequency. As the collected top candidates improve, the admission threshold rises.
• Final selection: Final results for the pass are chosen from the retained set based on the frequency scores. Each field runs its own pass, and the results are merged using field weights.

The algorithm produces a candidate set of relevant documents to be used in further ranking.

Expand

Docblob

After candidate pages are selected, the ranking phase needs term frequencies, field coverage, and the locations and proximity of occurrences. Obtaining this data through the inverted index would require scattered accesses across many term-document pairs. In Photon, we store a compact per-document record called docblob that contains term frequencies, which fields a term appeared in, and where in the document it occurred. As a result, ranking a candidate only requires one lookup per document.

The format is built to avoid decoding more than necessary. Within a docblob, fields are grouped by text analyzer. Each group stores a sorted term list encoded with Elias-Fano, so a query term can be looked up directly without expanding the entire list first. A field mask marks where a term occurred, a frequency count tracks how often, and actual positions live in separate blocks per term-field pair encoded as position differences. Only the blocks for terms that actually matched get decoded, while everything else in the record stays in the compressed format.

That selective decoding lets docblob support storing more data per document without a proportional cost increase. Growing the record doesn't force more work per lookup, because ranking only ever touches the pieces relevant to the matched terms rather than the whole record.

Expand

Batched reads and working with memory

Even with the more efficient data representations, RAM still runs out before other resources, so we frequently need to access data stored on disk. Photon uses asynchronous disk reads and a cache optimized for batch reads to speed up these accesses, cutting the per-request wait time. This contributes to a significant improvement in tail latency compared to the previous system.

Expand

Overlapping disk waits

Ranking requires obtaining the features of a hundred documents located in different parts of a file. Our previous system used mmap to map the file into virtual address space so it could be read through memory pointers. However, accessing a page that was absent from memory still required a disk read. The thread had to wait for the data before continuing the computation, and this wait time added up when the process needed to read cold records sequentially.

Photon takes advantage of the fact that record offsets are known in advance, so we can submit several reads at once. This is managed by the reactor: dedicated worker threads submit batches through io_uring and pass the data they read to the calling code. Each thread maintains its own io_uring; batches are sent to the least-loaded reactor. Preregistered file descriptors and buffer pools reduce the overhead of repeated reads. This asynchronous execution allows the waits for several operations to overlap, reducing tail latency.

Reducing the number of drive accesses

Before submitting reads, Photon checks the entire batch in the in-memory cache. Only requests for missing records go to disk; the data read then populates the cache.

The cache is designed for batched reads. Specifically:

• There’s no separate allocation on a hit. Instead, data is copied into a buffer prepared in advance by the calling code.
• There’s no lock acquisition by the reader. Instead, it checks the data version and retries if it conflicts with a write.
• It uses CLOCK eviction, rather than requiring updates to a shared LRU list on every read.
• It uses CPU cache prefetching, where metadata and data for upcoming records are requested in advance.

These optimizations decrease latency without harming performance on the workloads we care about.

Building and updating the index separately from serving

In Photon, index building and query serving run on different nodes. Indexers transform data into ready storage structures, while serving nodes read them and perform retrieval and ranking. As a result, index building doesn’t compete with search queries for CPU, and resources for these tasks can be scaled independently.

This separation addresses the recovery and synchronization bottlenecks present in the previous implementation. Because index building ran inside the same cluster that served queries, recovering a failed node or adding a new copy of the data could mean reindexing documents in place, a process that took days and increased partial responses during recovery. Photon removes that coupling; indexers run as distributed jobs that produce versioned, ready-to-serve index structures independently of the serving nodes, so building the next version of the index no longer competes with live traffic.

Expand

Preparation and distributed building

A build starts by selecting documents for the active index and producing the source export. The controller records the source version, input tables, and expected number of shards. MapReduce jobs in YTsaurus prepare text, metadata, fragments, features, and precomputed embeddings into versioned tables partitioned by shard. Each indexer streams the prepared table, builds the structures for its shard, and merges text segments. The ready index, together with a description of its components, is uploaded to object storage.

Shard count and the number of concurrently running indexers are configured independently, so a build can run in several waves, or more machines can be allocated to work in parallel. Distributed preparation and parallel indexers allow the full web index to be built in a single-digit number of hours, a significant improvement over the previous system.

Deploying a new version

Before deployment, the controller checks that the new index is ready. If the data is incomplete, deployment is blocked automatically.

When there are multiple serving groups, the controller updates them one at a time. It first removes a group from broker routing and starts it on the new index version. Before serving requests, the shards go through a separate warmup stage based on search logs, which prepopulates the initial state and makes the transition to real traffic smoother. After warmup, the controller checks the index state and readiness, then returns the group to routing. The other groups continue serving requests during this time. Because operation state is persisted, a restart doesn't force a build to start over; the controller can reattach to preparation already in progress, reuse a ready result, or continue an interrupted rotation from the saved step.

Data update times differ across search verticals. For verticals with high freshness requirements, document delivery takes a single-digit number of minutes. An additional copy of the data can be deployed in around half an hour; ready indexes are loaded and attached to shards without reindexing the source documents.

Expand

Migration and validation within our search stack

Our first goal was to replace the old system with an engine that preserved functionality and search quality while reducing response times. We validated Photon in stages, starting with reproducible load tests and offline quality evaluations before moving to live traffic. Photon now powers retrieval and ranking in our production search stack, with substantially lower response times than the old system.

Load testing and evaluations

For performance testing, we built a load-testing tool that replays queries from our search logs at configurable arrival rates. It compares results across runs and applies statistical tests to assess whether observed performance differences are consistent. Separately, we evaluated Photon and the old system on the same datasets in our internal offline evaluation systems to check for quality regressions.

After these checks, we sent live queries to both engines simultaneously using shadow traffic. This allowed us to compare their performance and results on production workloads before changing which engine served users. We then enabled Photon for small subsets of users and gradually expanded the rollout, monitoring quality and performance throughout. The first version passed acceptance testing without quality regressions, and we continued the rollout until Photon served all production traffic.

Measured improvements in production

In our production workflow, p99 response time fell from roughly 800 ms in the old system to roughly 65 ms in Photon. This measurement covers all stages inside Photon, but excludes subsequent stages in the higher-level search stack. We also no longer observe the latency spikes previously associated with switching index versions.

Photon uses approximately 20% fewer equivalent serving machines than the old system’s content nodes while storing approximately 2.5 times as much data per document. After the migration, we used this additional data to improve ranking quality beyond that of the old system.

Photon also reduces the amount of data that must remain in memory. Pinning the same Photon dataset in RAM with mlock would require an estimated 4.6 times as much resident memory as Photon currently uses.

Evaluation of the fast Search API preset

We also used Photon as the basis for a new fast preset for our Search API, designed for latency-sensitive workloads. Per our measurements, single-search-call latency for the fast preset is 160 ms at p50 and 230 ms at p95, lower than or comparable to other Search API providers' self-reported numbers, though the reported tail percentiles differ.

We evaluated the fast preset on six agentic benchmarks: WideSearch, BrowseComp, DSQA, FRAMES, SEAL-0, and SEAL-Hard. Across these evaluations, it had the lowest estimated total model-plus-search cost and the highest aggregate task score among the configurations tested.

The fast preset pairs Photon with ranking optimized for agentic workflows, delivering comparable performance to the default preset at substantially lower cost. However, it does come with a trade-off in broader search quality. The fast preset uses less compute for ranking than the default preset. On internal benchmarks covering long-tail queries, broad coverage, and result diversity, it scores 0.24 points lower on relevance and approximately 3 percentage points lower on answer availability. We believe this is a sensible trade-off for agentic workflows, as the model’s own reasoning and knowledge can help compensate for the difference.

Expand

We recommend the fast preset for day-to-day agentic tasks. For more challenging or ambiguous questions that are less time-sensitive, the default preset offers more robust quality, justifying its higher latency and cost.

Conclusion

We are steadily moving our search stack onto Rust-based, in-house infrastructure. Replacing the open-source engine with Photon, our custom retrieval and ranking engine, is the latest step. Photon gives us full control over data representations, how reads are organized, and how new indexes are deployed, allowing us to optimize each layer for our workloads.

Photon has enabled faster search across Perplexity’s products and provides the foundation for our new fast preset in the Search API. Following the migration, p99 retrieval-and-ranking latency fell from roughly 800 ms to 65 ms. Our compact data formats also let us store 2.5 times as much data per document, giving retrieval and ranking access to richer document information.

The fast preset delivers single-search-call latency of 160 ms at p50 and 230 ms at p95. Across the six agentic benchmarks evaluated, it reduced estimated model-plus-search cost per task by 68% relative to the default preset while maintaining comparable aggregate task quality. These results demonstrate the practical benefits of infrastructure designed around our search workloads, both for Perplexity’s products and for developers building with our API.

We’re continuing to improve Photon’s performance and storage efficiency. Our goal is to support a larger index and richer document data on the same hardware while keeping search fast as our workloads grow.

Citation

@online{perplexity2026Photon,
  author = {{Perplexity} and Maksim Morozov and Bogdan Savko and Ruslan Kovalev and Aleksei Bykov and Artur Zayats and Denis Bykov and Nikita Kireev and Jarrod Bryan and Claire Beamer and Jeremy Yang},
  title = {{Photon: Building a retrieval and ranking engine from scratch}},
  date = {2026-09-24},
  url = {https://perplexity.coolkeypoint.com/hub/blog/photon}
}