> For the complete documentation index, see [llms.txt](https://docs.rhinofcp.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.rhinofcp.com/data-harmonization/semantic-mapping/benchmark-performance.md).
# Benchmark Performance
**Semantic (Terminology) Mapping Benchmark Performance**
Rhino's self-hosted terminology mapping pipeline was benchmarked against frontier hosted LLMs across a 2,312-query biomedical test set spanning nine cross-vocabulary mapping tasks.
Running entirely inside the customer's firewall, the self-hosted pipeline (Rhino OSS) achieved 67.5% top-1 accuracy and 84.9% top-5 accuracy (the correct concept appears among the first five suggestions), at roughly 7.8x the speed of frontier hosted LLMs on the same task.
Against hosted frontier models using the same candidate lists and prompts, Rhino OSS lands within about five points of the strongest configurations tested: Rhino + Claude Opus (72.1% top-1 / 90.0% top-5) and Rhino + GPT (71.3% top-1 / 89.3% top-5). Frontier models performed comparably only when paired with Rhino's retrieval pipeline — on their own, general-purpose LLMs struggled with this task. This gives customers a self-hosted option that closes most of the gap to frontier-assisted mapping without sending terminology data outside the site, at a fraction of the per-query cost.
A follow-on evaluation added a MedGemma-27B classifier reranking stage over embedding-retrieved candidates (4-way fusion plus synonym expansion, k=30), scored across the same nine datasets. Accuracy varies meaningfully by vocabulary pair: strongest on ICD-O-3→SNOMED and OPCS-4→SNOMED (93–95% top-5/top-10), and weakest on ICD-10-CM→SNOMED (63.7% top-5, 68.5% top-10) and Sheba Labs→LOINC (76.4% top-5, 80.3% top-10). Across all nine sets, unweighted average accuracy was 83.4% top-5 and 86.6% top-10.
*Methodology note: both benchmarks score 1:1 mappings only. The first benchmark held candidate lists and prompts identical across all three model configurations; the second reflects classifier-stage accuracy on top of embedding retrieval rather than a standalone LLM comparison. Because accuracy is pair-dependent, results should be cited per-vocabulary-pair rather than as a single blended figure when responding to RFP evaluation criteria.*
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.
Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:
```
GET https://docs.rhinofcp.com/data-harmonization/semantic-mapping/benchmark-performance.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
