While implementing Sitecore Search with Search SDK we noticed an important UX issue. When a user enters a new keyword, all previously selected facets are reset.
This is not ideal for a real-world search experience – users expect filters to stay intact.
Use Case A user applies a few filters such as:
Brand: Nike
Category: Running Shoes
Price Range: $50–$100
Then the user types a keyword like:
“Air Zoom”
Expected Keep existing facets and apply the keyword on top.
Actual All facets reset. The keyword search behaves like a fresh search.
Why This Happens The Sitecore Search SDK provides an action:
onKeyphraseChange({ keyphrase: value })
This action is designed to restart the search, which includes clearing:
selected facets
pagination
sorting (depending on configuration)
So using onKeyphraseChange intentionally resets the search state, and this is expected behavior.
✅ Option 2 — Using updateSearchQuery() preserves facets
<button
onClick={() => searchQuery.getRequest().updateSearchQuery(searchFieldValue)}
>
Search with updateSearchQuery
</button>
Confirmation from Sitecore Support
We raised this with Sitecore Support and received the following confirmations:
✔ onKeyphraseChange() does reset facets – by design. ✔ updateSearchQuery() is valid and correct for preserving facets. ✔ Documentation updates are in progress. ✔ A feature request has been opened: SEARCH-5440.
So the behavior and the solution are now officially acknowledged by Sitecore.
The sample widgets provided in the Storybook use the onKeyphraseChange() method by default. These examples are meant to demonstrate baseline search behavior, which includes resetting facets when a new keyword is entered.
However, depending on the requirements of your application, you may need to adjust this behavior. The approach described in this blog highlights an alternative method, updateSearchQuery() – which allows you to update the keyword without losing the currently selected facets.
This gives you more control over the overall search experience and enables a more intuitive UX, especially in scenarios where users first filter results and then refine them with keyword search.
Acknowledgment
Special thanks to Karan Raghwani for his valuable inputs and the deep research he conducted during this implementation. His analysis played a key role in identifying how updateSearchQuery() can be used effectively to preserve facets while updating keyword searches.
In the previous part of this series, we explored how to manually push and delete records in Sitecore Search using the Ingestion API.
That setup laid the foundation – helping developers to learn & understand how authenticated API calls, payload structures, and index endpoints work together behind the scenes.
Now, in this part, let’s connect the dots and explore how we automated this process to ensure Sitecore Search always stays synchronized with the latest content updates from Sitecore XM.
⚙️ The Goal
Our objective was simple yet critical – to automate synchronization between Sitecore CMS and Sitecore Search
“Every time an item is saved, published, or deleted in Sitecore CMS – the corresponding record in Sitecore Search should be automatically updated or removed.”
This guarantees that search results reflect the latest content authors see in Sitecore, ensuring data freshness and consistency across environments.
But beyond synchronization, data accuracy is the top priority. Even a single mismatch between CMS content and Search Index can lead to broken experiences – outdated jobs, blogs, news or missing profiles.
Hence, accuracy and verification became a guiding principle in our approach.
🧩 The Foundation – Events and Pipelines
To achieve full automation, we leveraged Sitecore’s event-driven architecture and publishing pipelines, the two most powerful hooks for reacting to content changes.
Events – Detect and respond to item saves or deletions.
Publishing Pipelines – Detect when items are published to the Web database.
By combining both, we ensured all types of content actions (create, update, delete, publish) are captured in real-time and reflected in Sitecore Search using the Ingestion API.
🗂️ Configuration Setup – Registering Handlers and Processors
We began by defining a configuration patch file that registers custom handlers and custom processors – laying the groundwork for our automation hooks.
Item event handlers
Custom publish processors
🧠 Inside the Custom Publish Processor
The CustomPublishProcessor hooks into the publishing pipeline and inspects the ProcessedPublishingCandidates – the list of items successfully published.
For every item matching our target templates (e.g., Candidate, Job, Blog, News etc.), it calls the Ingestion API to update the corresponding record in Sitecore Search in a specific Source + Entity.
public override void Process(PublishContext context)
{
if (context.Aborted)
return;
var addUpdateItems = context.ProcessedPublishingCandidates.Keys
.Select(i => context.PublishOptions.TargetDatabase.GetItem(i.ItemId))
.Where(j => j != null);
foreach (var item in addUpdateItems)
{
if (item.TemplateID == ID.Parse(SpecificTemplateId) && item.Paths.IsContentItem)
{
Log.Info($"CustomPublishProcessor: Processing {item.ID} - {item.Paths.FullPath}", this);
_ = Task.Run(() => _sitecoreSearchService.IndexItemAsync(item));
}
//Similarly add different template based conditions.
//All your required code to push a specific entity into search will come here.
}
}
💡 Tip: Using Task.Run() ensures indexing happens asynchronously, avoiding delays during content publish operations.
🗑️ Handling Deletions Automatically
When an item is deleted, the OnItemDeleting event handler ensures that the record is also removed from Sitecore Search – keeping the index clean and consistent.
public void OnItemDeleting(object sender, EventArgs args)
{
var item = Event.ExtractParameter(args, 0) as Item;
if (item == null || !item.Paths.IsContentItem)
return;
if (SitecoreSearchHelperHI.IsValidItemForSitecoreSearchIndex(item))
{
if (item.TemplateID == ID.Parse(TalentBoardCandidateTemplateId))
{
_ = Task.Run(() => _sitecoreSearchService.DeleteItemAsync(item.ID.ToString()));
}
//Similarly add different template based conditions.
//All your required code to push a specific entity into search will come here.
}
}
By delegating deletions through the Ingestion API, we guarantee data parity between Sitecore and Sitecore Search — no orphaned records, no stale data.
💬 Why This Approach Works
This design bridges Sitecore’s event-driven architecture with the modern, API-first design of Sitecore Search.
It achieves real-time synchronization without compromising stability by: Encapsulating API logic within dedicated services Leveraging native Sitecore pipelines and events Executing non-blocking background tasks
🏁 Final Thoughts
Implementing the Sitecore Search Ingestion API establishes a direct, code-driven connection between your Sitecore content and the search index. While crawler-based indexing remains effective for many scenarios, the ingestion approach provides enhanced control over what gets indexed and when. By embedding validation and automation within the ingestion process, teams can ensure search results remain consistent, reliable, and always aligned with the most recent content updates.
Continuing our journey of mastering Sitecore Search backend operations – after exploring how to clear all records from a source and map ingestion schemas during migration it’s time to move to the next essential step: pushing / removing data from Sitecore Search using the Ingestion API.
🧭 Understanding the Ingestion API
The Sitecore Search Ingestion API allows developers to create, update, or delete individual documents (records) in a specific source. Unlike legacy crawlers or SDKs, this API gives you programmatic control over how and when content is indexed.
sourceId → The source you’ve configured in the CEC admin portal
entityId → Represents the logical entities that you have configured in CEC admin portal
documentId → Unique identifier for the record
locale → Language/region of the content (e.g., en-us)
🟢 Note: The Ingestion API supports single-record operations only. If you have multiple records to push, you must perform multiple PUT calls – ideally using parallelization or queued batch processing in your own implementation for efficiency.
_sitecoreSearchDomainId → The domain identifier for your Sitecore Search setup. This is the ID of the search domain configured in the Sitecore Search Admin Portal
_defaultLocale → Specifies the language/region of the content, e.g., en-us. This ensures the ingestion request indexes the content correctly according to the locale configuration in Sitecore Search.
sourceId → The source ID you’ve configured in the admin portal where your data is ingested.
entityId → Defines the logical grouping or schema of records within that source.
documentId → The unique identifier for the specific record you’re creating or updating. This typically maps to the Sitecore item’s GUID (e.g., item.ID.ToString()).
💡 Tip: These values can be typically stored in:
Environment variables
Or a Sitecore configuration item if you want it manageable from the CMS itself (so you can change it without redeploying code).
🔗 Helper Method – Send the Request
This method handles the HTTP call and logs both success and error scenarios.
private async Task<bool> SendIngestionApiRequest(HttpMethod method, string requestUri, HttpContent content = null)
{
try
{
using (var httpClient = new HttpClient())
{
httpClient.DefaultRequestHeaders.Add("Authorization", _sitecoreSearchIngestionApiKey);
using (var request = new HttpRequestMessage(method, requestUri))
{
if (content != null)
{
request.Content = content;
}
var response = await httpClient.SendAsync(request);
string responseContent = await response.Content.ReadAsStringAsync();
if (response.IsSuccessStatusCode)
{
// Try to capture incrementalUpdateId
try
{
var jsonResponse = JObject.Parse(responseContent);
var incrementalUpdateId = jsonResponse["incrementalUpdateId"]?.ToString();
if (!string.IsNullOrEmpty(incrementalUpdateId))
{
Log.Info($"Ingestion API Success - IncrementalUpdateId: {incrementalUpdateId}", this);
}
}
catch (Exception parseEx)
{
Log.Warn(
$"Ingestion API Success - Unable to parse incrementalUpdateId. Raw response: {responseContent}",
parseEx,
this
);
}
Log.Info($"Ingestion API Request Successful | Method: {method} | URI: {requestUri}", this);
return true;
}
else
{
Log.Error(
$"Ingestion API Error | URI: {requestUri} | StatusCode: {(int)response.StatusCode} | Response: {responseContent}",
this
);
return false;
}
}
}
}
catch (Exception ex)
{
Log.Error("Error sending request to Sitecore Search Ingestion API.", ex, this);
return false;
}
}
📝Explanation of Key Variables
_sitecoreSearchIngestionApiKey→ API key for authenticating requests. Stored in environment variables (recommended) or Sitecore settings.
HttpMethod method → HTTP method (PUT for create/update, DELETE for removal).
requestUri→ Full ingestion endpoint, typically generated via GetIngestionApiUrl.
content→ JSON-serialized payload representing the document.
IsSuccessStatusCode→ Validates request success.
true → Document successfully created or updated.
false → Error occurred; response content is logged.
incrementalUpdateId → A unique ID returned by the API to track the status of incremental updates. This ID can be stored (e.g., in a database) to monitor the progress of the request later. By using the incrementalUpdateId, you can query the API to check whether the update (create, update, delete) has been processed successfully or if there were any issues.
Logging → Logs successes, API errors, and exceptions for debugging.
💡 Tip: Wrapping HttpClient in a using block ensures proper disposal.
🧩 Constructing the Payload
Each record is represented as a document object containing an id and a set of fields. Field names must match exactly with the ones defined in your Sitecore Search schema (case-sensitive and lowercase).
Here’s an example with multiple field types:
var payload = new
{
document = new
{
id = item.ID.ToString(),
fields = new
{
// Text fields
title = item["Title"],
description = item["Description"],
// Boolean field
isactive = item["IsValid"] == "1" ? true : false,
// Date field
posteddate = DateUtil.IsoDateToDateTime(item["Posted Date"]),
// Array field
//This is just for example, you can write you custom logic to values from Sitecore item(s)
jobcategory = new[] { "Engineering", "Remote" },
// Geolocation field (latitude,longitude)
location = "44.977753,-93.2650108"
}
}
};
var jsonContent = JsonConvert.SerializeObject(payload, Formatting.Indented);
var httpContent = new StringContent(jsonContent, Encoding.UTF8, "application/json");
📦 Indexing a Record to Sitecore Search / Pushing the Record
Now that we have our helper methods ready, let’s see how to use them to index a Sitecore item into Sitecore Search via the Ingestion API.
Below is a generic example that can work for any entity type (e.g., job, product, article, etc.):
/// <summary>
/// Indexes a Sitecore item into Sitecore Search using the Ingestion API.
/// </summary>
public async Task IndexItemAsync(Item item)
{
try
{
// Initialize all Sitecore Search configuration settings
InitializeSettings();
// Build the ingestion endpoint dynamically
string url = GetIngestionApiUrl(_sourceId, _entityId, item.ID.ToString());
// Prepare the JSON payload representing the document
var payload = new
{
document = new
{
id = item.ID.ToString(),
// These fields are just examples.
// Replace them with the actual fields and any custom logic
// required to extract values for your specific Sitecore item.
fields = new
{
title = item["Title"],
description = item["Description"],
isactive = item["IsValid"] == "1" ? true : false,
posteddate = DateUtil.IsoDateToDateTime(item["Posted Date"]),
category = new[] { "General", "Public" },
location = "44.977753,-93.2650108"
}
}
};
// Serialize payload to JSON and prepare request content
var jsonContent = JsonConvert.SerializeObject(payload, Formatting.Indented);
var httpContent = new StringContent(jsonContent, Encoding.UTF8, "application/json");
// Send the request to Sitecore Search
await SendIngestionApiRequest(HttpMethod.Put, url, httpContent);
}
catch (Exception ex)
{
Log.Error($"Error indexing item to Sitecore Search.", ex, this);
}
}
InitializeSettings() → Ensures that all configuration values like _sourceId, _entityId, _sitecoreSearchBaseUrl, and _sitecoreSearchDomainId are properly loaded before making the API call.
GetIngestionApiUrl() → Dynamically builds the ingestion URL for the specific record using the item’s ID.
payload → Represents the document structure expected by Sitecore Search.
id → Unique identifier for the record (mapped from the Sitecore item ID).
fields → The actual searchable/filterable fields defined in your Sitecore Search schema.
title, description, isactive, posteddate, category, location → Example fields showing different data types – boolean, date, array, and geo, respectively. These should match your Sitecore Search schema configuration.
SendIngestionApiRequest() → Uses the helper method defined earlier to send the PUT request to the API.
Error handling → Logs any exception that occurs while preparing or sending the request.
💡 Tip: Ensure that the field names and types match exactly with your Sitecore Search schema field names, which are always stored in lowercase in the admin portal.
🧹 Deleting a Record from Sitecore Search
After successfully pushing records, there might be cases where you need to delete them. The following method handles that cleanup process via the Ingestion API.
/// <summary>
/// Deletes a document from Sitecore Search if it exists.
/// </summary>
public async Task DeleteItemAsync(string documentId)
{
InitializeSettings();
if (string.IsNullOrEmpty(_jobsSourceId) || string.IsNullOrEmpty(_jobsEntityId))
return;
string url = GetIngestionApiUrl(_jobsSourceId, _jobsEntityId, documentId);
await SendIngestionApiRequest(HttpMethod.Delete, url);
}
This ensures that the Sitecore Search index remains in sync with your content lifecycle – maintaining clean, relevant data and preventing stale records from appearing in search results.
✅ Verifying the Record
Once the request is sent, the response is validated using the built-in property IsSuccessStatusCode.
If this check returns true, it means the record was successfully pushed/deleted to Sitecore Search – whether it was newly created or updated. You can confirm the result by navigating to your Sitecore Search Admin Portal. Note: It may take a few seconds/minutes for the data to appear in the content collection after ingestion.
If IsSuccessStatusCode returns false, the error details are automatically logged by our helper method:
string errorContent = await response.Content.ReadAsStringAsync();
Log.Error($"Ingestion API Error: {errorContent}", this);
This ensures that any ingestion issues (like schema mismatches or invalid API keys) are captured in your Sitecore logs for quick debugging.
🧠 Best Practices
Always ensure field names and data types exactly match your schema.
Keep locale consistent with your source configuration.
Use PUT for both insert and update; DELETE for removal.
When ingesting many records, queue requests and send them asynchronously in controlled parallel batches to avoid rate limits.
Log every ingestion attempt with document ID and API response for traceability.
🏁 Wrapping Up
With this, we have completed the foundational part of the ingestion flow – pushing content from your application to Sitecore Search using the Ingestion API.
This post is focused on API-first ingestion flow – helping us to understand and test the process independently of Sitecore CMS.
In the next part of this series, we’ll take it a step further – integrating this ingestion logic directly into Sitecore CMS using custom pipeline processors and event handlers to automatically sync Search with publishing and deletion actions.
That’s where we’ll connect the dots between theory and real-world implementation.
One of the features we had in our old search provider – but is currently missing in Sitecore Search – is the ability to clear all records in a source with a single action. It’s a useful capability that I hope Sitecore will eventually add to the platform.
👉 Note: This approach applies to implementations where records are maintained and operated via the Ingestion (Push) APIs. For sites using the crawler-based setup, the process of clearing and repopulating data would be handled differently by Search itself.
Let’s walk through how we solved this problem using the Ingestion API.
Why would you ever need to clear all records?
There are several real-world scenarios where you might want to wipe out everything from a source before pushing new data:
Upstream data updates Your main data provider has made significant updates, and you need to replicate those changes in your index.
Business logic changes You’ve modified how certain attributes are generated or changed their data types. This requires pushing fresh records so that Search reflects the new logic.
Data mismatches Sometimes what Search shows doesn’t match what it should. A clean delete and re-publish ensures the index is in sync with your business rules.
In short: whenever you want to start with a clean slate in your Sitecore Search source, this feature is extremely handy.
🛠️ Our Approach
Since Sitecore Search doesn’t (yet) provide a “delete all” option from the UI, we built our own mechanism.
The idea is simple:
Fetch all document IDs from a source using the Search Query API.
Iterate and Delete them using the Ingestion API.
Re-publish records.
👉 In our case, the application runs on Sitecore 10.3 XM, so we had the flexibility to add this logic inside the solution. We basically create Command Buttons where we have used the following methods. You can use these methods as it suit you and your requirements. 👉 If we were on XM Cloud, we’d likely implement this as a PowerShell script or a standalone utility.
📌 Step 1: Fetching All Document IDs
The method FetchAllDocumentIdsAsync queries the Sitecore Search Query API in a loop until it retrieves all document IDs from the given source.
🔎 What it does in short:
Paginates results using limit and offset until all items are retrieved.
Builds a request payload with widget, source, entity, and locale details.
Sends requests to the Query API (discover/v2) and parses the response.
Collects all document IDs and ensures uniqueness.
👉 Outcome: A list of all documents currently stored in the source.
/// <summary>
/// Fetches all document IDs from a Sitecore Search source using the Query API.
/// The method:
/// 1. Paginates results with limit/offset until all items are retrieved.
/// 2. Sends requests to the Search API and parses responses.
/// 3. Collects document IDs and ensures uniqueness.
/// Returns a clean list of all unique document IDs from the specified source.
/// </summary>
private async Task<List<string>> FetchAllDocumentIdsAsync(string sourceId, string entityId, string widgetId)
{
const int maxLimit = 100;
var ids = new List<string>();
int offset = 0;
int total = int.MaxValue;
while (offset < total)
{
var payload = new
{
widget = new
{
items = new[]
{
new
{
rfk_id = widgetId,
entity = entityId,
search = new
{
content = new { },
limit = maxLimit,
offset = offset
},
sources = new[] { sourceId }
}
}
},
context = new
{
locale = new { country = "us", language = "en" }
}
};
var jsonContent = JsonConvert.SerializeObject(payload, Formatting.Indented);
var httpContent = new StringContent(jsonContent, Encoding.UTF8, "application/json");
//Send request to Sitecore Search
var body = await SendQueryApiRequest(GetSearchApiUrl(), httpContent);
if (string.IsNullOrEmpty(body)) break;
//Parse response
var json = JObject.Parse(body);
var contentArray = json["widgets"]?[0]?["content"] as JArray;
if (contentArray == null || contentArray.Count == 0) break;
//Collect IDs
ids.AddRange(contentArray.Select(c => (string)c["id"]).Where(s => !string.IsNullOrEmpty(s)));
//Determine total items
if (total == int.MaxValue)
total = (int?)json["widgets"]?[0]?["total_item"] ?? ids.Count;
//Increment offset
offset += contentArray.Count;
if (ids.Count >= total) break;
}
return ids.Distinct().ToList();
}
📌 Step 2: Deleting All Documents
The method DeleteAllDocumentsAsync takes the list of IDs and deletes them using the Ingestion API.
🗑️ What it does in short:
Calls FetchAllDocumentIdsAsync to get all document IDs.
Logs and exits if no documents exist.
Iterates over each ID and:
Builds a delete URL using the Ingestion API.
Sends a delete request.
Logs success or failure.
Logs a final confirmation once all documents are removed.
👉 Outcome: The source is now empty and ready for a fresh push.
/// <summary>
/// It will first fetch all the documents from a particular source and then delete them.
/// </summary>
/// <param name="sourceId">Source Id</param>
/// <param name="entityId">Entity Id</param>
/// <param name="widgetId">Widget Id</param>
/// <returns></returns>
public async Task DeleteAllDocumentsAsync(string sourceId, string entityId, string widgetId)
{
if (string.IsNullOrEmpty(sourceId) || string.IsNullOrEmpty(entityId)) return;
var documentIds = await FetchAllDocumentIdsAsync(sourceId, entityId, widgetId);
if (!documentIds.Any())
{
Log.Info("No documents found to delete for sourceId: " + sourceId + " EntityId: " + entityId + " WidgetId: " + widgetId, this);
return;
}
Log.Info($"Found {documentIds.Count} documents to delete for sourceId: " + sourceId + " EntityId: " + entityId + " WidgetId: " + widgetId, this);
foreach (var docId in documentIds)
{
var deleteUrl = GetIngestionApiUrl(sourceId, entityId, docId);
var success = await SendIngestionApiRequest(HttpMethod.Delete, deleteUrl);
if (success) Log.Info($"Deleted document: {docId}", this);
else Log.Warn($"Failed to delete document: {docId}", this);
}
Log.Info("All documents deleted for sourceId: " + sourceId + " EntityId: " + entityId + " WidgetId: " + widgetId, this);
}
📌 Step 3: Supporting Methods
🔗 GetIngestionApiUrl
Builds the Ingestion API endpoint for deleting documents.
Includes domain ID, source ID, entity ID, and optionally the document ID.
Appends the locale.
Logs the URL for traceability.
👉 Usage: Creates the exact endpoint used for delete requests.
FetchAllDocumentIdsAsync → retrieves all document IDs in batches.
GetIngestionApiUrl → builds delete endpoints for each document.
DeleteAllDocumentsAsync → deletes them one by one and logs results.
Re-publish new records → push fresh data into the clean source.
⚡ Final Thoughts
Until Sitecore adds a native “delete all from source” feature in the dashboard, this custom approach is a reliable way to keep your sources clean when using the Ingestion API.
It ensures you can:
Start fresh whenever data changes upstream,
Re-apply new business rules, and
Keep your Search results consistent.
Note: In upcoming blogs we will also try to implement a batch delete operation.
This pattern has been working well for us, and I believe many teams using Sitecore Search with Ingestion API will find it handy. 💡
This setup is quite standard across Sitecore XM Docker solutions. One folder worth highlighting is the deploy folder.
In a typical development workflow, when you publish code from Visual Studio, your output gets placed in the deploy folder. Docker then mounts this folder into the CM/CD containers, and Sitecore’s Deployment.ps1 script copies the files into the actual website path. This enables fast inner-loop development without rebuilding images.
Issue
When making code changes, everything works fine – until you rename or remove a DLL. For example:
You remove an old DLL version.
Or you rename/update an existing DLL.
After publishing from Visual Studio, the updated DLL is added to the deploy folder. But the old DLL isn’t removed automatically.
As a result, the container ends up with both the old and new DLLs, which can cause strange behavior.
Why This Happens
Visual Studio’s publish process copies and updates files – it doesn’t delete what’s already there.
dotnet publish makes sure new/changed files are written.
Some publish profiles (like MSDeploy) can remove extra files, but most Sitecore Docker setups rely on simple folder publishing, so that option doesn’t apply.
Since deploy is mounted into the container and Sitecore copies it as-is into the IIS site, stale files hang around unless you clear them.
Where Does the deploy Folder Come From?
The deploy folder isn’t a Sitecore artifact. It’s simply a publish target folder configured in your Visual Studio publish profile. In the Sitecore Docker setup, this folder is then mounted into the CM/CD containers via Docker Compose.
Visual Studio publishes your project output into deploy.
Docker mounts deploy into the container.
Sitecore’s Deployment.ps1 copies the contents into the IIS web root.
That’s why keeping the deploy folder clean before publish is so important — whatever is inside gets picked up and pushed into your container.
The Fix: Automate Cleanup in MSBuild
The best solution is to automate cleaning the deploy folder before every publish. You can add a BeforePublish target in your .Website.wpp.targets file like this:
This ensures the deploy folder is wiped and recreated automatically before every publish, so only the correct DLLs are deployed.
Tips
During large refactors with many file renames/deletions, this automated clean step prevents old DLLs from sticking around.
For local development, publishing to deploy is fast and convenient.
For production deployments, it’s better practice to bake your code into a custom Docker image for a clean, repeatable build.
✅ Bottom line: If a DLL rename or deletion doesn’t “take,” the root cause is that old files remain in deploy. Automating the cleanup with an MSBuild target ensures every publish starts fresh, keeping your Sitecore XM containers free of stale artifacts.
In Sitecore Search, an analyzer decides how text is broken down into searchable tokens. It works both when content is indexed and when someone types in a search query. Picking the right analyzer for each attribute can make a noticeable difference in result relevance, exact matches, and autocomplete quality.
Available Analyzers
Multi locale standard (rfk_standard_multi_locale) This is the one you’ll probably use most. It works across multiple languages, removes punctuation and common words, handles synonyms, and makes everything lowercase. It also reduces words to their root form — “results” becomes “result.” A safe default for most attributes.
Standard (rfk_standard) Similar to Multi locale, but only for English. No locale-specific processing, so use it only if your site is completely in English.
Alphanumeric only (rfk_alphanumeric_only_analyzer) Removes everything that’s not a letter or number, then keeps the result as one token. Example: 1235-abhe-3f34s becomes 1235abhe3f34s. Works well for IDs, SKUs, or codes.
Keyword (rfk_keyword) Keeps the whole value exactly as it is, treating it as a single token. Example: “Sitecore Search” stays as “Sitecore Search.” Good for exact phrase matches or filters.
Lowercase (rfk_lowercase) Stores the value as a single lowercase token. Helpful for case-insensitive comparisons or sorting when you don’t want any tokenizing or stemming.
Prefix match (rfk_prefix_match) Generates lowercase prefixes from the text, ignoring punctuation. Example: 978-3-16 turns into 978, 9783, 97831, 978316, and so on. Perfect for matching code prefixes or building type-ahead searches for things like ISBNs.
N-gram based matching (rfk_ngram_analyzer) Splits text into smaller character chunks (n-grams). Useful for languages without spaces, long compound words, and for making autocomplete or suggestions more forgiving.
Partial match (rfk_partial_match) Generates both split and joined versions of a term, all in lowercase, and removes stop-words. Example: “up-to-date” produces up, date, and uptodate. Great for catching both separated and combined versions of a word.
Shingle generator (rfk_shingle_analyzer) Creates sequences of words (word n-grams). Example: “how to improve search” produces how to, to improve, and improve search. Useful for phrase suggestions and proximity-based results.
Standard no stemmer (rfk_no_stemmer_analyzer) Same as Standard, but it doesn’t trim words to their root form. Keeps “improve” as “improve.” A good choice when stemming would cause mismatches, such as with brand names or technical terms.
When you first enable textual relevance for an attribute, Sitecore assigns Multi locale standard by default.
You can add more analyzers to the same attribute and adjust their weights to influence scoring.
Setup Steps
Enable the attribute for textual relevance Go to Administration → Domain Settings → Attributes, select your attribute, check Use for Features: Textual relevance, and publish.
Assign analyzers Go to Feature Configuration → Textual Relevance, select the attribute, add analyzer(s), and set weights.
Reindex content Always reindex after changing analyzers so the updated tokens are available in search.
By setting this up carefully and aligning analyzers with your content structure, you ensure that Sitecore Search interprets your data accurately, improving your query results and search accuracy.
In many projects, technology decisions evolve due to licensing, platform consolidation, or cost alignment. One such shift that we faced is the move from Algolia to Sitecore Search.
This blog focuses on real-world conceptual, API differences, Portal Configurations between the two platforms – especially for teams tasked with migrating search indices, data structures, and ingestion processes. The goal is not to compare features competitively, but to ensure technical readiness and accuracy when transitioning. Please note that this blog does not focus on front-end changes; instead, it covers configuration aspects and the different acronyms used by each platform.
🔁Concept Mapping: Algolia vs Sitecore Search
High-Level Terminology Shift The first adjustment is understanding the terminology and data model shift. Here’s a side-by-side comparison of the concepts:
Concept
Algolia
Sitecore Search
Notes / Migration Tips
Index
Index
Source + Entity
Map each Algolia index to a (Source, Entity) pairing.
Document
Object with objectID
document: { id, fields }
Sitecore requires top-level id and then nested fields. Examples of document formats for each platform are provided later in this blog.
Attributes
Dynamic per object
Schema-driven, case-sensitive
Define all attributes up-front in Sitecore Portal.
Schema
Implicit / flexible
No dynamic attributes. Define types, required flag, and other configuration
No dynamic attributes. Define types, required flag and other configuration
Geolocation
_geoloc reserved
Custom attribute of type GEO
Format must match { lat, lng }; Examples given below
Batch Ingestion
Array of objects
documents array
In Sitecore: { "documents": [ { id, fields } ] }. Examples given below
API Key Header
X-Algolia-API-Key + X-Algolia-Application-Id
Authorization: <raw-api-key>
Sitecore expects raw key in headers.
Case Sensitivity
Not case-sensitive
Source/Entity/Attributes are case-sensitive
Watch for naming mismatches.
Validation
Minimal
Strict: Type checks, Required fields, Valid attributes name.
Empty batches or invalid data will cause rejection.
🔁How Documents Differ
In Algolia, documents are flexible JSON objects with a required objectID. For example:
The id is mandatory and should be unique across the entity.
All fields must match the schema defined in the Sitecore Search UI.
Geolocation fields must be of type GEO and must exactly match the name defined in the schema.
🔁Schema Configuration
Unlike Algolia, which lets you index dynamic attributes, Sitecore Search enforces schema validation which makes sure that what ever we are passing in payload is pre-configured
Every attribute (field) must be defined with:
A name (case-sensitive)
A data type (string, double, integer, boolean, object, timestamp, geo, array of string, array of object, timestamp)
Whether it is required, searchable, filterable, facetable, and sortable (all defined in the schema UI).
Whether it supports faceting or full-text search
Missing or extra fields in your ingestion payload will result in an error, and the entire batch may be rejected.
🔁Common Migration Challenges
Case Sensitivity: Sitecore is case-sensitive across the board (Sources, Entities, Fields). Ensure your naming conventions are consistent.
Geolocation: Algolia supports _geoloc natively, while Sitecore requires you to define a geo field and match the lat/lng format exactly.
Validation Errors: Sitecore’s validation is strict. Even a missing required field or misnamed attribute can cause the full ingestion to fail. Please refer to this blog for more details if you want to learn how to identify errors.
Batch Structure: Ensure payloads follow this structure:
Migrating from Algolia to Sitecore Search introduces structural and schema validation considerations but allows for tight control and centralized schema governance. While Sitecore Search’s strictness may feel limiting initially, it leads to a more consistent and predictable search experience over time.
If you’re moving due to licensing or platform unification, we recommend:
Defining your schema before ingestion (required step).
Validating payloads manually or via script before posting.
Using tools like Postman or C# HttpClient to test and debug ingestion.
Creating a fallback log mechanism for failed ingestions. Please refer to this blog for more details.
Working with ingestion APIs often involves troubleshooting indexing issues, especially when the system doesn’t provide complete error details in the UI. To ensure smooth document ingestion in Sitecore Search, it’s important to know how to dig deeper into API responses and identify the root cause of failures.
While working on indexing documents with Sitecore Search and its Ingestion API, we recently came across a scenario where documents were not indexing as expected, and the “View Details” option in the error log wasn’t providing much insight.
After contacting Sitecore Support, we found an effective way to get detailed error information – by checking the API responses through the Network tab in the browser’s Developer Tools and thus wanted to share it with everyone.
This blog explains the situation we faced, the guidance we received from Sitecore Support, and the steps you can follow to debug indexing errors.
Situation
We are currently working on indexing documents into Sitecore Search using the Ingestion API. All required attributes are configured, and the payload we send in the API request includes these attributes. However, we notice that certain documents are not getting indexed.
When we check the Error Log in Sitecore Search, the message suggests:
“Click on ‘View Details’ to see more.”
But the “View Details” button is not functioning, which makes it difficult to understand the root cause of the issue.
Sitecore Support informed us that this is a known issue (SEARCH-2409). As a workaround, they suggested checking the Network tab to directly view the API error response.
Note: After we reported this issue, the “View Details” button is no longer visible. If you are following along with the screenshots in this blog, you may notice its absence. This is expected as Sitecore has updated the UI to temporarily hide the button while addressing the issue.
Note: The issue with the non-functional “View Details” button in the error log has been registered with Sitecore under the reference number SEARCH-2409. You can check the status of this bug on Sitecore’s public bug reference page. If it’s not listed yet, it may still be under internal review.
How to Check Errors Using the Network Tab
Here’s the quick approach to finding detailed errors when using Sitecore Search:
Open your source in Sitecore Search where you are verifying document ingestion.
Open Developer Tools (press F12 or right-click → Inspect).
Navigate to the Network tab and select the “Fetch/XHR” filter.
Use the search bar to filter requests using the keyword “document.”
Select the relevant request.
In the Response tab, you will find the detailed error message from the API, which may include schema mismatches, invalid field types or something that will guide you to find the root cause of a document not getting index.
mapper_parsing_exception failed to parse field [willingtorelocate] of type [boolean]...
Why This Approach is Helpful
Real-time debugging: View the exact error response from the Sitecore Search API.
Pinpoint failures: Understand why the ingestion API request is failing to index a document.
Identify misconfigurations: Detect schema mismatches or incorrect field types in the payload.
Quick resolution: Access detailed error context and understand the root cause.
Key Takeaways
Use the Network tab with Fetch/XHR filtering to directly inspect API responses.
Validate your payload against the source schema to ensure all required attributes and data types are correct.
Utilize detailed API responses to quickly resolve indexing issues.
If everything appears correct but the issue persists, raise a Sitecore Support ticket after thoroughly verifying the payload and configurations.
Final Thoughts
Debugging ingestion issues or why a document is not indexing in Sitecore Search becomes much easier when you use the browser’s Network tab to examine API responses. This approach provides direct insights into why a document might not be indexing and helps resolve potential misconfigurations quickly.
services.traefik.volumes.0.type must be one of the following: “bind”, “volume”, “tmpfs”, “cluster”, “image” – you’re not alone.
This post dives into what causes this issue, why it appeared in recent versions, and how you can resolve it. It’s especially important and helpful if you’re just starting with Docker or setting up a new XM Cloud or XM setup locally. Note: If you’re not updating Docker and have completed the setup earlier, you likely won’t face this issue.
🐳 The Problem
In many Sitecore Docker setups — especially when using Traefik on Windows — you’ll mount the Docker Engine socket using type: npipe.
In this setup, Traefik runs as a Docker container and acts as the reverse proxy, routing traffic to the appropriate Sitecore services.
This worked perfectly fine up to Docker Compose v2.34.0. But when I and my team mates tried running the same setup on latest Docker Desktop v4.42.1 (which includes Docker Compose v2.35.1), I hit this error:
type: npipe is no longer recognized as a valid volume type.
Of course at first, it seemed like a regression or misconfiguration but digging deeper revealed a layered dependency issue.
🔍 What Changed?
In Docker Compose v2.35.0 and v2.35.1, configurations using type: npipe fail validation. While npipe still works on Windows for named pipe mounts, the validation logic in the underlying compose-go library does not recognize npipeas a valid volume type.
To clarify the confusion, here’s a table mapping Docker Desktop versions with Docker Compose and Compose-Go versions – and whether they support npipe:
Docker Desktop
Docker Compose
Compose-Go
type: npipe Support
v4.40.0
v2.34.x
v2.4.x
✅ Yes
v4.41.0 – v4.42.1
v2.35.x
v2.4.9
❌ No (validation dropped)
Note: Although some earlier documentation claimed v4.41.0 was safe, multiple confirmations — including this comment – confirm that v4.41.0 is also affected.
✅ How to Fix It
Option 1: Downgrade Docker Desktop
Downgrade Docker Desktop to v4.40.0 — the last known version where type: npipe validation worked correctly.
This configuration passes validation and works properly on Windows for mounting the Docker Engine socket. Note: This issue is expected to be fixed in the upcoming version v4.43.0+ according to the latest comments.
🔄 Why Is This Important for Sitecore?
The official XM Cloud containerized repo includes this volume mount as part of the traefik configuration. Anyone doing a fresh setup on a newer Docker Desktop version will face this issue unless they:
Downgrade
Modify the Compose file
It may seem like a small issue, but for anyone setting this up for the first time or getting onboard or updating Docker, it’s important to be aware of.
🙌 Acknowledgments
Special thanks to the open-source contributors of the Docker Compose and Compose-Go repositories for actively maintaining these tools and supporting the community.
Thanks to Garima Thakore for surfacing this issue during her ongoing Sitecore XM setup after updating Docker – which led to this deep dive and solution.
I hope this guide helps you understand the root cause and fix the issue quickly if you encounter it.
When working with Sitecore, the Dictionary domain is a reliable way to store translatable text. Most of us using Sitecore would use Sitecore.Globalization.Translate.TextByDomain() to fetch phrases at runtime.
While this works perfectly in most controller rendering contexts, issues begin to arise when you try to use it in background processes like custom pipelines, processors, or scheduled tasks. Strangely, what happens is — Sitecore.Globalization.Translate.TextByDomain() works fine in normal rendering contexts, but returns null or the key itself when used in background processes.
For this blog, we’ll take the example of the publishing pipeline, as in our case, that’s where we had done our custom code. After digging into Sitecore’s translation APIs and how context behaves during publishing, I discovered two reliable ways to resolve dictionary values without failure.
🧭 In this blog we will have a walk through of the following
Why Translate.TextByDomain() (and Translate.Text()) fail in custom pipelines or few background processes.
What’s different in that case
Two alternate working solutions
How to apply each
🔍 The Problem
In our Sitecore solution, we used: Sitecore.Globalization.Translate.TextByDomain(“Dictionary”, “<<Dictionary Key>>”); This worked well inside renderings or controllers. But when used inside our custom PublishProcessor, the value returned is null or just the key and not the value of it.
Here’s a snippet from our custom publishing pipeline where this was happening:
public override void Process(Sitecore.Publishing.Pipelines.Publish.PublishContext context)
{
if (context.Aborted)
return;
foreach (var processedItem in context.ProcessedPublishingCandidates.Keys
.Select(i => context.PublishOptions.TargetDatabase.GetItem(i.ItemId))
.Where(j => j != null))
{
string year = Sitecore.Globalization.Translate.TextByDomain("Dictionary", "<<Dictionary Key>>"); // Fails here
}
}
💡 Note:This is not the exact code we used, as we had a few calls to Algolia where we were sending the data to be indexed whenever a specific set of items were published. This snippet is just for reference.
🤔 Why This Fails In Pipelines
1. Sitecore.Context.Language is not set during publishing The Translate API depends on Sitecore.Context.Language. If this is missing (which it usually is in a publish pipeline), the lookup fails. 2. Sitecore.Context.Site may also be null If you’re using Translate.Text() instead of TextByDomain(), it depends on the DictionaryDomain set in the current site context. Again, Sitecore.Sites.SiteContext is not initialized during publishing. 3. Dictionary domain cache is not populated The Translate APIs rely on internal domain and phrase caches. These may not be fully loaded or refreshed at the time your publish processor runs. 4. Dictionary item may not yet be available in the web database During publishing, dictionary items themselves might not be published yet or might not have propagated to the target DB (web), especially if running on the same publish cycle.
🧪 Attempts That Didn’t Work
Before landing on the right solutions, I tried several common approaches:
Clearing Sitecore cache using PowerShell or code like Sitecore.Globalization.Translate.ResetCache();
Republishing all dictionary items
Reordering publish processors
Run a Full Index Rebuild
None of these reliably fixed the issue – especially when working in the real-time publish pipeline. That’s when I realized the issue wasn’t a bug or timing problem – it was a missing context problem.
✅ Working Solution 1: Use Direct Database Access
The first approach is to bypass the Translate API and directly fetch the dictionary item from the content tree in the target database and language.
using Sitecore.Configuration;
using Sitecore.Data;
using Sitecore.Data.Items;
using Sitecore.Globalization;
namespace Demo.Foundation.Utility
{
public static class DictionaryHelper
{
public static string GetPhrase(string key, string language = "en", string databaseName = "web")
{
if (string.IsNullOrWhiteSpace(key))
return "[MissingKey]";
Sitecore.Data.Database db = Sitecore.Configuration.Factory.GetDatabase(databaseName);
if (db == null)
return $"[InvalidDb:{databaseName}]";
Sitecore.Globalization.Language lang = Sitecore.Globalization.Language.Parse(language);
Item dictItem = db.GetItem($"/sitecore/content/Dictionary/{key}", lang);
return dictItem?["Phrase"] ?? $"[{key}]";
}
}
}
✅ Usage
DictionaryHelper.GetPhrase("<<Dictionary Key>>");
💡 Note: You can also create Item ID constants, and instead of fetching items by path, fetch them by ID — especially useful if you have multiple folders under your main Dictionary folder.
✅ Working Solution 2: Use Context Switchers with Translate API
Anyhow if you still want to use Sitecore’s Translate.Text() or Translate.TextByDomain(), you can – but in that case you will need to explicitly set the site context and language as and when needed.
🌐Language Switcher
using (new Sitecore.Globalization.LanguageSwitcher("en"))
{
string text = Sitecore.Globalization.Translate.TextByDomain("Dictionary", "<<Dictionary Key>>");
}
🏷️ Site Context Switcher
Sitecore.Sites.SiteContext site = Sitecore.Sites.SiteContextFactory.GetSite("<<WebSite Name>>");
using (new Sitecore.Sites.SiteContextSwitcher(site))
{
string text = Sitecore.Globalization.Translate.Text("<<Dictionary Key>>");
}
🔁 Combine Both
Sitecore.Sites.SiteContext site = Sitecore.Sites.SiteContextFactory.GetSiteContext("<<WebSite Name>>");
using (new Sitecore.Sites.SiteContextSwitcher(site))
using (new Sitecore.Globalization.LanguageSwitcher("en"))
{
string text = Sitecore.Globalization.Translate.TextByDomain("Dictionary", "<<Dictionary Key>>");
}
With these in place, the translation works exactly as expected, even inside in your custom processors or backend services.
Thoughts
The Sitecore Dictionary system is very usefull – but it’s also context-dependent. During controller rendering, the context is available OOTB, but in publishing processors, background services, or scheduled jobs, you may need to manually supply that context.
You have two reliable solutions to choose from:
✅ Use Direct Database Access
Ideal when you want full control over how dictionary values are retrieved
No dependency on runtime context (Sitecore.Context.Site, Sitecore.Context.Language, etc.)
No risk of translation failure due to missing site or language context
⚠️ Be cautious when relying on item paths or IDs – if a dictionary item is moved or recreated, its path or ID will change, which can break lookups.
✅ Use Context Switcher
Keeps you within Sitecore’s built-in translation framework.
Enables the use of Translate.Text() and Translate.TextByDomain() even in non-rendering controller contexts.
It is useful when you want to preserve the same translation behavior seen in normal page controller renderings. This can be applied even in your background services.
Thanks for reading! If you’ve faced a similar issue or have your own findings or workarounds – let’s connect and exchange ideas!
Code Snipets 