Elasticsearch Memory Problems Caused by ElasticSuite Tracking Indices - Fixes for Developers

Tracking indices in ElasticSuite can silently explode in size and heap usage. Symptoms range from frequent GC pauses and node restarts to slow searches and 5xx spikes - issues that Hyvä’s lean frontend won’t mask on cache misses. This guide shows how to diagnose the problem, contain it fast, and implement long‑term fixes without breaking Magento 2 search or Core Web Vitals.

What’s happening under the hood

ElasticSuite can store behavioral and query data in dedicated tracking indices. On busy stores, these indices grow quickly (docs, fields, shards), increasing heap pressure and segment merges. Excessive cardinality fields and oversharding amplify memory use, eventually triggering long GC cycles, circuit breakers, and node instability.

• Typical symptoms: high heap.percent, slow queries, timeouts, frequent indexer/search errors.
• High‑risk scenarios: long retention of tracking data, too many shards/replicas, “ghost” indices left after upgrades, and unbounded attribute/facet growth.

Step 1 - Confirm the diagnosis

Start by mapping heap/memory hot spots to specific indices. Use safe read‑only calls in a maintenance window if possible.

# Node & heap overview
GET /_cat/nodes?v&h=name,ip,heap.current,heap.percent,ram.current,cpu,load_1m

# Largest indices first (watch tracking / analytics patterns)
GET /_cat/indices?v&s=store.size:desc

# Per-index stats (look for huge doc counts / field counts / memory)
GET //_stats

# Shard layout and imbalance
GET /_cat/shards/?v

# Find stale / "ghost" indices matching old patterns
GET /_cat/indices/?v

If tracking indices dominate store size, shard count, or memory, you’ve found your primary culprit. Note index patterns and retention age before any destructive action.

Step 2 - Immediate containment (safe & reversible)

Reduce pressure quickly while you plan a durable fix.

• Throttle writes: temporarily disable or limit tracking collection in admin (where supported) to stop index growth.
• Lower replicas: on tracking indices only, reduce number_of_replicas to 0 during the incident to free heap.
• Rollover & delete oldest: remove obsolete tracking indices first (oldest by date). Keep a short backup if compliance requires.
• Right‑size heap: ensure ES heap ≈ 50% of RAM (avoid >≈31 GB heaps) and -Xms == -Xmx; restart nodes in rotation.

# Example: set replicas to 0 on a tracking index
PUT //_settings
{ "index": { "number_of_replicas": 0 } }

# Delete an obsolete tracking index (double‑check name!)
DELETE /

Tip: Never purge product/search core indices during incident response. Target tracking/analytics indices first.

Step 3 - Remove ghost indices and bad shards

After upgrades or vendor changes, ghost indices can remain, consuming storage and heap. Identify and delete them explicitly, then force a clean reindex from Magento.

# List suspicious indices
GET /_cat/indices/<old_pattern*>?v

# Delete by exact name (irreversible)
DELETE /

# Rebuild application indices
bin/magento indexer:reindex
bin/magento cache:flush
</old_pattern*>

Ensure no services are writing to those indices anymore before deletion. Consider a snapshot if you need historical query analytics.

Step 4 - Long‑term fixes: sizing, shards, retention

Prevent recurrence with disciplined index and lifecycle management tailored to tracking data.

• Retention policy: rotate tracking indices daily/weekly and delete beyond a short retention window (e.g., 7–30 days).
• Shard strategy: avoid oversharding; use few, larger shards (e.g., 1–2 per index for small/medium stores) and set replicas conservatively.
• Mappings: disable field explosion (dynamic mappings), avoid text + keyword on every field, and turn off fielddata where not needed.
• ILM/rollover: move tracking indices to warm storage quickly and delete on schedule; don’t include them in snapshots unless required.
• Attribute discipline: in Magento, limit “Use in Layered Navigation” to truly valuable facets to reduce query and aggregation cost.

These controls keep heap stable even as catalog size and traffic grow seasonally.

Step 5 - Magento & ElasticSuite alignment

Keep application and search tiers in sync to avoid index churn and compatibility issues.

• Version sanity: run an ElasticSuite release that explicitly supports your Magento and Elasticsearch/OpenSearch versions.
• Indexer hygiene: schedule heavy indexers off‑peak; monitor for failures and long rebuilds.
• Template safety for Hyvä: keep search/PLP templates lightweight; never reintroduce legacy assets that inflate render time on cache misses.

Remember: Hyvä improves frontend cost, but it won’t fix backend heap starvation - that’s solved in Elasticsearch and index design.

Operational checklist (copy‑paste for incidents)

• Measure: check /_cat/nodes heap%, top indices by size, shards per node.
• Contain: throttle tracking writes, set replicas=0 on tracking indices, delete oldest tracking indices.
• Clean: remove ghost indices left from past upgrades; reindex; flush caches.
• Stabilize: right‑size heap and JVM flags; restart nodes in rotation.
• Prevent: ILM rotation + retention, shard policy, mappings hygiene, reduced facets.
• Monitor: alerts on heap%, rejected searches, cache hit ratio, and long GC.

Where qoliber modules help

While fixes happen at the search tier, frontend and URL design still influence query load and crawl behavior. qoliber extensions are Hyvä‑compatible and ElasticSuite‑aware, minimizing frontend bloat and helping control URL volume and indexing:

• SEO: Pretty Filter URLs - cleaner layered navigation URLs reduce parameter sprawl.
• SEO: Advanced Sitemaps - exclude low‑value faceted URLs to limit crawler pressure on search.
• SEO: Custom Tags - apply noindex to deep filter combinations to cut unnecessary index load.

Explore our performance‑safe stack here: qoliber Magento 2 extensions.

Conclusion

If Elasticsearch keeps running hot with ElasticSuite, suspect tracking indices first. Diagnose with node/indices stats, contain by throttling and trimming, then enforce ILM and shard discipline so the issue doesn’t return. Keep Magento and ElasticSuite aligned and your Hyvä templates lean and your search tier will stay stable under real‑world traffic.