Run Elasticsearch or OpenSearch for Django in Production

Problem statement

When PostgreSQL full-text search is no longer enough (relevance tuning, typo tolerance, facets across millions of documents, multiple languages), Django apps add Elasticsearch or its open-source fork OpenSearch. The search engine becomes a second datastore that must stay in sync with the database, be secured (exposed clusters are a classic data leak), have enough memory, and be reindexed when mappings change. This guide covers a production setup that avoids those traps.

1. Prefer a managed service (Elastic Cloud, Amazon OpenSearch Service, Aiven) unless you have search operations experience.
2. Use django-elasticsearch-dsl or django-opensearch-dsl to define documents mapped from models.
3. Disable synchronous signal-based indexing in production and index via Celery tasks after commit.
4. Secure the cluster: TLS, authentication, private network only; never expose port 9200 publicly.
5. Use index aliases so you can build a new index and switch atomically when mappings change.
6. Size memory for the JVM heap (about half of RAM, under 32 GB) and monitor cluster health.

Step-by-step solution

1. Choose the engine and hosting

Elasticsearch (Elastic license or AGPL) and OpenSearch (Apache 2.0) share most concepts and query DSL for typical Django use, but their clients and newer features have diverged. Pick one and use its matching client library. A self-hosted single node is fine for development; production clusters need at least three nodes for resilience, careful upgrades, and snapshot backups, which is why managed services are popular.

2. Define documents

# search/documents.py  (django-opensearch-dsl)
from django_opensearch_dsl import Document, fields
from django_opensearch_dsl.registries import registry
from shop.models import Product

@registry.register_document
class ProductDocument(Document):
    category = fields.KeywordField(attr="category.slug")

    class Index:
        name = "products"
        settings = {"number_of_shards": 1, "number_of_replicas": 1}

    class Django:
        model = Product
        fields = ["name", "description", "price", "updated_at"]
        queryset_pagination = 1000

OPENSEARCH_DSL = {
    "default": {
        "hosts": env("SEARCH_URL"),             # https://user:pass@search.internal:9200
        "verify_certs": True,
    }
}
OPENSEARCH_DSL_AUTOSYNC = False

Turning off autosync stops every model save from making a synchronous HTTP call to the search cluster inside the request.

3. Index asynchronously

# search/tasks.py
from celery import shared_task
from search.documents import ProductDocument
from shop.models import Product

@shared_task(autoretry_for=(Exception,), retry_backoff=True, max_retries=5)
def index_product(pk):
    obj = Product.objects.filter(pk=pk).first()
    if obj is None:
        doc = ProductDocument.get(id=pk, ignore=404)   # deleted in the database
        if doc:
            doc.delete()
    else:
        ProductDocument().update(obj)

@receiver(post_save, sender=Product)
def product_saved(sender, instance, **kwargs):
    transaction.on_commit(lambda: index_product.delay(instance.pk))

Handle deletes the same way. Schedule a periodic full reindex during quiet hours to repair drift.

4. Secure and connect privately

• Enable TLS and authentication (built in for current Elasticsearch and OpenSearch security plugins, and on all managed services).
• Keep the cluster on a private network or VPC; allow only app and worker hosts.
• Create a role for the app with access only to its indices.

5. Reindex with aliases

Mapping changes (a new analyzer, a field type change) need a new index. Build products_v2, fill it, then switch the products alias in one atomic call:

POST _aliases
{ "actions": [
  { "remove": { "index": "products_v1", "alias": "products" } },
  { "add": { "index": "products_v2", "alias": "products" } }
]}

The app always queries the alias, so search never goes offline during a reindex.

Verification check: cluster health is green (GET _cluster/health), the document count in the index matches the queryset count, a product saved in the admin appears in search within seconds, and port 9200 is unreachable from the public internet.

Explanation

A search engine maintains inverted indexes optimised for relevance ranking and fuzzy matching, which relational databases do less well at scale. Because it is a separate system, consistency with PostgreSQL is eventual: changes flow from the database to the index. Treating the database as the source of truth, indexing through a queue with retries, and periodically rebuilding makes that eventual consistency dependable. Aliases decouple the name the app queries from the physical index, which is what allows schema changes without downtime.

Edge cases / notes

• Memory. Both engines run on the JVM; set heap to about half the node's RAM, under ~31 GB, leaving the rest for the filesystem cache.
• Exposed clusters are scanned constantly; an unauthenticated public cluster is a data breach waiting to happen.
• Bulk loads. Temporarily set refresh_interval: -1 and replicas to 0 during a full build, then restore.
• Smaller needs. For modest catalogs, PostgreSQL full-text search or Meilisearch may be simpler.

Next steps

• Check whether you need a search cluster at all with PostgreSQL Full-Text Search for Django in Production.
• Run indexing tasks reliably with Configure Redis and Celery for Django in Production.

FAQ

Elasticsearch or OpenSearch for Django?

Both work. OpenSearch is Apache-licensed; Elasticsearch has Elastic's licensing and features. Use the matching Django DSL library.

Should Django index documents on every save?

Not synchronously. Queue an indexing task after commit so requests do not depend on the search cluster.

How do I change a mapping without downtime?

Build a new index, fill it, and switch an alias to it atomically.

Do I need a managed search service?

Not strictly, but running a resilient cluster with backups and upgrades takes real expertise.