Overview
Manticore Search is an open-source full-text search engine. This template deploys a replicated Manticore cluster using Galera replication, together with an orchestrator that performs coordinated, zero-downtime CSV imports from an S3 bucket into every replica, optional scheduled backups to S3, and a web dashboard for cluster operations. It is built for datasets that are regenerated upstream and republished as CSV: data arrives in a bucket, and the orchestrator turns it into a distributed table that keeps serving searches while it is reloaded.What Gets Created
Prerequisites
Two things must exist before you install: the agent-token secret and the S3 source bucket with a cloud account that can read it. Only AWS S3 is supported.Create the agent-token secret
orchestrator.agent.tokenSecretName to this name.Create the source bucket and a read-only IAM policy
imports/addresses.csv). Imports only ever read from this bucket — every scratch artifact is written to the shared volume — so a read-only policy is sufficient. Create an IAM policy with the following JSON, replacing my-manticore-bucket:buckets.sourceBucket to the bucket name and buckets.awsRegion to its region.Create a cloud account and reference the policy
buckets.cloudAccountName to the cloud account name and buckets.awsPolicyRefs to the policy name. Custom policies are named without a prefix; only AWS-managed policies take the aws:: prefix (for example aws::AmazonS3ReadOnlyAccess).Installation
Install from the marketplace registry into your existing GVC, naming the agent-token secret and the source-bucket resources you created above:tables value defines one example table, addresses, read from imports/addresses.csv. Define your own tables in a values file (see Tables) and pass it with -f as described in the CLI installation guide.
UI
CLI
Terraform
Pulumi
Configuration
Buckets
Tables
One entry per searchable table.csvPath is a path within buckets.sourceBucket, or a list of paths for a multi-segment table.
schema.columns; otherwise an ID is generated.
csvPath to a list and segmentCount to its length. The chart refuses to render if they disagree. A backup of a multi-segment table covers all segments as one archive, and a restore replaces all of them.
clusterMain: true have not been verified at the shipped resource sizes. Treat them as untested rather than known-good, and validate per-replica row counts after an import if you rely on either.Manticore
Orchestrator
The orchestrator block configures the cron job, the REST API, the agent sidecar, the web UI and the optional backup job.Domain
/api/* to the orchestrator API (bearer token required) and everything else to the UI (no authentication — see the warning above).
Load Test
Connecting
orchestrator.agent.tokenSecretName:
manticore container ships the mysql client, and searchd listens on loopback, so this works from any replica (substitute your manticore.clusterName for manticore in the status key if you changed it):
primary. To use a MySQL client on your own machine instead, forward the SQL port and connect to 127.0.0.1:9306:
First Run
An install indexes nothing: the tables you defined exist but are empty until the cluster has been initialized once and an import has run. Both are orchestrator actions. Run them from the UI (reachable from inside the GVC on port 3000), or from the CLI by pointing the cron job at the action and starting it — see Running Cluster Actions. Runinit once, then import; repeat import whenever the CSVs in your source bucket change.
Operations
Running Cluster Actions
TheRELEASE_NAME-orchestrator-job cron workload ships suspended. orchestrator.action selects what a run does — init bootstraps the Galera cluster, import loads the CSVs for orchestrator.tableName into a fresh slot and swaps it in without interrupting searches, health reports cluster state, and repair recovers from a split brain. Trigger a run on demand:
orchestrator.suspend: false; orchestrator.schedule then applies.
Backing Up
Backups are logicaldelta (incremental) and main (full) archives written to a separate S3 bucket by the RELEASE_NAME-manticore-backup cron workload, under its own identity. Do not widen the source-bucket policy; give backups their own bucket and policy.
Create the backup bucket and a writable policy
my-manticore-backup-bucket. GetObject and DeleteObject let restores read an archive back and lifecycle cleanup remove one; a backup-only install can drop them.Attach it to a cloud account and enable backups
orchestrator.backup.enabled: true together with orchestrator.backup.cloudAccountName, s3Bucket, s3Region and s3Policy.orchestrator.backup.schedules drives the automated runs (daily delta and monthly main by default). You can also run a backup on demand from the UI, or from any workload inside the GVC against the orchestrator API:
Restoring a Backup
Restore is driven through the UI or the orchestrator API; there is noorchestrator.action: restore. List the available archives (type defaults to delta, so pass type=main to see full backups), then restore one by filename:
main backup, use the UI’s Rotate Main control to swap the active slot.
Upgrading from 2.0.1 or Earlier
2.0.1 and earlier shipped a live working bearer token as a values default, published in the template repository, so every install that did not override it shared one publicly-known credential. The management UI was also public by default while having no login of its own, and the chart’s secret policy grantedreveal on every secret in the organization rather than the four it needed. Treat the token on any earlier install as compromised: rotate it rather than simply upgrading.
Create the agent-token secret
Replace the removed keys
orchestrator.agent.token and orchestrator.ui.allowExternalAccess from your values, then set orchestrator.agent.tokenSecretName. If you were relying on external UI access, decide deliberately whether to set orchestrator.ui.publicAccess.enabled: true — it publishes cluster admin.Narrow the source-bucket policy
s3:PutObject or s3:DeleteObject on your source data bucket, reduce it to the read-only policy shown in Prerequisites.Verify backups if you use them
Upgrading from 2.1.0
The agent sidecar’s resource block exposes both a reservation and a limit, so 2.2.0 names the limitmaxCpu / maxMemory instead of cpu / memory. Rename the two keys under orchestrator.agent.resources in your values:
minCpu and minMemory are unchanged, and blocks that expose only a limit (manticore.resources, orchestrator.resources, orchestrator.api.resources, orchestrator.ui.resources, orchestrator.backup.resources, loadTest.resources) keep the bare cpu / memory names. An upgrade that still carries the old names is refused at render:
Scaling and Availability
manticore.autoscaling.minScaleis both the minimum replica count and the replica count the orchestrator coordinates imports across. Changing it after the cluster is initialized requires arepairrun.- Every
helm upgrade— including one that changes nothing — restarts the Manticore replicas one at a time (scalingPolicy: OrderedReady). Indexed data survives, and searches keep serving from the replicas that are still up, but budget for the full serial rollout. Editingtableschanges a mounted secret and therefore triggers the same rollout. manticore.firewall.internalAccess.typemust staysame-gvc: Galera replication between replicas depends on it, and Manticore’s SQL and HTTP ports have no authentication of their own, so they rely on that firewall. Allow a few minutes for any access change to take effect.
Load Testing
WithloadTest.enabled: true, trigger a run by starting the controller; it scales the k6 workload up to loadTest.replicas, waits duration plus testDurationBuffer, then scales it back to zero:
http_req_duration, http_req_failed, a pass/fail mark per threshold) followed by k6 exited with status N — non-zero means a threshold was breached. Set loadTest.controller.schedule to a cron expression to run tests automatically.
Troubleshooting
Deployment never becomes ready after install and cpln logs returns nothing
Deployment never becomes ready after install and cpln logs returns nothing
orchestrator.agent.tokenSecretName did not exist at install time. The containers never start, so there is no log output; the only place the missing secret is named is status.versions[].message.Fix: read the message, create the secret, and the deployment recovers on its own (or force a redeployment to skip the wait):orchestrator.agent.token was REMOVED in manticore 2.1.0
orchestrator.agent.token was REMOVED in manticore 2.1.0
orchestrator.ui.allowExternalAccess was REMOVED in manticore 2.1.0.Fix: follow Upgrading from 2.0.1 or Earlier.orchestrator.agent.resources.cpu was RENAMED to orchestrator.agent.resources.maxCpu
orchestrator.agent.resources.cpu was RENAMED to orchestrator.agent.resources.maxCpu
orchestrator.agent.resources.memory was RENAMED to orchestrator.agent.resources.maxMemory.Fix: rename the two keys as shown in Upgrading from 2.1.0.csvPath has N entries but segmentCount is M — they must match
csvPath has N entries but segmentCount is M — they must match
config.segmentCount does not equal the number of csvPath entries (or csvPath is a single string while segmentCount is not 1).Fix: set segmentCount to the number of CSV paths for that table.Searches return no results
Searches return no results
init once and then import as described in Running Cluster Actions, then confirm with SELECT COUNT(*) on the table.Row counts differ between replicas after a restore
Row counts differ between replicas after a restore
synced.Fix: count rows on each replica as shown in Restoring a Backup; if they disagree, run a fresh import of the table, which rebuilds every replica from the source CSVs.Important Notes
- Create the agent-token secret before installing; without it the deployment wedges silently and
status.versions[].messageis the only diagnostic. - Enabling
orchestrator.ui.publicAccessor routing a domain to the UI puts an unauthenticated admin console on the public internet. Only do it behind your own authenticating proxy. - Keep
manticore.firewall.internalAccess.typeatsame-gvc; Galera replication requires it and Manticore’s 9306/9308 ports have no authentication. - Tables are empty until you run
initfollowed byimport; the install alone does not load data. - The source bucket only ever needs read access. Only the optional backup bucket needs write, and it has its own identity and policy.
- Every
helm upgraderestarts all Manticore replicas one at a time; data survives, but plan the rollout window. - Verify per-replica row counts after a restore rather than trusting cluster status.
- Uninstalling deletes both volume sets and all indexed data. Take a backup first if you need it.