Cloud Platform
Indexing documents on Acquia Search powered by SearchStax — limits and queueing
Document extraction on Acquia Search powered by SearchStax requires:
searchstax Drupal module 1.8.0 or later. This release added direct integration with SearchStax PDF Extractor Service. For details, visit searchstax 1.8.0.search_api_attachments module installed and enabled. For details, visit Search API attachments.Document extraction is not synchronous with page requests. It runs as part of the Drupal indexing pipeline. The pipeline is queue-based. To queue and process extraction:
Save content. When a Drupal entity saves, or hook_entity_update fires, the Search API tracker marks the entity as dirty in the search_api_item table.
Trigger cron. On the next cron run, or when drush search-api:index runs, Search API pulls a batch of dirty items from the tracker. The drush queue:process search_api_attachments command runs the specific queue from the database.
Preprocess items. For each item, Search API runs the configured processors. When search_api_attachments is enabled on the index, the Filecontent processor iterates over the file fields on the item.
Request extraction. For each attached file, the searchstax module later than 1.8.0 sends the file to SearchStax PDF Extractor Service over HTTPS through the Discovery API key. The extractor is Solr Apache Tika. For supported formats, visit Indexing with Solr Cell and Apache Tika.
Substitute text. The extracted text is injected into the Search API item as an additional fulltext field on the indexed document.
Index Solr. Search API POSTs the enriched document to the /update endpoint of Solr on the SearchStax collection.
Because extraction runs inside the indexing pipeline and the pipeline is driven by cron and the Search API queue, all normal Search API tuning applies. This includes batch size, cron frequency, and drush search-api:index --batch-size=N for one-off backfills.
If an extraction request fails due to a network error, 429 rate limit, or extractor timeout, the item remains in the Search API tracker and the system retries it on the next indexing run.
To verify against your site, navigate to Configuration > Search API > your index > Track in the Drupal administrator account. Items awaiting extraction appear as not indexed. Run drush search-api:status <index-id> to view the queue depth.
SearchStax PDF Extractor Service returns at most 100 KB of extracted text per file. The platform silently truncates text beyond that byte offset. The extractor does not error, and Drupal cannot detect truncation from the response.
Practical consequences include:
Long PDFs such as reports, ebooks, and transcripts are only searchable on their first 100 KB of extracted text. After the extracted stream passes that offset, the rest of the document contributes nothing to relevance and cannot be highlighted.
The truncation applies to the extracted text stream, not the source file size. A 20 MB image-heavy PDF might extract to under 100 KB and remain fully searchable. A text-dense 2 MB PDF might hit the cap on page 30 and lose everything afterward.
This limit is set on SearchStax. Acquia cannot raise the limit, and you cannot adjust it from Drupal.
If you need the full text of long documents indexed, you must choose one of the following methods:
Pre-split large documents into smaller files on the client side, such as chapters or sections, before you upload to ensure that each fragment stays under the extractor cap.
Extract the full text through a client-side pipeline, such as Tika, textract, or a Large Language Model extractor. Store the result in a plain fulltext field on the Drupal entity to bypass the SearchStax extractor.
Request an increase through the Acquia product team to SearchStax. This is not guaranteed.
The PDF Extractor Service enforces 120 extraction requests per minute. This is tracked by the X-Forwarded-For header value that Acquia sends to SearchStax.
When exceeded, the extractor returns HTTP 429. The search_api_attachments module treats this as a transient failure. The item remains dirty, and the system retries it on the next cron.
On Acquia Cloud Next, the requests that originate from applications use the IP of a Kubernetes node. The node might host pods from more than one customer. A well-behaved customer might hit 429s because a co-tenant is also extracting.
For details on API errors, visit 429, 401, and Other API Errors.
The search_api_attachments module exposes several settings under Configuration > Search API > Attachments. These settings affect what is sent to the extractor and how much is stored. Confirm current defaults on your installation. The typical settings include:
excluded_extensions: A comma-separated list of extensions to skip. For example, png, gif, jpg, jpeg, mp3, and mp4. These are never sent to the extractor.
excluded_private: If true, skips files in the Drupal private file scheme. This is on by default in some releases.
max_filesize: A hard file-size ceiling before the extractor call. Files above this size are skipped entirely on the Drupal side. A value such as 10 MB is a useful guardrail against a 500 MB PDF that ties up the queue and fails extraction.
number_indexed: A Drupal-side byte cap on the extracted-text-per-file that is injected into the Solr document. This is a second, Drupal-side ceiling independent of the SearchStax 100 KB cap. Typical defaults are 0, which means no Drupal-side cap. Whatever the extractor returns goes in. Set to a lower value if you specifically want to store less than the extractor returns.
The two size ceilings compound. The effective per-file text cap is the minimum of SearchStax 100 KB and search_api_attachments number_indexed.
The most common cause is one of three configuration gaps. Verify each in order:
The searchstax Drupal module is older than 1.8.0. Upgrade first. The PDF Extractor integration does not exist below 1.8.0.
The PDF Extractor endpoint and Discovery API key are not populated in the module settings. Obtain them from SearchStax Studio > App Settings > All APIs > PDF Extractor and paste them into the module configuration.
The search_api_attachments Attachments processor is not enabled on the Search API index. Enable it under the Processors tab of the index and re-index.
The SearchStax 100 KB per-file extraction cap is hit. You must split the document upstream or pre-extract with a client-side pipeline that bypasses the SearchStax extractor. For details, visit the Per-file extraction size cap section previously mentioned.
The extractor uses Solr Apache Tika. Anything Tika does not handle, such as some proprietary CAD formats and custom binary formats, returns empty extraction silently. To confirm format support against the Solr Tika, visit Indexing with Solr Cell and Apache Tika.
Cloud Platform