At nine in the morning, the operations dashboard says last night's Gemini batch has finished. The product database tells a different story. Some records have new labels, some still show yesterday's values, and nobody can explain whether the remaining items failed or were never imported.

The model request is only part of an asynchronous API workflow. A completed batch still needs to become an accountable set of business results. For teams accessing Gemini through an API relay, this includes another practical question: does the selected route expose the job and result operations the workflow needs, or only ordinary generation?

This article is an operations runbook for text-processing batches: product categorization, document tagging, and offline evaluation. It follows one hypothetical overnight job from preparation to handover. The goal is a result ledger that survives worker restarts and explains every submitted item, without pretending that a successful job status proves a successful business import.

Before the shift: decide whether batch fits the deadline

A batch workflow is suitable when items can wait and do not depend on one another's immediate answers. A catalog-enrichment job can often run in the background. An interactive support conversation usually cannot wait for an asynchronous processing window.

Google describes batch processing as an asynchronous option in its Gemini inference optimization guide. Treat the provider's documented timing and pricing as information about that service. A relay's enabled operations, billing, and delivery terms still need separate confirmation.

Write the business deadline before writing the submission script. “Imported by the merchandising review” is more useful than “the batch should finish quickly.” Allow time for collecting output, validating it, investigating exceptions, and handing unresolved items to a person.

If a delayed result would block a customer action, keep that action on a suitable interactive route. Batch should remove unnecessary waiting from the application architecture, not move an urgent interaction into an invisible queue.

This runbook concerns generated text and extracted records. Embedding workloads produce a different output contract; the Gemini embedding migration guide covers vector-related requirements. Keep those result validators separate instead of treating every Gemini batch as interchangeable.

18:00 — create the manifest your morning team will need

Before submitting anything, create a manifest containing one immutable identifier per business item. Include the input revision, prompt version, intended model, output contract, and the internal owner of the job.

Suppose the task categorizes product descriptions. The item identifier should refer to the specific product revision being processed, not merely its position in a spreadsheet. If a product description changes overnight, you need to know whether the returned category applies to the old or new version.

Save a digest of the serialized request specification when your data policy allows it. A digest can help detect accidental changes, but it is not a substitute for access control, and it may reveal information about predictable inputs. Keep confidential source material in its approved store rather than scattering it through debug logs.

The manifest is your application record. It remains valuable even if you change SDKs, gateways, or storage formats. It tells the next operator what was intended, independently of what the provider eventually returns.

For document-based source material, the Gemini PDF table extraction article covers content preparation and validation. Keep those domain checks separate from the batch controller so a processing-format change does not quietly alter the meaning of a product record.

18:10 — verify the whole Gemini route

A working Gemini chat request is insufficient evidence for batch support. The workflow may require submission, status lookup, output retrieval, and possibly file handling. Each operation must work through the selected access path and credential scope.

Start with the existing Gemini endpoint selection guide. Then check the actual batch operations against Google's Batch API reference. That reference describes job resources and polling; an OpenAI-compatible chat wrapper does not automatically provide those resources.

For a Treerouter pilot, confirm the supported batch interface before planning a production migration. If the selected route does not provide the required job lifecycle, keep this workflow on a supported direct-provider path until an appropriate relay route is available. A transparent limitation is more useful than a demonstration that only proves synchronous text generation.

Also record where the job was created. A job identifier returned by one service is not a portable instruction to query any other provider account. Preserve the endpoint family and credential reference alongside the identifier; store the credential reference, not the secret itself.

18:20 — submit once and save the returned identity

On successful submission, immediately store the exact resource identifier returned by the service. Link it to your manifest and record submission time. Do not reconstruct the identifier from a display name or assume that two jobs with the same label are the same job.

A worker can fail after the provider accepts the job but before your database stores its identity. Design for this uncertainty. Use a local submission record and single-worker ownership, then follow the route's documented recovery options if the outcome becomes unknown.

Do not automatically submit a second full batch merely because your HTTP client timed out. You may create duplicate work. Your recovery procedure should first attempt to determine whether the original submission exists, using supported service operations and your recorded metadata.

This is also a good point to record the model identifier exactly as requested. A general label such as “Gemini production” is useful to people, but insufficient for reproducing an unexpected result after the route's configuration changes.

During the night — poll the job, not every business item

Let a scheduled worker inspect the job state at a controlled interval. Persist the last observation and the next intended check. A process restart should resume observation of the same job rather than create a new batch.

Use the field names and state values documented for your chosen interface. SDK objects and REST responses can present job information differently; do not compare an SDK enum to a guessed REST string and silently keep polling forever.

The official Gemini rate-limit documentation treats batch limits separately from ordinary calls. Build scheduling around the current limits for your project and model, and avoid a tight polling loop that adds traffic without making the underlying work complete sooner.

Define an application deadline independently from the provider's terminal state. If the deadline passes while the job is still running, mark it late and notify the responsible workflow. “Late,” “failed,” and “cancelled” should remain distinguishable to the morning team.

06:00 — collect output before declaring the job imported

When the service reports a terminal outcome, inspect the documented output representation. Retrieve the result through the supported operation and save a controlled copy before transforming it into application records.

Maintain separate milestones for provider completion, output collection, validation, and database import. If output retrieval fails after the provider finishes, the correct state is “result collection pending,” not “everything succeeded” and not “resubmit the batch.”

Where your route uses files, check the Gemini Files API documentation for applicable behavior and resource lifetime. Do not use a temporary provider resource as your sole long-term record. The operational rule is to collect required output promptly and retain it according to your own approved policy.

For large result sets, write into a staging area first. Record the artifact size, a checksum, and the collector version. A partially downloaded file should not become a partially imported production dataset merely because the first few lines parsed correctly.

06:15 — reconcile identities before accepting answers

Normalize the provider's output into your own small result shape. Preserve the original response in protected storage, then map the documented item key or correlation metadata to the manifest identifier. Do not rely on output row position as your business identity.

The following Python example runs locally. Its rows are an application-defined format, not the Gemini wire response. Your adapter must obtain item_id from the route's documented correlation mechanism and place the validated extracted object in result.

def reconcile(expected_ids, rows):
    expected = set(expected_ids)
    if len(expected) != len(expected_ids):
        raise ValueError("Manifest contains duplicate identifiers")

    seen = set()
    accepted, rejected = {}, {}
    allowed_categories = {"books", "electronics", "home"}

    for row in rows:
        item_id = row["item_id"]
        if item_id not in expected or item_id in seen:
            raise ValueError("Unknown or repeated result identifier")
        seen.add(item_id)

        if row.get("error") is not None:
            rejected[item_id] = "provider_error"
            continue
        result = row.get("result")
        if not isinstance(result, dict):
            rejected[item_id] = "invalid_result"
            continue
        if result.get("category") not in allowed_categories:
            rejected[item_id] = "invalid_category"
            continue
        accepted[item_id] = result

    return {
        "accepted": accepted,
        "rejected": rejected,
        "missing": sorted(expected - seen),
    }


report = reconcile(["p1-r3", "p2-r8", "p3-r1"], [
    {"item_id": "p2-r8", "result": {"category": "home"}},
    {"item_id": "p1-r3", "error": {"code": "example_error"}},
])
assert list(report["accepted"]) == ["p2-r8"]
assert report["missing"] == ["p3-r1"]

Notice that the example accepts an out-of-order result, rejects a failed item, and reports an absent item separately. These are different outcomes. A missing item should trigger investigation of collection and correlation before you assume the model failed.

In a real catalog application, add semantic checks beyond the small category whitelist shown here. A correctly shaped object can still categorize a product incorrectly. Google's structured-output guidance is useful for defining output shape; business acceptance still belongs to your application.

06:30 — import with a revision check

Use a unique database key that includes the job and item identity. Replaying the importer for the same collected artifact should not create a second business record. If an existing record has different content, surface the conflict rather than silently overwriting it.

Before applying an accepted result, compare the manifest's input revision with the current source revision. If the product changed, hold the result for a deliberate policy: discard, review, or regenerate against the new source. Do not let yesterday's input silently overwrite today's corrected description.

Commit imports in controlled transactions and record their outcomes. A database error after model generation is an import failure, not a model failure. This distinction lets you rerun the importer without paying for another inference job whose answers already exist.

The Gemini production engineering checklist provides broader deployment context. The specific contract here is narrower: collected results should be traceable to the source revision and safe to import again.

08:30 — hand over an exception ledger

The morning report should account for every manifest item. Show how many were imported, held for business review, rejected by validation, returned with provider errors, or remain unexplained. Include the collection and import states so operators can see where action is needed.

Do not reduce every exception to “empty answer.” If an individual Gemini response needs safety or termination analysis, the empty-response and safety-block guide addresses that separate investigation. Keep the original reason rather than replacing it with a generic batch failure.

Assign an owner and next action to each exception category. A source revision conflict belongs to a different workflow from a temporary output-download failure. Clear ownership prevents the entire batch from being resubmitted as a convenient but expensive way to make the dashboard green.

Measure delivery by accepted business results. For budgeting, divide the applicable job cost by the number of accepted results and label the time window and cost basis. This is an internal operating metric, not a universal vendor price comparison.

What to bring to an API gateway evaluation

Bring a small manifest containing ordinary inputs, an intentionally invalid business result fixture, and a changed-source scenario. Ask the service to demonstrate submission, status retrieval, result correlation, and output collection on the interface your application will use.

For Treerouter, confirm current Gemini batch support and account conditions first. If available, register and run a small end-to-end pilot that includes importing the results twice. The second import should prove your local replay handling, not trigger another model job.

Your acceptance evidence is the complete chain: one recorded submission, one durable job identity, a collected artifact, an explanation for every item, and a controlled database update. That is what turns Gemini batch access into a useful enterprise API service. A green completion label is only one milestone along the way.