Batch API

Current status: no dedicated batch endpoint yet

A dedicated batch-processing API comparable to OpenAI's POST /v1/batches (upload a JSONL job file, process asynchronously, settle at a discounted rate) is not yet available on SDKMAX. If your code depends on /v1/batches, /v1/files, or similar, don't build a production dependency on it yet — this page will be updated once the capability ships.

WARNING

To be explicit: don't assume the Batch API is available until it's officially announced.

Interim approach

Until a dedicated Batch API ships, for bulk workloads we recommend:

1. Client-side bounded concurrency

Issue many calls to a normal synchronous endpoint (like /v1/chat/completions) with bounded concurrency instead of firing everything at once:

import asyncio
from openai import AsyncOpenAI

client = AsyncOpenAI(api_key="sk-your-key", base_url="https://api.sdkmax.com/v1")
semaphore = asyncio.Semaphore(5)  # cap concurrency to avoid rate limits

async def call_one(prompt: str):
    async with semaphore:
        resp = await client.chat.completions.create(
            model="gpt-4o",
            messages=[{"role": "user", "content": prompt}],
        )
        return resp.choices[0].message.content

async def run_batch(prompts: list[str]):
    return await asyncio.gather(*[call_one(p) for p in prompts])

2. Exponential backoff

On 429 (rate limit/quota) or 5xx, retry with exponential backoff so a handful of failures don't take down the whole batch:

import time

def call_with_retry(fn, max_retries=5):
    for attempt in range(max_retries):
        try:
            return fn()
        except RateLimitError:
            time.sleep(2 ** attempt)
    raise RuntimeError("retries exhausted")

3. A job queue with persistence

For large or long-running batches, put a real task queue in front of it (Celery / BullMQ / a homegrown jobs table) — call SDKMAX per item and persist results, rather than relying on one long-lived process to push everything through synchronously.

Roadmap

A Batch API is on the roadmap: file-based job submission, async processing, and result retrieval. Watch FAQ or console announcements for updates.