HocTapAZ Docs

Appearance

ai-service Agent Tasks ​

Agent workflow: follow README.md for Audit -> Investigate -> Code -> Test -> Fix; keep this pack's writable scope and verification commands authoritative.

Dispatch type: per-service

Dispatch ID: SVC-015

Current implementation: services/ai-classifier-service.

Owns: AI provider adapters, AI jobs, model usage, prompt/result logs, generation/explanation/grading AI result envelopes.

Must read:

  • docs/agents/handoff-phase-6-ai-classifier-*.md
  • legacy AI provider/settings and usage files: /Users/velikho/Desktop/WORKING/HOCTAPAZ/node-platform/apps/api/src/modules/admin/ai-settings.service.ts, /Users/velikho/Desktop/WORKING/HOCTAPAZ/node-platform/apps/api/src/modules/admin/ai-usage.service.ts, /Users/velikho/Desktop/WORKING/HOCTAPAZ/node-platform/apps/api/src/modules/questions/question-classification.service.ts, /Users/velikho/Desktop/WORKING/HOCTAPAZ/node-platform/apps/api/src/modules/questions/question-classification.queue.ts, /Users/velikho/Desktop/WORKING/HOCTAPAZ/node-platform/apps/api/src/modules/questions/question-classification.worker.ts, /Users/velikho/Desktop/WORKING/HOCTAPAZ/node-platform/apps/api/src/modules/questions/question-generation.service.ts, /Users/velikho/Desktop/WORKING/HOCTAPAZ/node-platform/apps/api/src/modules/questions/question-generation.service.spec.ts, /Users/velikho/Desktop/WORKING/HOCTAPAZ/node-platform/apps/api/src/modules/questions/question-solution.service.ts, /Users/velikho/Desktop/WORKING/HOCTAPAZ/node-platform/apps/api/src/modules/questions/question-solution.service.spec.ts, /Users/velikho/Desktop/WORKING/HOCTAPAZ/node-platform/apps/api/src/modules/grading/grading.spec.ts.

Legacy source evidence:

  • /Users/velikho/Desktop/WORKING/HOCTAPAZ/node-platform/apps/api/src/modules/admin/ai-*.ts
  • /Users/velikho/Desktop/WORKING/HOCTAPAZ/node-platform/apps/api/src/modules/questions/question-classification*.ts
  • /Users/velikho/Desktop/WORKING/HOCTAPAZ/node-platform/apps/api/src/modules/questions/question-generation*.ts
  • /Users/velikho/Desktop/WORKING/HOCTAPAZ/node-platform/apps/api/src/modules/questions/question-solution*.ts
  • /Users/velikho/Desktop/WORKING/HOCTAPAZ/node-platform/apps/api/src/modules/grading/**
  • legacy provider usage and queue specs under /Users/velikho/Desktop/WORKING/HOCTAPAZ/node-platform/apps/api/src/modules/**.

Writable files:

  • services/ai-classifier-service/**
  • contracts/openapi/services/ai-classifier-service.yaml
  • docs/api/ai-classifier-service.md

Orchestrator-owned files:

  • AI route examples under deploy/gateway/routes.ai-*.json, deploy/gateway/routes.question-classification-*.json, route tests under scripts/test/ai-*.sh and scripts/test/question-classification-*.sh, provider secrets, default route table, shared deploy manifests, and billing/ wallet/usage integrations unless explicitly delegated.

Delegated gateway verification scope:

  • Only when the orchestrator assigns an AI route/live proof subtask, this service agent may update non-default deploy/gateway/routes.ai-*.json, deploy/gateway/routes.question-classification-*.json, scripts/test/ai-*.sh, scripts/test/question-classification-*.sh, docs/qa/question-classification-*.md, and root Makefiletest-ai-* / test-question-classification-* targets.
  • Provider secrets, billing/wallet/usage integrations, and the default gateway route table remain orchestrator-owned; public routes stay legacy until browser/runtime proof and rollback notes exist.

Initial tasks:

  • Extend classification foundation toward generate/explain/essay-grade task slices only after contracts are explicit.
  • Done: add an AI contract slice before implementation: typed internal request/result envelopes for classify, generate, explain, and grade, provider feature flags, idempotency keys, cancellation semantics, usage event fields, and rollback notes. The contract is internal-only and does not promote public /api/questions/ai-* routes.
  • Done: add a generation slice from QuestionGenerationService: internal POST /v1/ai/generation/preview accepts prompt metadata and optional document references, uses the configured non-mock provider for preview questions, records success or failure provider usage events, and carries an explicit question-service handoff for canonical question writes. The default mock is deterministic for local development. Public /api/questions/ai-generation/* routes stay legacy-proxied.
  • Done: add an explanation slice from QuestionSolutionService: internal POST /v1/ai/explanation/preview accepts editor question content, optional answer key, and existing solution text; uses the configured non-mock provider for preview solution text/steps, records success or failure question-solution usage metadata, and keeps canonical question solution writes in question-service. The default mock is deterministic for local development. Public /api/questions/ai-solutions/* routes stay legacy-proxied.
  • Done: add a non-default Gateway rehearsal for the two teacher preview flows: exact authenticated POST /api/questions/ai-generation/generate and POST /api/questions/ai-solutions/editor map to the internal preview owner endpoints through routes.teacher-ai-preview-native-*.json. The default route table remains legacy-proxied, the service returns preview payloads only, and neither flow creates canonical questions or solutions.
  • Audit status: generation and explanation previews are provider-backed when a non-mock adapter is configured, while mock remains deterministic in local development. Reviewed generation drafts and explanation solutions both use Question Bank versioned, idempotent owner APIs; generation settlement is also available behind its disabled-by-default usage/wallet adapter. Do not cite an AI preview alone as proof of a canonical Question Bank write.
  • Done: the teacher UI can submit a reviewed generation candidate through the exact opt-in Gateway path to Question Bank POST /v1/questions/ai-preview-drafts. AI service still returns only previews; Question Bank owns the forced DRAFT write, provenance, and preview-id idempotency. Reviewed explanation previews now use the separate Question Bank POST /v1/questions/ai-preview-solutions owner boundary to append a canonical solution version; AI remains preview-only and does not own Question Bank writes.
  • Done: add the bounded P6-047 generation monetization adapter. It is disabled by default and, when explicitly enabled, checks four usage-service quota windows before a provider call, reads wallet balance, then settles successful output through idempotent usage/consume and wallet USAGE_DEBIT requests. The adapter uses the legacy default of three AZ Credits per generated question and one thousand token units per credit. AI remains only the workload orchestrator; it does not own quota, balance, counters, or ledger storage.
  • Done: reduce P6-047 owner-call latency without changing owner boundaries. The four read-only quota checks fan out concurrently before the wallet read; after an accepted provider result and one wallet debit, the four resource-keyed usage/consume calls fan out concurrently. Both phases retain the documented resource-order error precedence, skip wallet reads on quota denial, and preserve the existing owner idempotency keys so P6-049 can reconcile any partial settlement.
  • Done: extend P6-048 charged-generation replay recovery. Immediately after provider output, the ai-classifier owner DB stores a 24-hour pending replay by organization, actor, and normalized generation hash. A same-key retry after owner settlement or telemetry-recording failure skips Question Bank reference loading, preflight, and provider execution, then retries settlement with the persisted provider usage. Only settlement plus telemetry success turns the record into a settled cache response. Pending replay save failures still return an error and can re-enter the provider later. Recovery relies on the existing owner-service idempotency keys; it does not compensate a partially applied wallet/usage sequence, and same-instance singleflight is not evidence of multi-pod in-flight coordination. This remains opt-in behind AI_GENERATION_USAGE_WALLET_ENABLED; it does not promote a default route.
  • Done: add P6-049 durable settlement reconciliation. With both AI_GENERATION_USAGE_WALLET_ENABLED=1 and AI_GENERATION_SETTLEMENT_WORKER_ENABLED=1, a separate worker claims due unsettled generation replays through the AI-owned Postgres store using SKIP LOCKED, lease, persisted attempt count, retry timestamp, and last error. Pending rows remain recoverable after preview-cache expiry until the original wallet/usage idempotency keys settle successfully. The worker never reruns provider, preflight, Question Bank context, or Question Bank writes. Partial owner sequences advance forward only; no compensation or public route promotion is implied.
  • Done: add a hermetic HTTP fault-injection proof for the P6-049 worker. It sends one real AI preview request through the HTTP monetization client, fails one usage-consume call after a wallet debit and three usage mutations, then runs the production settlement worker loop. Owner idempotency yields one effective debit and one effective consume per resource without rerunning the provider or quota preflight. This uses in-process owner fixtures only and does not prove real usage/wallet binaries, stores, or public-route promotion.
  • Done: add a Docker-independent P6-049 owner-runtime proof. The local PostgreSQL runner builds the three current service binaries, applies their current migrations to separate fresh databases, and restarts the AI worker after a real Wallet debit plus a fail-once real Usage consume. It asserts the durable replay, owner counters, immutable debit, audit outcome pair, and cached replay without a second owner call. It is local mock-provider evidence only and does not promote a Gateway route.
  • Done: the equivalent disposable Compose owner-runtime recovery passed at output/qa/ai-generation-settlement-runtime-20260714-084542/. It used unique owner Postgres volumes, observed the durable 1:true:0 to 1:false:1 recovery across AI recreation, then removed all project containers, volumes, and its loopback fault proxy. It remains local mock-provider evidence only.
  • Done for the bounded P6-046 preview contract: generation accepts typed fresh, advanced, and similar modes. Similar mode requires at most five referenceQuestionIds and resolves them through the existing scoped question-bank context API; it does not accept raw reference content from a caller. This remains a no-write preview and does not replace the approved question-service write or usage/wallet-preflight production slice.
  • Done: a fresh Compose Physics DOCX owner-runtime recheck starts from a real imported Question Bank reference, proves the similar and explanation calls both remain mock PREVIEW payloads with no usage/wallet consumption, then sends teacher-reviewed results to the two versioned Question Bank APIs. The similar draft and solution-version replays are idempotent, and the reviewed solution keeps the source DOCX importJobId. This is local owner evidence, not provider-credential, default Gateway, browser, or promotion proof. See docs/qa/docx-fixture-persistence-runtime-proof.md.
  • Done: add an essay-grade slice from grading evidence: internal POST /v1/ai/essay-grade/preview accepts attempt/question/answer snapshots by value plus rubric criteria, uses a configured OpenAI-compatible, Gemini, local, or Ollama adapter for rubric-constrained advisory scores with a deterministic mock fallback, records essay-grade provider usage metadata without Usage/Wallet consumption, and carries an explicit attempt-service handoff for stored grades. Public /api/attempts/:attemptId/submit stays legacy-proxied.
  • Done: add owner provider-status slice: GET /v1/ai/providers/status reports environment or database-selected provider/model readiness for admin-service ai.providers source snapshots without returning provider secrets or calling the live provider network.
  • Done: wire the AI-owned provider_settings default selection into startup and runtime. Internal ADMIN-only GET/PUT /v1/admin/ai/providers/settings stores provider/model/base URL plus an allowlisted secret env reference only; it validates the candidate before persistence, swaps the active provider, and does not add a Gateway/public route.
  • Done: add owner queue/failure projection and native retry command: GET /v1/admin/ai-classifier/queue-summary summarizes service-owned classification_jobs counts, pending/running depth, and recent failures for admin queue.depths; POST /v1/ai-classifier/jobs/{id}/retry retries only FAILED or CANCELLED native jobs through the configured provider. Admin-service/BFF may call these owner APIs but must not read the AI database directly.
  • Done: generation settlement integrates usage-service and wallet-service only through their owner HTTP check, consume, balance, and ledger APIs; the AI service never reads or writes either owner database.
  • Done: add workflow coverage that an in-flight CancelJob is not overwritten by a later provider result from ProcessJob.
  • Done: add workflow coverage for provider retry recovery usage, circuit-breaker-open usage, and SSE job.updated delivery after a pending job is cancelled.
  • Done: enforce defense-in-depth actor scope on interactive AI previews and classification jobs. The owner service now rejects missing actor/organization headers, allows only ADMIN/TEACHER preview/job callers, scopes list/read commands to that actor and organization, and requires ADMIN for the global queue projection. Gateway remains the authenticated header source; this does not promote any default public route.
  • Expand classifier jobs to request bounded question content, current classification, and taxonomy candidates from question-service before provider calls. Done locally through the optional QUESTION_BANK_SERVICE_BASE_URL / QUESTION_BANK_SERVICE_URL context client; keep it configured in runtime manifests and do not fall back silently when a configured client fails.
  • Done: add queue worker/batching semantics before public job route cutover. Deferred native classification jobs can be claimed from service-owned classification_jobs with attempts/max-attempts, nextRunAt, lockedUntil, and workerId; the opt-in runtime worker drains due jobs, processes provider calls in fixed batches of 5 question IDs, records one provider usage event per batch, and requeues worker-owned failures with exponential backoff. Public /api/questions/ai-classify/* routes remain legacy-proxied.
  • Done: add bounded per-job provider batch concurrency for latency. The runtime defaults AI_CLASSIFIER_BATCH_CONCURRENCY=1 and clamps it to five, independently of AI_CLASSIFIER_WORKER_CONCURRENCY. Higher values fan out a wave of fixed-size provider batches while retaining ordered result items, batch-indexed usage telemetry, and sequential progress persistence. The focused race test proves three concurrent provider calls and stable output; this is not provider-rate-limit, cross-pod, browser, or default-route proof.
  • Done: add an opt-in disposable owner-runtime smoke for the native worker. It starts an in-memory or fresh Postgres-backed ai-classifier-service with the mock provider, creates a deferred twelve-question job, proves worker drain, native job detail/SSE and queue summary, then removes the process, container, and temporary data. Its question-bank mode separately starts a disposable memory-backed Question Bank, creates twelve canonical questions through the owner API, and proves the completed AI job retains all twelve bounded prompt rows. Its optional job-read mode uses the dedicated non-default Gateway route table to prove authenticated native list/detail/SSE reads, route headers, and a student-role deny without changing routes.json. See docs/qa/ai-classification-worker-runtime-smoke.md. A separate Teacher read-only job surface now consumes its exact list/detail Gateway paths with a disposable browser session and renders worker context/result fields. The worker harness itself still does not claim concurrent Postgres lock, a deployed Gateway, public route, provider credential, Question Bank Postgres/taxonomy, or usage/wallet settlement coverage.
  • Add an apply endpoint or explicit apply handoff to question-service with idempotent results and audit events.

Acceptance:

  • AI-service does not decide quota/payment locally.
  • Provider secrets remain env/secret managed.
  • Public job routes stay legacy until BullMQ/runtime/browser parity is proven.
  • Generate/explain/grade results are previews or explicit handoff payloads; they do not mutate question-service, exam-service, or attempt-service state without a versioned owner-service API.
  • Provider settings are AI-owned runtime selection data, while usage-service and wallet-service remain the only owners for quota checks and credit movement.
  • Provider status is exposed through ai-classifier-service owner API only; admin-service may probe it but must not own provider settings or secrets.
  • Queue depth/failure state and retry commands are exposed through ai-classifier-service owner APIs only; admin-service and BFF must not inspect classification_jobs directly.
  • The first generation/explanation/grading slices must keep mock fixtures and provider failure paths testable without live provider credentials.
  • Prompt payloads include enough real question context for parity; ids-only payloads are not considered production-ready.
  • Provider usage events are test-covered on success and failure. Generation's opt-in usage/wallet settlement is tested through owner APIs; explanation and grade charging remain separate workload decisions.
  • Queue worker claims, batch progress, retry backoff, and final max-attempt failure are test-covered before any public route promotion.
  • Generation settlement worker claim/lease/backoff is test-covered before any charged public route promotion. It has no terminal retry cap because an already-debited wallet must stay recoverable through owner idempotency keys.

Verification:

  • GOTOOLCHAIN=go1.25.11 go test ./services/ai-classifier-service/...
  • make test-ai-generation-settlement-reconciliation
  • make test-ai-generation-settlement-fault-injection
  • AI_GENERATION_SETTLEMENT_RUNTIME_CONFIRM=disposable-runtime make test-ai-generation-settlement-runtime starts real AI, Usage, and Wallet binaries with fresh Compose volumes, injects one credits_weekly consume failure after the real wallet debit, then checks durable worker recovery and replay-cache behavior. It passed at output/qa/ai-generation-settlement-runtime-20260714-084542/, requires Docker, and is local evidence only; it does not promote a Gateway route.
  • AI_GENERATION_SETTLEMENT_LOCAL_POSTGRES_CONFIRM=local-postgres make test-ai-generation-settlement-local-postgres starts the same proof with a fresh /tmp PostgreSQL cluster and native binaries. It does not depend on Docker or a configured local database.
  • The same target guards Compose, Helm, static K8s, and the service env example for the opt-in settlement worker; both charging and settlement-worker flags stay false by default.
  • AI_CLASSIFIER_POSTGRES_DSN=postgres://... GOTOOLCHAIN=go1.25.11 go test ./services/ai-classifier-service/internal/repository -run TestPostgresGenerationSettlementClaimPersistsAcrossStores -count=1
  • Focused owner queue/retry regression: GOTOOLCHAIN=go1.25.11 go test ./services/ai-classifier-service/internal/http -run TestAIClassifierAdminQueueSummaryAndRetry -count=1
  • GOTOOLCHAIN=go1.25.11 go test ./services/ai-classifier-service/internal/usecase -run 'TestClassifierServiceGeneratePreview|TestClassifierServiceExplainPreview|TestClassifierServiceGradePreview|TestClassifierServiceProcessJob|TestHTTPQuestionContextClient' -count=1 -v
  • GOTOOLCHAIN=go1.25.11 go test ./services/ai-classifier-service/internal/http -run 'TestAIProviderStatus|TestAIClassifierGenerationPreview|TestAIClassifierExplanationPreview|TestAIClassifierEssayGradePreview|TestAIClassifierJobUsesQuestionContextClient|TestNewQuestionContextClientFromEnv' -count=1 -v
  • make test-ai-provider-settings
  • make test-ai-classification-job-route-guard
  • make test-ai-classification-job-routes
  • make test-teacher-ai-preview-routes
  • pnpm exec vitest run packages/ui/src/modules/teacher/teacher-workspace.test.tsx
  • AI_CLASSIFICATION_JOB_SELF_TEST=1 make test-ai-classification-job-live
  • AI_CLASSIFICATION_WORKER_SMOKE_CONFIRM=disposable-runtime make test-ai-classification-worker-runtime
  • AI_CLASSIFICATION_WORKER_SMOKE_CONFIRM=disposable-runtime AI_CLASSIFICATION_WORKER_SMOKE_STORE=postgres make test-ai-classification-worker-runtime
  • AI_CLASSIFICATION_WORKER_SMOKE_CONFIRM=disposable-runtime AI_CLASSIFICATION_WORKER_SMOKE_CONTEXT=question-bank make test-ai-classification-worker-runtime
  • AI_CLASSIFICATION_WORKER_SMOKE_CONFIRM=disposable-runtime AI_CLASSIFICATION_WORKER_SMOKE_STORE=postgres AI_CLASSIFICATION_WORKER_SMOKE_CONTEXT=question-bank make test-ai-classification-worker-runtime
  • AI_CLASSIFICATION_WORKER_SMOKE_CONFIRM=disposable-runtime AI_CLASSIFICATION_WORKER_SMOKE_CONTEXT=question-bank AI_CLASSIFICATION_WORKER_SMOKE_GATEWAY=job-read make test-ai-classification-worker-runtime
  • AI_CLASSIFICATION_WORKER_SMOKE_CONFIRM=disposable-runtime AI_CLASSIFICATION_WORKER_SMOKE_STORE=postgres AI_CLASSIFICATION_WORKER_SMOKE_CONTEXT=question-bank AI_CLASSIFICATION_WORKER_SMOKE_GATEWAY=job-read make test-ai-classification-worker-runtime
  • For an owner-local browser diagnostic only: AI_CLASSIFICATION_WORKER_SMOKE_CONFIRM=disposable-runtime AI_CLASSIFICATION_WORKER_SMOKE_STORE=postgres AI_CLASSIFICATION_WORKER_SMOKE_CONTEXT=question-bank AI_CLASSIFICATION_WORKER_SMOKE_GATEWAY=job-read AI_CLASSIFICATION_WORKER_SMOKE_HOLD_SECONDS=240 make test-ai-classification-worker-runtime, then use the in-app browser against the temporary Gateway and clear its disposable session before cleanup.
  • make test-question-classification-apply-routes
  • Runnable AI classifier smoke before route promotion: make test-question-classification-apply-live.
  • make test-ai-classification-job-live is a live legacy-route guard for job list/SSE headers. Native job SSE/cancel route promotion still needs a later disposable-fixture smoke before any public route changes.
  • Future generate/explain/grade slices must add focused service tests under services/ai-classifier-service/internal/**, OpenAPI mirror checks when contracts change, and route guards proving default public routes stay legacy.

Go-platform documentation is generated from repository Markdown.

Copyright 2026 HocTapAZ