Capability · Reshape

The workbook you need, from the one you got.

The workbook you were sent is not shaped like the workbook you need. Describe the one you need and get it back — your own template, its macros intact.

Start mapping Read the docs

Round-trips with macros: .xlsm
One input, many outputs: Multi-sheet

The job

The data is all there. The layout is wrong.

Months across the top instead of down the side. One sheet that should be twelve. A header band three rows deep with units under the names. A join nobody wrote down, living in a lookup tab. The columns exist — they are just not arranged the way the system that has to read them expects, and rearranging them by hand is a morning that comes back every month.

What changes

Say what the output should look like, in a sentence. You get a workbook — transposed, un-pivoted, joined, split across sheets — and if you handed us your own template, you get that template back with its macros untouched.

What you get

Not a demo. The thing that runs every day.

Detect

Untangle the grid before any model sees it

Headers that are not on row 0, multi-row and grouped headers, unit rows, and fully transposed layouts are all found and normalized into clean logical tables.

Howlib/structure.ts is deterministic and source-agnostic: it flattens a grouped header into Instrument › Variable, strips the unit row, and auto-un-pivots a transposed sheet — all before a model is called, so the model reasons about a table, not a mess.

Understand

One turn that can stop the run

Before any transform is generated, the engine restates the task from the inputs it has. If the restatement does not fit, the run ends with a question instead of an expensive wrong answer.

Howlib/reshape/understand.ts returns an analysis that anchors every generation attempt and rides back on the response. A fit: 'mismatch' verdict carrying a question ends the run as 422 needs_clarification; you answer and re-run with skip_clarification.

Route

Three strategies, one PHI-free router

A classifier picks the cheapest strategy that fits the job: direct, plan, or code. It runs before any data gate, because it never touches data.

HowheuristicRoute() in lib/reshape/router.ts sees table shapes and the target sheet list only — never a row value. Dynamic output (pass-through columns, per-group sheet fan-out) is routed straight to code, because the plan DSL provably cannot express it.

Plan

A declarative transform, executed in-process

The default path. The model writes a JSON reshape plan — rename, derive, filter, unpivot, pivot, join — and AdaptivMapr executes it. No eval, byte-reproducible, and the plan comes back with the result.

Howlib/reshape/execute.ts runs the ops inside the Worker. Nothing is compiled and nothing is evaluated, so a re-run of the same plan on the same input produces the same bytes — and you can read the plan before you trust it.

Code

A generated program, in a sealed sandbox

When the job is beyond a plan, the model writes Python/pandas, R or SQL/DuckDB. Generation and execution are two separate phases, and only the first involves a model.

HowPhase 1 shows the model table shapes plus ≤3 clamped sample rows. Phase 2 posts { language, code, input } to a jurisdiction-pinned, network-isolated container with no model in the loop — so the run is byte-stable and the program is a re-runnable, auditable artefact.

Output

One mess in, a whole workbook out

Split a single tangled input into a tidy multi-sheet workbook — one logical table per sheet, mapped to your target schema. Excel macros survive the round trip when you ask for them.

Howoutput: "xlsm" returns the full workbook: template sheets, rewritten data and the source template's VBA project intact. Every other output value strips macros — the macro-enabled FORMAT is the consent, so nothing ships executable content unless you named it.

How it works

Three strategies, and when the router picks each

Real spreadsheets are not tidy tables. Headers sit three rows down, units live on their own row, grouped headers span columns, and the whole thing is transposed. Reshape handles the structure the cascade assumes away: a deterministic detection pass first, then one understanding turn that restates the task, then a PHI-free router that picks the cheapest strategy that can actually express the job.

01
Detect the real grid
lib/structure.ts finds the header row wherever it sits, flattens grouped headers, strips the unit row and un-pivots a transposed sheet — deterministically, before a model is called.
02
Restate the task
One understanding turn says back what it thinks you asked for. A mismatch verdict carrying a question ends the run there, rather than spending a long generation on a guess.
03
Route on shape alone
A classifier picks direct, plan or code from table shapes and the target sheet list. It never reads a row value, so it runs before any data gate.
04
Generate against the analysis
The model writes a JSON plan, or Python/pandas, R or SQL/DuckDB. Every attempt is anchored to the same restated analysis, and a critic reads the result.
05
Execute, then validate
A plan runs in-process with no eval; a program runs in a network-isolated, region-pinned sandbox with no model in the loop. Every produced row goes through your field validators.

Strategy	Chosen when	Where it runs	What comes back
direct	One source table maps straight onto one target sheet	In-process — it is the cascade	Mapped rows
plan	Structural work: transpose, un-pivot, pivot, join, multi-sheet split	In-process, deterministic, no eval	The JSON plan + the sheets
code	Dynamic output — pass-through columns or per-group sheet fan-out the DSL cannot express	Jurisdiction-pinned, network-isolated sandbox	The program + the sheets

The router reads table shapes and the target sheet list — never a row value — so the decision is PHI-free and happens before any data gate. You can force one with strategy, or supply a stored plan or code artefact to replay verbatim with no generation at all.

Try it

A messy workbook in, clean sheets out

Post the file and name the target. The response tells you which strategy the router chose, how the engine understood the task, the transform it produced, and the sheets it wrote.

curl

curl https://api.adaptivmapr.com/v1/reshape \
  -H "Authorization: Bearer $MAPR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "file": { "name": "lab-export.xlsx", "content_base64": "UEsDBBQAB…" },
    "target": { "template_id": "lab_results_v1" },
    "output": "json"
  }'

response

{
  "ok": true,
  "strategy": "plan",
  "decision": { "reason": "structural reshape: transposed layout, unit row", "confidence": 0.6 },
  "analysis": "One sheet holds analytes as COLUMNS per draw. Un-pivot Na/K/Cl into
               analyte + value, carry the unit row down, keep patient_id + taken_at.",
  "plan": {
    "sheets": [{ "name": "results", "ops": [
      { "op": "unpivot", "id_cols": ["patient_id", "taken_at"],
        "value_cols": ["Na", "K", "Cl"], "into": ["analyte", "value"] },
      { "op": "derive", "field": "unit", "from": "analyte_unit_row" }
    ]}]
  },
  "sheets": [{ "name": "results", "headers": ["patient_id","loinc_code","value","unit","taken_at"],
               "rows": 1840, "warnings": [] }],
  "review": { "approved": true, "attempts": 1 },
  "requires_hitl": true,
  "usage": { "prompt_tokens": 3120, "completion_tokens": 486, "ai_calls": 2 }
}

→ strategy plan · 1 840 rows · requires_hitl because lab_results_v1 is a medium-risk template

Every produced row goes through validateRowsDetailed() — the same chokepoint the commit path uses. Reshape output is not exempt from your field validators, and a critic-rejected result is reported as rejected rather than shipped as done.
POST /v1/me/reshape/stream streams phase frames over SSE and ends with the same terminal JSON as the one-shot — that is what the Workbench renders live.
Reshape materializes transformed data, so it needs the high-risk commit scope on the key, and schema-only clamps every source table to ≤3 rows × ≤80 characters before the engine sees them.

The surface

Every route this page actually has.

One bearer route for programmatic use, and the session-cookie twins the Workbench drives. Both planes share one compute core — lib/reshapeRequest — so they cannot diverge on the PHI gate, the schema-only clamp or the validators.

POST/v1/reshapeA file or a stored upload_id in, the transform and the reshaped sheets out. 30 requests a minute per workspace.commit scope
POST/v1/uploadsStage a workbook up to 10 MiB first, then reshape it by upload_id instead of re-sending the bytes.transform scope
POST/v1/me/reshapeThe dashboard twin of the bearer route — the same core, without a bearer key in the browser.dashboard only
POST/v1/me/reshape/streamSSE phase frames while the run happens, ending with the identical terminal JSON. This is what the Workbench renders live.dashboard only
POST/v1/me/detectInspect a workbook before you reshape it: sheets, detected header rows, and a deterministic macro report.dashboard only
POST/v1/schemas/from_textTurn a plain-English description — or an existing file’s columns — into the target schema the reshape writes into. Bearer key or session cookie.transform scope

Limits & failure modes

What it refuses to do — and the code it says it with.

Reshape is the most expensive thing the engine does, so most of its rails exist to stop a run BEFORE it spends. Each of these ends the run early and says why.

422 needs_clarification

The understanding pass cannot reconcile your instruction with the file, and has a specific question.

Answer it and re-run with skip_clarification. A verdict without a question is ignored — a stop must be actionable to be worth the interruption.

422 needs_program_sandbox

The target needs the code strategy — a header-band sheet, per-group fan-out — and the sandbox is off.

The engine stops instead of "trying plan": the DSL provably cannot build a band, and three attempts of junk cost more than a clean refusal.

422 no_tables

No source table could be detected in the uploaded workbook.

Usually a picture of a table, or a sheet with no coherent grid at all. Route it through /v1/convert instead.

400 xlsm_requires_template

output: "xlsm" asked for on a rows-to-file surface with no source workbook.

The best that path could produce is a file NAMED .xlsm that declares itself plain inside — the exact mismatch Excel refuses to open.

review.approved: false

The critic rejected the produced result.

Reported as rejected, with the objection — never dressed up as done. The artefact is still returned so you can see what it got wrong.

429 rate_limited

More than 30 reshape runs a minute on one workspace.

The shared counter is authoritative across isolates; the per-isolate one only ever denies earlier.

What it costs

One prepaid wallet, drawn down per call.

No free tier. Top up from $10 — the balance is shared across the phi-cloud suite — and every operation draws it down at the rate below. An optional $6/mo plan grants a credit that resets each billing cycle instead; overage falls back to the wallet.

Every reshape

$0.001

The flat per-map fee, the same as any other map. A replayed plan or code artefact pays this and nothing else.

Generation

cost × 2

The phi-cloud tokens the understanding, codegen and critic turns actually consumed — × 0.5 with your own LLM key. Codegen-class turns are promoted to the strongest in-catalogue model.

PHI-routed run

+20%

On the whole charge, flat fee included — and only when the run actually got that routing. A downgraded run is billed standard and carries a phi_downgraded flag plus a visible warning.

A PHI/enterprise-routed run multiplies the whole charge by 1.2 (+20%), flat fee included — and only when the run genuinely got that routing. PHI stays locked until the workspace accepts the BAA in Settings → Security & Data. Full pricing

Questions

The ones asked before signing.

What it costs, what leaves your network, and the claims we will not make.

How is this different from the mapping cascade?

The cascade maps columns to fields on an already-tidy table. Reshape changes the table’s structure first — transposing, un-pivoting, pivoting, joining and splitting — so a spreadsheet that is not a clean grid can still land on your schema. Reshape often finishes by handing its clean output to the cascade.

Does reshape send my data to the model?

The detection pass is deterministic and local, and the model is shown table shapes plus up to three clamped sample rows — never the full dataset as prompt text. A plan is then executed in-process; a generated program runs over the full data in a network-isolated, region-pinned container with no model in the loop.

Why generate a JSON plan instead of just running code?

A declarative plan is inspectable, deterministic and byte-reproducible, and nothing arbitrary executes — so it is the safe default. Code generation is the escape hatch for transforms a plan provably cannot express, such as source-defined pass-through columns or fanning one prototype sheet out into many.

What happens if the request is ambiguous?

The understanding pass runs before any transform is generated. If it cannot reconcile your instruction with the file, the run stops with 422 needs_clarification and a specific question instead of burning a long generation on a guess. You answer, re-run with skip_clarification, and it proceeds.

Will my Excel macros survive?

Only if you ask. Generated workbooks are macro-free by default; output: "xlsm" returns the full round trip — template sheets, rewritten data and the source template’s VBA project intact. The macro-enabled format is the consent, so no other output value can ship executable content.

Keep reading

The rest of the same engine.

Schema mapping Any-to-any convert Schema-only vs full-data

One key, from a messy file to your schema.

Top up a $10 prepaid wallet and start mapping. In schema-only mode only your headers and three clamped rows ever leave you.

Get an API key Read the docs