Integration · Object storage

A GCS bucket in, your schema out.

A service-account key, a bucket and a prefix. AdaptivMapr lists, takes the newest object since the last run, parses it by extension and maps it — and writes the mapped result back as one timestamped file.

Get an API key Read the docs

Method: POST
Connector kind: 1

How it works

What actually happens on a sync run

The GCS connector is the same shape as the S3 one — list a prefix, take the newest object modified since the last successful run, parse it by its own key, map it — with a different signature at the bottom. Auth is a service-account key file exchanged for a scoped access token; the JSON API answers the listing, and the response is normalized into the same object shape every provider returns, so nothing downstream branches on which cloud it came from.

1
Mint a token
The service-account JSON is parsed for client_email and private_key and exchanged for a scoped access token — refused at save time if either is missing.
2
List and filter
The JSON API lists config.prefix; objects modified after the watermark survive and the newest is taken.
3
Parse by key
Extension-driven, including xlsx, xlsm, parquet and docx.
4
Map and land
An upload under your retention setting, then the cascade against the connector’s template.

What you get

Built for files that keep arriving

Auth

A service account, scoped to one bucket

Grant the service account only the bucket it needs. AdaptivMapr exchanges the key for a scoped token per call and holds nothing else.

HowgcpAccessToken() signs a JWT with the key and exchanges it for a token scoped to storage. The key itself is KEK-encrypted at rest and masked on every read.

Incremental

The watermark is the object’s own modified time

Only objects newer than the last successful run are candidates, and only the newest is read. A run that fails does not move the watermark.

HowOverridable per call with ?since=<iso>. Nothing new comes back as status: "not_modified" — success with zero rows.

Write-back

One object per run, timestamped

A destination write serializes the accepted rows into a single file under your prefix. Yesterday’s export is never silently overwritten.

HowThe key template is export-{date}-{time}.{ext}; pass destination.key to name it yourself. Both tokens are substituted at write time.

Secrets

The credential never travels in a request body

You reference a connector by id. Keys, tokens and service-account JSON are encrypted at rest and read only by the code that makes the call.

HownormalizeSecretField() folds every provider spelling — private_key, token, secret_access_key, account_key, service_account_json — into one auth_value field, which is KEK-envelope-encrypted before the row is written. A GET masks it to a 4-character hint. If encryption fails the field is dropped rather than stored in plaintext.

Configuration

The connector record, field by field

The key file is validated by parseGcpCredentials() BEFORE the record is saved, so a paste that lost its newlines fails now rather than on the first scheduled run.

Key	Required	What it is
bucket	Required	Bucket name.
service_account_jsonsecret	Required	The whole service-account key file. Must parse and carry `client_email` and `private_key` or the save is refused. Stored as `auth_value`.
prefix	Optional	Key prefix scoping the read and the write.
format	Optional	Override extension sniffing.
url	Optional	Legacy presigned-URL form. Read-only.

In code

A bucket export, mapped and delivered.

The connector holds the service-account key. The call that uses it carries a bearer token and an id — nothing else.

POST/v1/connectorsSave the connector. The secret is encrypted before it reaches Postgres.session
POST/v1/connectors/{id}/testMake a real call and report what was actually proven.session
POST/v1/connectors/{id}/syncPull now. Accepts ?since=<iso> to override the watermark.bearer
POST/v1/gatewayAny input in, this destination populated, a delivery report out.bearer
POST/v1/connectors/{id}/rotate-secretReplace the credential in place; the old one becomes unrecoverable.session

A not_modified answer is deliberately a 200. A scheduler that treats “no new file” as an error will page someone every quiet night.
The service-account key is validated at save time, so a malformed paste is 400 config_invalid immediately rather than a 3am failure.
Objects are capped at 25 MiB and calls time out at 60 seconds.

POST /v1/connectors

{
  "kind": "gcs",
  "name": "Warehouse drop",
  "template_id": "orders",
  "config": {
    "bucket": "acme-analytics-drop",
    "prefix": "orders/",
    "service_account_json": "{\"client_email\":\"mapr@acme.iam.gserviceaccount.com\",\"private_key\":\"-----BEGIN PRIVATE KEY-----\\n…\"}"
  }
}

curl

curl -X POST "https://api.adaptivmapr.com/v1/connectors/con_31ab…/sync?since=2026-09-01T00:00:00Z" \
  -H "Authorization: Bearer $MAPR_API_KEY"

response

{
  "ok": true,
  "status": "not_modified",
  "upload_id": null,
  "row_count": 0,
  "incremental": "incremental"
}

→ nothing modified since the cursor · reported as success, not as a failed run

Limits & failure modes

What it refuses, and what it tells you

Code	When	What to do
`400 ssrf_blocked`	The configured host resolves to a private, link-local or loopback address.	Every outbound request is DNS-resolved and checked before it is made, on the scheduled path and the on-demand path alike. The reason is returned with the code.
`400 config_invalid`	A credential or identifier the provider needs is missing or malformed.	Refused at save time rather than at 3am during a scheduled run. The message names the exact key.
`400 config_invalid`	The service-account JSON does not parse, or lacks client_email / private_key.	Checked at save time by `parseGcpCredentials()`, so an unusable connector is never created.
`413 fetch_too_large`	The chosen object exceeds 25 MiB.	Split the export upstream; the ceiling matches the parser’s own memory envelope.
`401 auth_failed`	The token exchange is refused.	Usually a revoked key or a missing role on the bucket. The provider message is passed through.

Incremental sync

Real, by object modified time. The listing is filtered against the watermark and only the newest survivor is read. A failed run leaves the watermark where it was, so the next tick covers the same window again rather than skipping the file that failed.

PHI & residency

A GCS sync runs under the workspace’s routing policy, not around it. PHI routing is a separate axis from the data mode: it sends X-PHI and X-Region to phi-cloud so a regulated run lands on an in-region, BAA-eligible model, it costs +20% on the whole charge, and it is locked until the workspace accepts the BAA in Settings → Security & Data. An explicit PHI ask without an acceptance is 403 agreement_required, never a silent downgrade. A standard run keeps the workspace’s region pin — the region decides where compute may run, and the sandbox refuses a region-less run.

What it costs

Billed on the same prepaid wallet

Moving bytes is not a line item. A sync that pulls a file and a destination write that lands the rows are both part of one map, and the map is what the wallet sees. There is no free tier, no seats and no contract — top up from $10, a balance shared across the phi-cloud suite.

Charge	Rate	Notes
Every map	$0.001	A flat per-map fee — a few tokens — charged even when the run was fully deterministic or hit the layout cache and used no AI at all.
AI, only when it ran	at cost × 2	Layer-5 cleanup, any-to-any convert and structural reshape bill the phi-cloud tokens actually consumed. Bring your own model key and it is × 0.5.
PHI / enterprise routing	+20%	Multiplies the whole charge, flat fee included — and only when the run genuinely got that routing. Locked until the workspace accepts the BAA in-app.

Questions

Before you wire it up

What IAM role does the service account need?

Enough to list and read objects under the prefix you configure, and to write objects if you use the connector as a destination. Grant it on the single bucket rather than the project: AdaptivMapr only ever lists the configured prefix, reads one object per run and — as a destination — writes one object per run.

Can I use a presigned URL instead of a key?

For reading, yes — a presigned https URL is the legacy form and still works. It cannot be a destination, because there is nothing to sign a write with, and attempting one returns config_invalid rather than failing halfway.

Which file formats can it parse out of the bucket?

The object’s own key picks the parser, so csv, tsv, json and xml go through the text path and xlsx, xlsm, parquet and docx go through the binary one. Set config.format to override the sniffing when a producer writes .txt files that are really TSV.

Does a sync count as a map for billing?

The map does. Moving the bytes is not separately metered — a sync that pulls a file and runs the cascade draws the flat per-map fee, and AI tokens are billed only if the metered layer actually ran. A PHI-routed run multiplies the whole charge by 1.2.

Amazon S3 →BigQuery →Connectors & webhooks →

Verified against lib/objectStore.ts · lib/cloudAuth.ts · lib/connectorSyncRunner.ts · app/api/v1/connectors/route.ts

Google Cloud Storage is a trademark of Google LLC. Named here to describe interoperability only — no affiliation, endorsement or partnership is claimed.

Point it at Google Cloud Storage. Get your schema back.

Start with a $10 prepaid wallet. Every map draws a few tokens; in schema-only mode only headers and up to three sample rows, clamped to 80 characters a cell, ever leave you.

Get an API key All integrations