Skip to content

vsync-s3-client v0.9.0 — Spec

Status: design · new sibling packages under secret-lib/libraries/{python,typescript,go}/ · read-side runtime library, not a CLI feature. CLI in this repo (@muthuishere/vsync) is unchanged by this spec.

One theme: the CLI writes; this library reads. vsync (the CLI) remains the canonical writer of the encrypted S3 bundle. vsync-s3-client is a tiny in-process library that runs inside an application on boot, pulls the bundle directly from S3, decrypts it with a passphrase, and exposes a 360° accessor. No file on disk, no daemon, no refresh API. One round trip, in memory, done.

This is the first of a per-backend family — vsync-s3-client ships in v0.9. Future siblings (vsync-gh-client, vsync-vault-client, …) are out of scope for this spec but will follow the same shape (same accessor API, same fallback chain, same trust ceiling — different fetch path).

For prior context, see v0.2-secret-lib.md (original spec — source of truth for the RQE1/RQEM0001 envelopes, do not re-spec here), v0.4-audit-log.md (audit.csv is written by the CLI; this library doesn't write to it), v0.8-multi-target-sync.md (positions S3 as the canonical bundle store; other backends are downstream fanout).


1. Positioning

CLI (@muthuishere/vsync)vsync-s3-client (this spec)
Rolecanonical writerread-side runtime
Lives ona teammate's laptop, CI runnerinside the application process
Disk usagegzipped config at ~/.config/vsync/..., keychain keynone (memory-only)
Writes to S3?yes (push, audit log)never
Refresh?every pull is explicitone fetch at Open(), that's it
LanguagesBun-native (TypeScript)Python, TypeScript, Go

The library is intentionally narrow. It does:

  • Decode the bootstrap config blob, fetch the manifest + bundle from S3, decrypt with the passphrase.
  • Expose getEnv(key), hasEnv(key), envSource(key), getAsContent(name), generation(), remoteGeneration(), hasNewVersion(), close(). Two open paths: open() (reads env) and openWith(config, passphrase) (accepts strings directly).
  • Apply a deterministic fallback chain when a key is absent from the vault.

It does NOT: write to S3, mutate the bundle, manage rotation, run as a daemon, watch for changes, expose a network port, re-read its config after boot, or know what vsync sync did with the values downstream.


2. Bootstrap contract — two inputs, full stop

Every language binding reads exactly two process inputs at Open() time. No discovery, no .vsyncrc, no XDG lookup, no DNS.

InputPurpose_FILE variant
VSYNC_CONFIGgzip+base64url JSON blob (S3 endpoint, bucket, IAM key, prefix, env, KDF salt + iterations)VSYNC_CONFIG_FILE
VSYNC_PASSPHRASEpassphrase for the RQE1 envelope ([v0.2 §3])VSYNC_PASSPHRASE_FILE

Rules:

  • _FILE wins if both forms are set. (Matches PostgreSQL / Docker secrets convention — operator who mounts a file means it.)
  • Read once, on Open(). Held in process memory for the life of the handle. Mutating the env var after Open() returns has no effect.
  • If neither form is set for a given input → ConfigMissingError. Don't fall back to anything; fail loud.
  • Trailing whitespace / newlines on the file variant are stripped. The env variant is taken verbatim (no trim — a leading space in a passphrase is part of the passphrase).

2.1 VSYNC_CONFIG blob format

vsync-cfg-v1:<base64url-no-pad(gzip(json))>

The vsync-cfg-v1: magic prefix is required. Wrong prefix → ConfigMissingError with a hint that the operator likely passed raw JSON or a different version blob. The magic is also the schema-version handle (§11).

Encoding is base64url (RFC 4648 §5), no padding. Keeps the blob safe in env vars, YAML, shell args, and CLI flags without quoting. Standard base64 (with +, /, =) MUST NOT be accepted by readers.

Inner JSON (v: 1):

json
{
  "v": 1,
  "endpoint": "https://s3.amazonaws.com",
  "region": "us-east-1",
  "bucket": "acme-secrets",
  "accessKeyId": "AKIA...",
  "secretAccessKey": "...",
  "prefix": "myapp/",
  "env": "prod",
  "salt": "<base64 16 bytes>",
  "iterations": 600000
}
  • endpoint is required and explicit — works with AWS, R2, MinIO, Backblaze B2, any S3-compatible store. No "default to AWS" guessing.
  • prefix is the bundle-namespace prefix used by the CLI on push. <prefix>manifest + <prefix>v=<ts> is the read path (the CLI scopes <env> into the prefix at init time, so the prefix typically already ends with <env>/ — e.g. myapp/prod/). Matches v0.10 §3.3 — the CLI's writer side is authoritative.
  • env selects which environment's bundle this process reads. One client = one env. (To read multiple envs in one process, open multiple clients.)
  • salt is the PBKDF2-SHA256 salt emitted verbatim from the CLI's cfg.encryption.salt (a 24-char base64url ASCII string from vsync init). Readers MUST feed the UTF-8 bytes of this string directly to PBKDF2 as salt input — do NOT base64-decode first. The field's base64-like alphabet is a CLI storage artefact, not a transport-encoding marker; the bytes PBKDF2 actually sees are the 24 ASCII characters' UTF-8 representation. This convention matches both src/crypto.ts::deriveKey (the CLI's own encrypt/decrypt path) AND the test-vector corpus (docs/specs/test-vectors/rqe1-decrypt/* were generated with this convention). Length is variable; readers SHOULD reject < 8 UTF-8 bytes of salt for sanity. Field name is salt for v: 1 backward compatibility; a future v: 2 blob may rename to salt_string to signal the opacity explicitly.
  • iterations is the PBKDF2 iteration count. Default and reference value is 600000 (v0.2 spec). Carried explicitly for forward-compat — future libs may support different work factors.
  • Unknown fields ignored on read (forward-compat with field additions inside v: 1).
  • v: 2 (or higher) hitting a v1 lib → ConfigUnsupportedVersionError. No silent downgrade.

2.2 Producing the blob

Out of scope for this spec — the CLI subcommand that emits it lives in v0.10 (vsync runtime-token). For v0.9, operators can hand-roll it: printf '...json...' | gzip | base64 | sed 's/^/vsync-cfg-v1:/'. Document this in the library README, not here.


3. Delivery split — VPS vs cloud

Same library, two operator patterns. The library doesn't care which is in use; the trust story differs.

VPS / bare-metal / Docker (file-backed):

bash
# /etc/myapp/env (root-owned, 0600)
VSYNC_CONFIG_FILE=/run/secrets/vsync-config
VSYNC_PASSPHRASE_FILE=/run/secrets/vsync-passphrase

Host filesystem trust is the perimeter. Use _FILE variants. The library warns on world-readable mode bits (§14).

Cloud (env-direct from a platform secret store):

Vercel:       Environment Variables UI → VSYNC_CONFIG, VSYNC_PASSPHRASE
AWS:          Task definition → "secrets": [{ "valueFrom": "arn:aws:secretsmanager:..." }]
GCP Cloud Run: --set-secrets=VSYNC_CONFIG=projects/.../secrets/cfg:latest
Azure:        Key Vault reference → @Microsoft.KeyVault(SecretUri=...)

Platform injects directly into the process environment; no filesystem involved. Use the env variants. The library treats them identically.

Operators pick one pattern per environment. Mixing (config from file, passphrase from env) is allowed and works — the _FILE-wins rule resolves per-variable independently.


4. Public API — per language, idiomatic

The shape is consistent but not forced-identical. Each binding looks like it was written by someone who lives in that language.

4.1 Python

python
import vsync_s3_client

# Two ways to open
v = vsync_s3_client.open()                                       # reads VSYNC_CONFIG + VSYNC_PASSPHRASE from env
v = vsync_s3_client.open_with(config=blob, passphrase=pw)        # accept strings directly — from KMS, file, anywhere

# Get scalar env-style values
db_url = v.get_env("DATABASE_URL")                # → str | None
exists = v.has_env("STRIPE_KEY")                  # → bool
src    = v.env_source("DATABASE_URL")             # → "vault" | "env" | "default" | "missing"

# Get binary content (replaces both asset_bytes + asset_path)
sa_json = v.get_as_content("service-account.json")  # → bytes; in-memory always

# Generation & explicit-poll
gen    = v.generation()                           # → int (gen at open time)
remote = v.remote_generation()                    # → int (HEAD manifest; raises on network fail)
stale  = v.has_new_version()                      # → bool (convenience: remote > local)
v.close()

# Context manager
with vsync_s3_client.open() as v:
    db_url = v.get_env("DATABASE_URL")
  • Defaults: vsync_s3_client.open(defaults={"PORT": "8080"}) or vsync_s3_client.open_with(config=..., passphrase=..., defaults=...).
  • Module-level singleton helper: vsync_s3_client.get_env("DATABASE_URL") opens (and caches) a default instance on first call — convenience for scripts. Long-running apps should hold the handle explicitly.
  • No asset_path(). If an SDK demands a filesystem path (GCP GOOGLE_APPLICATION_CREDENTIALS, OpenSSL cert file, JVM keystore), write the bytes yourself:
    python
    import tempfile, os
    bytes_ = v.get_as_content("gcp-sa.json")
    tf = tempfile.NamedTemporaryFile(delete=False, mode="wb")
    tf.write(bytes_); tf.close()
    os.chmod(tf.name, 0o600)
    os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = tf.name
    Three lines is cleaner than a lib that has to manage tempfile lifecycle, perms, /dev/shm preference, and SIGKILL leak warnings forever.

4.2 TypeScript

typescript
import { open, openWith } from "@muthuishere/vsync-s3-client";

// Two ways to open
const v = await open();                                      // env — VSYNC_CONFIG + VSYNC_PASSPHRASE
const v = await openWith({ config, passphrase, defaults });  // direct strings — from anywhere

// Get scalar env-style values (DHH-style — explicit about what's being fetched)
const dbUrl  = v.getEnv("DATABASE_URL");              // → string | null
const exists = v.hasEnv("STRIPE_KEY");                // → boolean
const src    = v.envSource("DATABASE_URL");           // → "vault" | "env" | "default" | "missing"

// Get binary content (replaces both assetBytes + assetPath)
const saJson = v.getAsContent("service-account.json"); // → Uint8Array; in-memory always

// Generation & explicit-poll
const gen    = v.generation();                        // → number
const remote = await v.remoteGeneration();
const stale  = await v.hasNewVersion();
await v.close();
  • open / openWith / remoteGeneration / hasNewVersion / close are async (network or cleanup). getEnv / hasEnv / envSource / getAsContent / generation are sync (in-memory).
  • Defaults: open({ defaults: { PORT: "8080" } }) or openWith({ config, passphrase, defaults }).
  • No assetPath(). Operators write a tempfile if an SDK needs one:
    typescript
    import { mkdtempSync, writeFileSync } from "node:fs";
    import { join } from "node:path";
    import { tmpdir } from "node:os";
    
    const dir = mkdtempSync(join(tmpdir(), "vsync-"));
    const path = join(dir, "gcp-sa.json");
    writeFileSync(path, v.getAsContent("gcp-sa.json"), { mode: 0o600 });
    process.env.GOOGLE_APPLICATION_CREDENTIALS = path;

4.3 Go

go
import vsync "github.com/muthuishere/vsync/libraries/go"

// Two ways to open
v, err := vsync.Open(ctx)                                          // env
v, err := vsync.OpenWith(ctx, configBlob, passphrase)              // direct strings
defer v.Close()

// Get scalar env-style values
dbURL, ok := v.GetEnv("DATABASE_URL")                              // string, bool
hasIt     := v.HasEnv("STRIPE_KEY")                                // bool
src       := v.EnvSource("DATABASE_URL")                           // vsync.Source enum

// Get binary content
saJSON, err := v.GetAsContent("service-account.json")              // []byte

// Generation & explicit-poll
gen    := v.Generation()                                           // int64
remote, err := v.RemoteGeneration(ctx)                             // int64
stale,  err := v.HasNewVersion(ctx)                                // bool
  • Explicit context.Context on Open, OpenWith, RemoteGeneration, and HasNewVersion (network calls). Pure-memory accessors (GetEnv/HasEnv/EnvSource/GetAsContent/Generation) don't take a context.
  • Defaults: vsync.Open(ctx, vsync.WithDefaults(map[string]string{"PORT": "8080"})) or vsync.OpenWith(ctx, blob, pp, vsync.WithDefaults(…)).
  • Errors are sentinel values for errors.Is matching (§11). No panics.
  • No AssetPath. Materialize yourself:
    go
    bytes, _ := v.GetAsContent("gcp-sa.json")
    dir, _ := os.MkdirTemp("", "vsync-")
    path := filepath.Join(dir, "gcp-sa.json")
    os.WriteFile(path, bytes, 0o600)
    os.Setenv("GOOGLE_APPLICATION_CREDENTIALS", path)

4.4 Java

java
import io.github.muthuishere.vsync.s3client.client.Vsync;
import io.github.muthuishere.vsync.s3client.client.VsyncClient;

// Two ways to open
try (Vsync v = VsyncClient.open()) {                                       // env
try (Vsync v = VsyncClient.openWith(configBlob, passphrase)) {             // direct strings

    String  dbUrl   = v.getEnv("DATABASE_URL");           // → null if missing
    boolean hasIt   = v.hasEnv("STRIPE_KEY");
    var     source  = v.envSource("DATABASE_URL");        // Source enum
    byte[]  saJson  = v.getAsContent("service-account.json");

    long    gen     = v.generation();
    long    remote  = v.remoteGeneration();
    boolean stale   = v.hasNewVersion();
}
  • Implements AutoCloseable for try-with-resources. JDK 17+.
  • remoteGeneration / hasNewVersion block on a single HEAD call; wrap in a CompletableFuture if you need async.
  • Maven coordinate: io.github.muthuishere:vsync-s3-client.
  • No assetPath(). Materialize yourself:
    java
    byte[] bytes = v.getAsContent("gcp-sa.json");
    Path dir = Files.createTempDirectory("vsync-");
    Path path = dir.resolve("gcp-sa.json");
    Files.write(path, bytes);
    Files.setPosixFilePermissions(path, PosixFilePermissions.fromString("rw-------"));
    System.setProperty("GOOGLE_APPLICATION_CREDENTIALS", path.toString());

4.5 Cross-language invariants

These hold in every binding regardless of naming style:

  • Two open paths: open() (reads bootstrap from env vars) and open_with(config, passphrase) (accepts strings directly). Both return the same handle; behavioral parity from then on. Operators choose based on where their config lives — platform secret store → env-direct; custom secrets layer (KMS, Hashicorp Vault, a CI variable) → open_with.
  • One Open → one S3 round trip for the bundle. No retries on success-path. (Caller wraps in a retry loop if they want one.)
  • getEnv/hasEnv/envSource/getAsContent are pure-memory after Open returns. No I/O.
  • envSource is the only honest way to know where a value came from. Log it freely; never log getEnv results (§13).
  • generation is the gen=N integer captured from the manifest at Open time. Monotonically increases per env (bumps on every vsync rotate-passphrase).
  • remote_generation / has_new_version are the explicit poll carve-out from §7's pull-once rule — one HEAD on the manifest, no background thread, no callbacks, no state mutation. Operator calls them on demand (healthcheck, sidecar cron, admin endpoint) and decides whether to trigger a restart. Errors propagate (S3UnreachableError, ManifestNotFoundError); the local generation() is never mutated by polling.
  • No asset_path() / filesystem materialization in the lib. get_as_content(name) returns bytes; operators write a tempfile themselves if their SDK demands a path (3 lines per language, shown in each §4 sub-section). Rationale: the lib has no good defaults for tmpdir choice, mode bits, /dev/shm preference, SIGKILL cleanup; pushing this to the caller keeps the lib's contract minimal and honest.
  • close is best-effort: zero the in-memory plaintext buffer, drop references. Idempotent.

5. The fallback chain — locked order

Every get_env(key) resolves in exactly this order. No reordering, no per-key overrides, no "if env then vault" toggles. The order is part of the contract.

  1. vault[env][key] — the decrypted bundle, scoped to the env from the config blob. Returns → envSource = "vault".
  2. process env[key]os.environ / process.env / os.Getenv at lookup time (not at Open time — env mutations after Open are visible to step 2). Returns → envSource = "env".
  3. defaults[key] — the dict passed at open(defaults=…) time. Returns → envSource = "default".
  4. missing — returns the language-idiomatic null (None / null / ("", false)). source = "missing".

Rationale:

  • Vault wins so operators can override a bad value by re-pushing, not by SSHing to the box.
  • Env beats defaults so a deploy can hot-patch (DATABASE_URL=... on the unit file) without a code change.
  • Defaults are the floor — "if literally nothing else said anything, here's a value."
  • "Missing" is a real, returnable state. Callers decide if missing is fatal; the library doesn't impose that policy.

has(key) returns true iff steps 1–3 would resolve a value. source(key) returns the label of the step that won (without actually returning the value — safe to log).


6. Assets — getAsContent returns bytes, full stop

The bundle carries two kinds of payloads: scalar string values (KVs accessed via getEnv) and binary blobs (JSON keys, certs, etc., inlined via the CLI's --inline-file-suffix).

getAsContent(name) — always in memory, returns the raw bytes. No filesystem, no tempfile, no path. Same lookup as getEnv but for binary payloads:

python
key_bytes = v.get_as_content("service-account.json")    # → bytes
cert_pem  = v.get_as_content("tls/server.crt")          # → bytes

There is no asset_path() / assetPath(). Earlier drafts of this spec exposed a lazy-tempfile materializer for SDKs that demand a filesystem path (GCP GOOGLE_APPLICATION_CREDENTIALS, OpenSSL cert paths, JVM keystores). It's been removed. Rationale:

  • The lib had no good default for tmpdir choice (per-OS), mode bits, /dev/shm preference, SIGKILL cleanup, or cross-platform path quoting. Each was a footgun.
  • Three lines at the call site is cleaner than carrying that machinery forever:
    python
    bytes_ = v.get_as_content("gcp-sa.json")
    with tempfile.NamedTemporaryFile(delete=False, mode="wb") as tf:
        tf.write(bytes_); os.chmod(tf.name, 0o600)
        os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = tf.name
  • The operator controls lifecycle and perms locally — they know better than the lib whether the tempfile should be in /dev/shm, deleted on process exit, or persisted across restarts.

Each §4 sub-section shows the 3-line materialization recipe for its language.


7. Pull-once semantics

One S3 round trip on Open() — manifest fetch + bundle fetch + decrypt. After that, the bundle stays in memory forever.

  • No background refresh thread.
  • No refresh() method that re-pulls the bundle.
  • No filesystem cache. (No "fall back to last-known-good bundle on S3 outage.")
  • No If-Modified-Since background polling.

Pod restart is the refresh mechanism. To pick up a new vault, restart the process. This composes with every orchestrator (Kubernetes rollout, systemd restart, Vercel redeploy, Cloud Run new revision) without the library having to know about any of them.

If the operator wants live-reload, the right place to implement it is one layer up (a sidecar that touches a file → app watches the file → exits → orchestrator restarts), not inside this library.

7.1 The has_new_version carve-out

The pull-once rule covers the bundle — the decrypted vault stays in memory and is never re-fetched. It does NOT prohibit a lightweight, operator-explicit check for whether a newer version exists upstream. That's what remote_generation() and has_new_version() are for (§4):

  • One HEAD on the manifest per call — no background thread, no callback, no automatic anything.
  • Returns a fresh number read from the remote manifest's meta.gen field; doesn't mutate the lib's generation() (the local gen stays whatever Open() captured).
  • Designed for healthcheck endpoints and sidecar crons — caller decides what to do with the answer. Restart is still the only way to actually pick up new secrets.
  • Fail-loud on the network pathS3UnreachableError / ManifestNotFoundError propagate. Apps usually handle by treating "unknown" as "don't restart for now."

The boundary: this method only tells you the answer to a question. It never changes the bundle in memory. The orchestrator (k8s, systemd, your platform) is still responsible for actually rolling the process when the answer is "yes, stale."


8. Bootstrap failure policy — fail loud

If Open() cannot produce a coherent in-memory vault, it raises. It does NOT silently degrade to "env vars only."

Failures that raise from Open:

  • S3 unreachable (network, DNS, TLS) → S3UnreachableError
  • IAM key rejected (403) → S3UnreachableError with a permission hint
  • Bucket exists but <prefix><env>/latest.manifest is missing → ManifestNotFoundError
  • Manifest fetched but bundle fetch 404s → BundleCorruptError (manifest pointed at a non-existent object — the bucket is in a torn state)
  • Decrypt: GCM tag mismatch → WrongPassphraseError
  • Magic byte mismatch on the bundle → BundleCorruptError
  • Unknown RQE1/RQEM0001 version → UnsupportedSpecVersionError

Rationale: a process that boots with "env vars only because S3 was down" is a process running with the wrong secrets, possibly a stale subset of the real config, and is harder to debug than a process that refused to boot. Healthcheck failure → orchestrator restart → eventual consistency wins.

The fallback chain (§5) applies only to per-key misses inside an already-open vault. Never to bootstrap. This is the most important sentence in this spec.


9. Trust Boundaries and Honest Limits

The two-variable split (VSYNC_CONFIG + VSYNC_PASSPHRASE) is a separation-of-leak-channels design, not multi-factor authentication. State the boundary plainly so operators don't model the wrong threat.

Protects against (asymmetric leakage):

  • Bucket misconfiguration. A world-readable bucket leaks the encrypted bundle. Without the passphrase, the bundle is ciphertext.
  • Infrastructure-repo leak. A leaked Terraform / Helm chart that contains VSYNC_CONFIG leaks the S3 location and IAM key but not the passphrase (kept in the platform secret store).
  • Partial log capture. A logger that prints process.env minus a denylist may catch one variable; a logger that prints /etc/myapp/env may capture the other. Splitting reduces the chance one log dump has both.
  • Operator error inside one system. Someone pastes VSYNC_CONFIG into a Slack channel; the passphrase lives elsewhere.

Does NOT protect against (the process is its own attack surface):

  • Full process compromise. Anything that can read /proc/<pid>/environ has both halves. Anything that can attach gdb to the process has the decrypted vault.
  • CI log dumps that print all env vars (env, printenv, set -x near a curl). If the runner logs both, both are gone.
  • Sentry / Datadog / Honeycomb auto-capturing process.env on a crash. Same channel.
  • A malicious or compromised dependency inside the application. The library hands plaintext to the caller; the dependency runs in the same process.
  • Backups that copy the host filesystem (/run/secrets/...) and the platform secret-store dump together. Both halves on one backup tape = no split.

Explicit anti-claims:

  • This is not MFA. A second factor would be something the operator presents at boot (hardware token), not a second env var that lives next to the first.
  • This is not end-to-end encryption from the operator to the application. The passphrase is in the platform secret store; the platform admin can read it.
  • "Defense in depth" describes this accurately. "Zero trust" does not.

Document this section in the README of each language binding verbatim. The worst failure mode is an operator who believes the wrong story.


10. Asset materialization — removed

Earlier drafts of this spec defined a lazy-tempfile asset_path() accessor; v0.11+ dropped it. See §6 for the replacement (get_as_content returns bytes; operator materializes if needed). Section retained as a redirect so external references to §10 don't 404.


11. Error taxonomy

One conceptual hierarchy, three language renderings.

ErrorMeaningCaller's recourse
ConfigMissingErrorVSYNC_CONFIG / VSYNC_PASSPHRASE unset; or magic prefix wrongfix the deploy config
ConfigUnsupportedVersionErrorinner JSON v is newer than this library understandsupgrade the library
S3UnreachableErrornetwork, DNS, TLS, or HTTP 4xx/5xx on the fetchcheck S3 connectivity & IAM
ManifestNotFoundErrorbucket reachable, <prefix><env>/latest.manifest absentrun vsync push <env> once
WrongPassphraseErrorGCM auth tag rejected the passphraserotate / re-key
BundleCorruptErrormagic byte mismatch, truncated read, manifest→bundle pointer danglingre-push from the CLI
UnsupportedSpecVersionErrorunknown RQE1 / RQEM0001 envelope versionupgrade the library

Latest-only policy (decision I): the library reads the current envelope version and nothing else. Pre-1.0, when an envelope bumps, the library bumps with it. No multi-version readers in v0.x.

Language renderings:

python
# Python — exception subclasses of a single root
class VSyncError(Exception): ...
class ConfigMissingError(VSyncError): ...
class WrongPassphraseError(VSyncError): ...
# ... one subclass per row above
typescript
// TypeScript — typed subclasses of a single root
export class VSyncError extends Error { readonly code: string; }
export class ConfigMissingError extends VSyncError { readonly code = "VSYNC_CONFIG_MISSING"; }
export class WrongPassphraseError extends VSyncError { readonly code = "VSYNC_WRONG_PASSPHRASE"; }
go
// Go — sentinel errors for errors.Is
var (
    ErrConfigMissing            = errors.New("vsync: config missing")
    ErrConfigUnsupportedVersion = errors.New("vsync: config version unsupported")
    ErrS3Unreachable            = errors.New("vsync: s3 unreachable")
    ErrManifestNotFound         = errors.New("vsync: manifest not found")
    ErrWrongPassphrase          = errors.New("vsync: wrong passphrase")
    ErrBundleCorrupt            = errors.New("vsync: bundle corrupt")
    ErrUnsupportedSpecVersion   = errors.New("vsync: unsupported spec version")
)

Idiomatic per language; the set is identical. Cross-language conformance test vectors (same bad bundle → same labeled error in every binding) are deferred to v0.11.


12. Redaction policy

The handle object must not leak secret values through accidental serialization.

  • Python __repr__ / __str__: <vsync:redacted gen=N env=prod>. No keys, no values, no S3 endpoint.
  • TypeScript toJSON() and the default util.inspect hook: "<vsync:redacted>". Property enumeration on the handle exposes only generation (a number).
  • Go String() (i.e., fmt.Stringer): "<vsync:redacted>". Fields holding the plaintext map are unexported.

envSource(key) is safe to log. hasEnv(key) is safe to log. generation() is safe to log. getEnv(key) / getAsContent(name) results are never safe to log; document that prominently in every language README.

The library does not install global panic handlers, monkey-patch console.log, or filter Sentry breadcrumbs. Application-level observability hygiene is the caller's job. We just make sure the handle itself doesn't betray the caller.


13. File-permissions policy on _FILE

When VSYNC_CONFIG_FILE or VSYNC_PASSPHRASE_FILE is set, the library stats the path before reading.

ModeBehavior
0600 / 0400 (owner-only)read, no warning
0644 / 0640 (group/world readable)read, but log a warning to stderr: vsync: <path> is world/group-readable (mode 0644); narrow to 0600
0666 / 0777 (world-writable)refuse to read → raise ConfigMissingError with mode in the message
ENOENTraise ConfigMissingError (path set but missing)
EACCESraise ConfigMissingError with permission hint

Matches the libpq ~/.pgpass pattern. 0666 is almost certainly a mistake (someone chmod 666'd for a debugging session and forgot); failing closed is friendlier than silently reading a file anyone on the box can rewrite.

On Windows, stat mode bits are meaningless; the library skips the check and logs vsync: file-permission check skipped (Windows).


14. Layout & registries

Per decision H, the three bindings live in one repo:

secret-lib/
  libraries/
    python/        → publishes vsync-s3-client to PyPI
    typescript/    → publishes @muthuishere/vsync-s3-client to npm
    go/            → module github.com/muthuishere/vsync-s3-client-go

Each subtree is its own package with its own version, its own tests, and its own release cadence. The shared documentation (this spec) is the contract; no shared code crosses the boundary.

Per decision K: Python first (publishable artifact lands first), then TypeScript, then Go. The Python README + this spec are the reference for the other two bindings.


15. What's intentionally out of scope

  • CLI subcommands to mint the config blob and rotate the passphrasevsync runtime-token and vsync rotate-passphrase are in v0.10.
  • Cross-language conformance test vectors (same bad bundle → same labeled error in every binding) — v0.11.
  • RQE1 / RQEM0001 envelope internals — locked by [v0.2 §3]. Do not re-spec here.
  • Audit log writes from the client — the library never writes to S3, and audit format is owned by v0.4.
  • Backwards-compat shims for pre-1.0 envelope versions — explicitly dropped (decision I). v0.x = latest-only, fail-loud.
  • Envelope-type byte / format multiplexing inside RQE1 — explicitly dropped (decision F).
  • Multi-env in one handle — open multiple clients if you need multiple envs.
  • STS / instance-profile auth instead of long-lived IAM keys — parked for v2 of the config blob (v: 2 would add roleArn + webIdentityTokenFile).
  • KMS-wrapped data-encryption-key flow — parked for v3. The passphrase model is sufficient for v0.x; KMS adds a third trust party we don't need today.
  • rotate-iam-key from the CLI side — out of scope. Operators rotate IAM at the cloud console and re-mint the config blob.
  • Live reload / refresh() / file-watch — see §7. Restart the process.

Released under the MIT License.