> ## Documentation Index
> Fetch the complete documentation index at: https://docs.hexr.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Credential Exchange

> Three-tier credential cache delivers sub-millisecond cloud credentials. JWT-SVID → OPA → STS exchange, cached from memory to Valkey to live refresh.

## Overview

When you call `hexr_tool("aws_s3")`, Hexr returns an authenticated boto3 S3 client — without any API keys in your code. Behind the scenes, a three-tier caching system makes this near-instantaneous.

```python theme={null}
# This is all you write:
s3 = hexr_tool("aws_s3")
bucket = s3.list_buckets()

# What actually happens:
# 1. Check in-memory cache (L1) → ~0.001ms
# 2. Check Valkey cache (L2) → ~1-3ms  
# 3. Full credential exchange (L3) → ~50-200ms
#    JWT-SVID → OPA check → STS AssumeRoleWithWebIdentity
```

***

## Three-Tier Cache Architecture

<Steps>
  <Step title="L1: In-Memory (Process-Local)" icon="bolt">
    **\~0.001ms latency.** Credentials live in the Python process's memory (`ContextVar`-based).
    TTL = credential expiry minus 10 minutes. Dies when the process dies.
  </Step>

  <Step title="L2: Valkey (Distributed)" icon="database">
    **\~1-3ms latency.** Shared across all pods in the cluster (3-node HA). Key format:
    `cred:{spiffe-id}:{service}:{region}`. If agent A already fetched S3 credentials,
    agent B can use the cached result.
  </Step>

  <Step title="L3: Credential Exchange (Full Round-Trip)" icon="right-left">
    **\~50-200ms latency.** Agent → Envoy (mTLS) → Credential Injector → OPA policy check →
    Cloud STS `AssumeRoleWithWebIdentity`. Issues temporary credentials (15-60 min TTL).
  </Step>
</Steps>

<Note>
  On a cache **hit**, the call returns in microseconds (L1) or low milliseconds (L2).
  The full 50-200ms exchange only happens on first access or after credential expiry.
</Note>

***

## Exchange Flow

What happens when both L1 and L2 cache miss — the full credential exchange:

<Steps>
  <Step title="hexr_tool('aws_s3') called" icon="code">
    Your agent calls `hexr_tool("aws_s3")`. SDK checks L1 (memory) → miss. Checks L2 (Valkey) → miss.
  </Step>

  <Step title="Request sent via Envoy" icon="arrow-right">
    Agent sends `POST /exchange {service: "aws_s3"}` to the Envoy sidecar (localhost).
    Envoy adds the X.509-SVID as client certificate and forwards over **mTLS** to the
    Credential Injector in `hexr-system`.
  </Step>

  <Step title="JWT-SVID verification" icon="certificate">
    The Credential Injector verifies the agent's JWT-SVID via the SPIRE Workload API.
    This confirms the caller's SPIFFE identity is legitimate.
  </Step>

  <Step title="OPA policy check" icon="shield-check">
    CI queries OPA: `{spiffe_id, service: "aws_s3", tenant: "acme-corp"}`.
    OPA evaluates the Rego policy and returns **ALLOW** or **DENY**.
  </Step>

  <Step title="Cloud STS exchange" icon="cloud">
    CI calls `AssumeRoleWithWebIdentity` on AWS STS, presenting the JWT-SVID as the web
    identity token. AWS trusts Hexr's OIDC endpoint and returns temporary credentials.
  </Step>

  <Step title="Credentials cached + client returned" icon="key">
    `{AccessKeyId, SecretAccessKey, SessionToken}` with 15-min TTL flows back through Envoy
    to the agent. Stored in **L1 + L2** cache. SDK creates and returns an authenticated
    `boto3.client('s3')`.
  </Step>
</Steps>

***

## Supported Cloud Providers

<CardGroup cols={3}>
  <Card title="AWS" icon="aws">
    **Exchange:** JWT-SVID → STS `AssumeRoleWithWebIdentity`

    **Services:** S3, EC2, DynamoDB, SQS, Lambda, Bedrock, and any AWS SDK service.

    **Credential TTL:** 15 minutes (configurable up to 12 hours)
  </Card>

  <Card title="GCP" icon="google">
    **Exchange:** JWT-SVID → Workload Identity Federation → Service Account token

    **Services:** BigQuery, Cloud Storage, Vertex AI, Pub/Sub, and any Google Cloud API.

    **Credential TTL:** 1 hour
  </Card>

  <Card title="Azure" icon="microsoft">
    **Exchange:** JWT-SVID → Federated Token → Managed Identity token

    **Services:** Blob Storage, Cosmos DB, Azure OpenAI, and any Azure SDK service.

    **Credential TTL:** 1 hour
  </Card>
</CardGroup>

***

## Multi-Cloud in One Agent

An agent can use tools from multiple clouds simultaneously:

```python theme={null}
@hexr_agent(
    name="multi-cloud-analyst",
    tenant="acme-corp",
    resources=["aws_s3", "gcp_bigquery", "azure_storage"]
)
def analyze():
    # Each call goes through the same credential exchange
    # but targets different cloud STSes
    s3 = hexr_tool("aws_s3")              # → AWS STS
    bq = hexr_tool("gcp_bigquery")        # → GCP WIF
    blob = hexr_tool("azure_storage")     # → Azure Federated Token
    
    # All three clients are authenticated and ready
    data = bq.query("SELECT * FROM dataset.table")
    s3.put_object(Bucket="results", Key="output.json", Body=data)
```

***

## OPA Policy Enforcement

Before any credential exchange, OPA validates the request:

```rego theme={null}
# Example policy: only allow S3 access for data-pipeline agents
package hexr.credentials

default allow = false

allow {
    input.service == "aws_s3"
    startswith(input.spiffe_id, "spiffe://hexr.cloud/agent/acme-corp/data-pipeline")
}

# Deny all EC2 access
deny {
    input.service == "aws_ec2"
}
```

Policies are distributed via Kubernetes ConfigMaps and reload within 30 seconds.

***

## Proactive Refresh

A background daemon proactively refreshes credentials before they expire:

| Property           | Value                                              |
| ------------------ | -------------------------------------------------- |
| **Check interval** | Every 60 seconds                                   |
| **Refresh buffer** | 10 minutes before expiry                           |
| **Behavior**       | Silent background refresh — no disruption to agent |

```
Timeline:
  T=0:00  → Credential issued (TTL: 15 min)
  T=4:00  → Background check: 11 min remaining (OK)
  T=5:00  → Background check: 10 min remaining (REFRESH!)
  T=5:01  → New credential fetched, cached in L1 + L2
  T=15:00 → Old credential would have expired (already replaced)
```

This means agents **never see credential expiry errors** during normal operation.

***

## Observability

Every cache lookup and exchange emits OpenTelemetry spans:

```
Span: hexr.cache.lookup
  ├── tier: "L1" | "L2" | "L3"
  ├── hit: true | false
  ├── service: "aws_s3"
  └── duration_ms: 0.001 | 2.3 | 150

Span: hexr.credential.exchange
  ├── provider: "aws" | "gcp" | "azure"
  ├── service: "aws_s3"
  ├── spiffe_id: "spiffe://hexr.cloud/agent/..."
  └── duration_ms: 150
```

Grafana dashboards show cache hit rates, exchange latencies, and credential refresh patterns in real-time.