---
title: "Provenance and archiving: an agent run as evidence"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Provenance and archiving: an agent run as evidence}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include=FALSE}
knitr::opts_chunk$set(
  collapse = TRUE, comment = "#>",
  eval = identical(tolower(Sys.getenv("LLMRAGENT_RUN_VIGNETTES", "false")), "true")
)
```

Each LLMRagent run records what was asked, which model answered, what it
cost, and whether anything failed. This vignette shows how that record is
kept and how to write an archive a reader can open months later.

```{r setup}
library(LLMRagent)
cfg <- LLMR::llm_config("groq", "openai/gpt-oss-20b", temperature = 0.7)
```

## One object, five views

Anything you run, whether a single chat, a deliberation, or an experiment,
converts to one object through `as_agent_run()`. That object answers five
questions, and you ask each by naming a level:

```{r run}
panel <- list(
  agent("Optimist", cfg, persona = "You see opportunity first."),
  agent("Skeptic",  cfg, persona = "You probe for hidden costs.")
)
debate <- deliberate(panel, "Should the department adopt a four-day week?",
                     rounds = 1)

run <- as_agent_run(debate)
```

The levels are deliberately plain tibbles, so the rest of your analysis is
ordinary data work:

```{r levels}
tibble::as_tibble(run, level = "utterance")  # who said what, in order
tibble::as_tibble(run, level = "call")       # one row per model call
tibble::as_tibble(run, level = "event")      # the full event graph
tibble::as_tibble(run, level = "tool")       # every tool invocation
tibble::as_tibble(run, level = "state")      # each agent's memory at the end
```

The `utterance` level is the transcript: the thing you usually read. The `call`
level is the instrument log: every model call as a row, with the served model
version, the token counts, a success flag, and a `request_hash` that uniquely
identifies the exact request that produced it. The agent recorded all of this
while it answered.

## Diagnostics for a run

`diagnostics()` reads the call level and returns a one-row summary. The numbers
come from the same place the wider 'LLMR' ecosystem reads them, so they agree
across tools rather than reporting two different figures:

```{r diag}
diagnostics(run)
```

A high failure count, an unexpected model version, a token bill larger than the
work warrants -- these surface here before they surface in your conclusions.

## Reporting within scope

`report()` writes a short methods paragraph and, importantly, refuses to
overstate. An agent run is model-conditioned output, not a sample from a human
population, and `report()` says so. Claim labels can describe an instrument
pilot, theory probe, or coding exercise, but none removes that scope condition.

```{r report}
report(run)
```

## The manifest: identity without the transcript

Two runs are "the same study" when their design is the same: same personas,
same tools, same turn policy, same model and settings. They are emphatically
not the same when any of those change, even if the transcripts happen to read
alike. `agent_manifest()` captures the declared design as a set of content
hashes, keeping outcome (the transcript) separate from recorded apparatus:

```{r manifest}
m <- agent_manifest(run)
m$manifest_hash    # changes when a hashed declared component changes
```

A reworded persona, a changed tool body, or a different temperature moves the
hash. Values captured in a tool closure are not part of that identity; record
such state in the design when it matters. This is the same split
'LLMR' makes between a request's identity and its content, lifted to the level
of a whole study.

## Archiving: seal the evidence

`archive_agent_study()` writes a self-describing directory. It contains the
transcript, event graph, manifest, artifacts, drafted methods note, a data-only
RDS projection, and a checksum file that seals the whole thing. The centerpiece
is `calls.jsonl`, which is a genuine 'LLMR' audit log that 'LLMR' can read.

```{r archive}
dir <- file.path(tempdir(), "four_day_week_study")
archive_agent_study(run, dir)
list.files(dir)
```

The archive never serializes live agents, caller closures, tool functions, or
model configurations. The archive keeps the original hashes. They are computed over the
original text, so they match the live run even if you redact the stored copy for
privacy. And the request hash in `calls.jsonl` joins to the `request_hash` you
saw at the call level. That join is the integrity check: it lets a skeptic
confirm that the archived log describes the run it claims to describe.

```{r join}
calls   <- tibble::as_tibble(run, level = "call")
archived <- LLMR::llm_log_read(file.path(dir, "calls.jsonl"))

# every archived call is one of the calls the run actually made
all(archived$manifest$request_hash %in% calls$request_hash)
```

If you keep a live audit log during the run (call
`LLMR::llm_log_enable("calls.jsonl")` before you start), the archive copies that
real file, filtered to this run's calls, rather than reconstructing it, and so
preserves the log's own record hashes.

## Redaction without lying

Survey responses and interview text often cannot be stored verbatim. `redact`
scrubs the free-text columns of the stored copy, while the hashes stay computed
over the original. The archive remains internally consistent: it does not claim
a request body it did not send, and the join still holds.

```{r redact}
archive_agent_study(run, file.path(tempdir(), "redacted_study"),
                    redact = c("four-day", "salary"))
```

For privacy that must omit the request entirely, pass `include_messages = FALSE`:
the archive then carries metadata and hashes but no message text at all,
including in `run.rds`. A nonempty destination is refused unless
`overwrite = TRUE`.

## Why archive

A reviewer asks which model you used and whether you cherry-picked. A
replication team, a year on, has only your files. The archive preserves the
recorded design, transcript, calls, and hashes that join them. It makes a
model-generated result something one can inspect, verify, and defend; it is not
an executable checkpoint.