Jamf, Kandji, and Santa Test Plan

End-to-end validation runbook for testing thoth deployment through Jamf or Kandji with Santa trust enforcement.

Use this runbook to test your enterprise rollout path before broad production deployment.

Scope

This plan validates:

Thoth deployment through Jamf Pro.
Thoth deployment through Kandji.
Santa trust policy enforcement for thoth and thothctl.

Test environment

Prepare:

2-4 dedicated pilot Macs (macOS 13+ and macOS 14+ if possible).
One Jamf-managed device.
One Kandji-managed device.
One optional Santa-enforced validation device (recommended).
Local admin shell access for validation commands.
Admin access to Jamf Pro and Kandji consoles.

Use a single approved release tag for all tests (example: v0.3.4).

Required release assets

From atensecurity/thoth release v<version>, download:

thoth-macos-universal.pkg
checksums.sha256
santa-metadata.json
signing-metadata.json
provenance.json
sbom.cdx.json
sbom.spdx.json

Validate integrity before deployment:

sha256sum -c checksums.sha256
jq -e '.provenanceType == "release-artifact-provenance"' provenance.json
jq -e '.bomFormat == "CycloneDX"' sbom.cdx.json
jq -e '.spdxVersion | startswith("SPDX-")' sbom.spdx.json

Phase 1: Jamf validation

Follow Jamf macOS Runbook for policy setup.

Execute:

Scope policies to Jamf test smart group only.
Run policy sequence: prerequisites -> config -> setup.
Confirm package install and script completion in Jamf logs.

On the endpoint:

thoth --version
thothctl --version
thoth health --json
thoth status
test -f "/Library/Application Support/Thoth/thoth-config.json"

Pass criteria:

All commands succeed.
Version matches target release.
thoth health --json reports healthy registration.
Claude config is governed.

Phase 2: Kandji validation

Follow Kandji macOS Runbook for Custom App and Script setup.

Execute:

Deploy notarized PKG to Kandji test assignment.
Deploy managed thoth-config.json.
Run recurring setup script for Claude config governance.

On the endpoint:

thoth --version
thothctl --version
thoth health --json
thoth status
test -f "/Library/Application Support/Thoth/thoth-config.json"

Pass criteria:

Install succeeds without manual intervention.
Managed config remains present after reboot.
Health and status checks are stable across at least two check-ins.

Phase 3: Santa trust validation

Install Santa on test devices through your normal endpoint tooling and ensure santa-cli is available.

Apply trust rules from release metadata:

TEAM_ID="$(jq -r '.teamId' santa-metadata.json)"
THOTH_SHA="$(awk '$2=="thoth-macos-universal" {print $1}' checksums.sha256)"
THOTHCTL_SHA="$(awk '$2=="thothctl-macos-universal" {print $1}' checksums.sha256)"
 
sudo santa-cli rule --add --teamid "$TEAM_ID" --policy ALLOWLIST
sudo santa-cli rule --add --sha256 "$THOTH_SHA" --policy ALLOWLIST
sudo santa-cli rule --add --sha256 "$THOTHCTL_SHA" --policy ALLOWLIST

Verify rules:

sudo santa-cli rule --check --teamid "$TEAM_ID"
sudo santa-cli rule --check --sha256 "$THOTH_SHA"
sudo santa-cli rule --check --sha256 "$THOTHCTL_SHA"

Verify binary identity:

codesign -dv --verbose=4 "$(command -v thoth)" 2>&1 | grep -E 'TeamIdentifier|Authority='
codesign -dv --verbose=4 "$(command -v thothctl)" 2>&1 | grep -E 'TeamIdentifier|Authority='

Optional negative test (Santa enforcement device):

cp "$(command -v thoth)" /tmp/thoth-unsigned
codesign --remove-signature /tmp/thoth-unsigned
/tmp/thoth-unsigned --version

Expected:

In Santa lock mode: unsigned binary execution is blocked.
In monitor mode: execution may proceed but event is logged.

Evidence capture checklist

Capture and store:

Jamf policy execution screenshots/log export.
Kandji Custom App and Script execution screenshots/log export.
thoth --version, thothctl --version, thoth health --json, thoth status output.
Santa rule query outputs and enforcement logs.
codesign identity output for both binaries.

Rollout gates

Promote only when all three gates pass:

Test: 1-5 devices per management path.
Pilot: 5-15% of scoped fleet.
Production: full rollout after 24h stable pilot window.

Stop rollout if:

registration failures exceed your SLO threshold,
Santa blocks signed release binaries unexpectedly,
managed config drift cannot self-heal.

Jamf, Kandji, and Santa Test Plan

On this page