Analysis Methodology

Complete transparency on how we analyzed 1,250 interviews from the Anthropic Interviewer dataset.

01

Data Source

Dataset

Anthropic/AnthropicInterviewer on HuggingFace (MIT License)

Sample Composition
1,065
Workforce
134
Creatives
51
Scientists
Interview Format

Each interview is a full conversation transcript between an AI interviewer and a human participant about their experiences with AI tools at work. Interviews vary in length from ~500 to ~3000 words.

02

LLM-Based Structured Analysis

All 1,250 interviews were analyzed using GPT-4o-mini with structured outputs, extracting consistent fields across 47 dimensions per interview.

03

Analysis Schema (47 Dimensions)

AI Conceptualization

How participants think about and describe AI

primary_metaphorrelationship_framingagency_attributionanthropomorphization_level
Emotional State

Emotional responses during the interview

primary_emotions[]emotional_intensityemotional_valenceemotional_triggers[]
Task Boundaries

What tasks are delegated vs protected from AI

high_ai_tasks[]protected_tasks[]boundary_logic
Workplace Context

Social and organizational dynamics

workplace_culturepeer_influencedisclosure_patternclient_considerations
Trust Profile

Trust levels, what builds/destroys trust, verification behaviors

overall_trusttrust_drivers[]distrust_drivers[]verification_habittrust_trajectory
Identity Dimension

How AI affects professional identity and sense of meaning

professional_identity_strengthidentity_threat_levelmeaning_sourceskill_anxietyexpertise_relationshipmeaning_disruption
Adaptation Journey

The arc of AI adoption over time

experience_leveladoption_trajectorypivotal_moments[]learning_approachfuture_intentions
Ethical Dimension

Ethical reasoning and moral responses to AI use

ethical_concerns_presentprimary_ethical_frameguilt_or_shamemoral_language[]
Tensions

Internal conflicts about AI (multiple per interview)

tension_namemanifestationresolution_status
Key Quotes

Verbatim quotes with thematic tagging (3-5 per interview)

quotethemesignificance
Emergent Themes

Open-ended theme discovery

theme_nameevidence
Summary

High-level characterization

one_sentence_summaryarchetyperesearch_valuestandout_insight
04

Struggle Score Calculation

The "struggle score" is a composite index we created to quantify psychological difficulty with AI adoption. It is NOT a validated psychological measure—it's an exploratory metric derived from the structured analysis.

Formula
struggle_score =
  identity_threat (0-3) +
  skill_anxiety (0-3) +
  meaning_disruption (0-1) +
  guilt_shame (0-1) +
  unresolved_tensions (0-1) +
  ethical_concerns (0-1)
Identity Threat

none=0, low=1, moderate=2, high=3

Skill Anxiety

none=0, low=1, moderate=2, high=3

Meaning Disruption

false=0, true=1

Guilt/Shame

false=0, true=1

Unresolved Tensions

If any tension has resolution_status="unresolved", score=1

Ethical Concerns

false=0, true=1

Score Range

0 (no struggle) to 10 (maximum struggle). In practice, scores ranged from 0 to 8.

05

Exploratory Data Analysis

After structured extraction, we ran comprehensive EDA across 1,250 analysis results:

  • Cross-tabulations (group × trust, group × identity threat, trust × identity threat)
  • Segment analysis (high threat, high anxiety, guilt expressers, high trust)
  • Keyword frequency analysis (trust drivers, distrust drivers, moral language)
  • Archetype distribution and group patterns
  • Tension pattern analysis (resolution rates, common tensions by group)
  • Composite struggle score calculation and distribution
06

Limitations & Caveats

LLM Interpretation Bias

The structured analysis is performed by an LLM (GPT-4o-mini), which introduces potential interpretation biases. Different models or prompts might yield different results.

Sample Size Variation

The scientists group (n=51) is significantly smaller than workforce (n=1,065) or creatives (n=134). Percentages for scientists should be interpreted with caution.

Composite Scores Not Validated

The "struggle score" is an exploratory composite index we created, not a validated psychological measure. It should be treated as directional, not definitive.

Self-Report Data

All data comes from self-reported interviews. Participants may not accurately represent their actual behaviors or feelings.

Point-in-Time Snapshot

These interviews were conducted at a specific moment in AI development. Attitudes and behaviors may have changed since data collection.

07

Reproducibility

We encourage independent verification of our findings.

Public Dataset

The Anthropic Interviewer dataset is publicly available on HuggingFace under MIT license.

Analysis Code

Our analysis scripts and Pydantic schema are available on request. Contact: research@playbookatlas.com

Full Results

The complete JSON output of all 1,250 structured analyses is available on request for independent verification.