Commonwealth-Online-Public/docs/animation-sync-analysis.md

# Animation Sync Analysis: Commonwealth Online vs TiltedEvolution

> **Resolution update (2026-06-05):** The "variable names may be wrong" hypothesis below is
> now confirmed and fixed. The real FO4 humanoid graph variable names + types were extracted
> from `MTBehavior.hkx` (unpacked in the `F4-Animation-Research` repo) and applied to the code.
> Authoritative set actually driven by F4T:
> - `Speed`, `SpeedSmoothed` (float / `VARIABLE_TYPE_REAL`)
> - `IsSprinting`, `bInJumpState` (bool / `VARIABLE_TYPE_BOOL`)
> - `iIsInSneak` (int / `VARIABLE_TYPE_INT32`, 0 = standing, 1 = sneaking)
>
> Earlier guesses (`direction`, `isMoving`, `isSneaking`, `isCrouching`, `speedSampled`,
> `speedDamped`, `SpeedSampled`) do **not** exist on this graph and silently failed. `Direction`
> exists but is movement-relative-to-facing (not world heading), so it is intentionally left
> unwritten to avoid strafing artifacts. See `docs/dev-log.md` (2026-06-05) for the change set.

## Problem Statement

Your Commonwealth Online animations aren't working on proxy actors. You have animation sync code in `F4TProxyAnimationSync.cpp` attempting to set graph variables by name (`Speed`, `IsSprinting`, etc.), but the variables aren't being applied or synced correctly.

---

## High-Level Comparison

| Aspect | **TiltedEvolution (Skyrim SE)** | **F4T (Fallout 4)** |
|--------|--------|--------|
| **Sync strategy** | Snapshot entire animation graph variable **cache** (by index) + replay `TESActionData` events | Write individual graph variables **by string name** every frame |
| **Remote actor type** | Real `Actor` instances with `SetRemote(true)` flag; AI suppressed | CK-placed dummy actors with custom animation handling? (needs clarification) |
| **Variable storage** | Pre-computed `AnimationGraphDescriptor` tables (per graph type); bulk copy via `LoadAnimationVariables` | Runtime string → variable lookup via `SetGraphVariableFloat`/`SetGraphVariableBool` |
| **Timing** | Action replay **queue** + interpolation of position/rotation; graph vars updated **once per action start** or during interpolation | Every frame (`ApplyProxyAnimationFromRemoteState`) |
| **Graph safety** | Locks (`BSScopedLock`) on `BSAnimationGraphManager` | No explicit locking visible; reads graph manager once per call |
| **Transition handling** | `TESActionData` + actor state flags (`flags1`, `flags2`) drive graph FSM | Continuous graph variable writes; no explicit action replay |
| **Sneak/Jump/Weapon** | Synced as graph booleans in descriptor | Disabled (`kEnableProxySneakSync`, etc. = false) |

---

## TiltedEvolution's Animation Flow

### 1. **Local Capture** (100 ms intervals)

```
PlayerCharacter.Update()
  ↓
CharacterService::RunLocalUpdates()
  ↓
TESObjectREFR::SaveAnimationVariables(
    animationVariables []  ← direct copy from engine graph cache
  )
  ↓
ClientReferencesMoveRequest {
    position, rotation, direction,
    AnimationVariables { bools[], floats[], ints[] }
}
  ↓ Network → Server → Other clients
```

### 2. **Remote Application** (Continuous)

#### Path A: Interpolation (movement + locomotion)
```
InterpolationSystem::Update()
  ↓
actor→ForcePosition(interpolated.position)
actor→LoadAnimationVariables(interpolated.Variables)  ← Write cache by index
actor→currentProcess→middleProcess→direction = direction
actor→SetRotation(...)
  ↓ Result: Walking/running animation driven by replicated graph state
```

#### Path B: Action Replay (transitions)
```
RemoteAnimationComponent receives ActionEvent {
    Type, EventName, State1, State2,
    Variables, idleForm, target, ...
}
  ↓
AnimationSystem::Update()
  ↓
actor→actorState.flags1 = State1
actor→actorState.flags2 = State2
actor→LoadAnimationVariables(Variables)
  ↓
ActorMediator::ForceAction(TESActionData(...))
  ↓ Result: Combat idle, attack, emote, or other discrete action
```

### 3. **Key Implementation Details**

#### Graph Variable Cache Management
- **Descriptor tables** stored in `AnimationGraphDescriptor_Master_Behavior.cpp`
- Each graph (humanoid, creature, werewolf, vampire lord) has its own descriptor
- Descriptor lists which **indices** to sync (not names)
- At runtime, `BSAnimationGraphManager::GetDescriptorKey()` identifies the graph type
- `LoadAnimationVariables()` writes directly to `behaviorGraph→animationVariables→data[index]`

#### Thread Safety
```cpp
BSScopedLock _{pManager→lock};
// Copy/write animationVariables→data[] within lock
```

#### Which Variables?
For humanoid (player):
- **Booleans**: `kIsSprinting`, `kIsSneaking`, `kbInMoveState`, `kbMotionDriven`, ...
- **Floats**: `kSpeed`, `kSpeedSampled`, `kSpeedDamped`, `kDirection`, `kSpeedWalk`, `kSpeedRun`, ...
- **Ints**: `kiRightHandEquipped`, `kiLeftHandType`, `kiIsInSneak`, ...

---

## F4T's Current Approach

### Current Code (F4TProxyAnimationSync.cpp)

```cpp
// Desired state built from remote player telemetry
struct DesiredProxyAnimationState {
    bool isMoving, isSprinting, isSneaking, isJumping, weaponDrawn;
    float graphSpeed, direction;
};

// Applied every frame
void ApplyProxyAnimationFromRemoteState(...) {
    // TrySetGraphFloat("Speed", desired.graphSpeed)
    // TrySetGraphBool("IsSprinting", desired.isSprinting)
    // ...
}
```

### Issues This May Face

1. **String lookup overhead** — Every frame calls `BSFixedString { a_variableName }` and searches graph for that variable name. This is slower than indexed access.

2. **Missing manager checks** — Code has a guard (`CanWriteProxyGraph`) but may not handle all failure modes (manager destruction, graph reload, etc.).

3. **Graph variable name mismatches** — If the actual FO4 animation graph uses different variable names than expected, `SetGraphVariableFloat` returns false silently. The code logs it once but continues.

4. **No action replay** — Transitions (idle → combat, equipping, etc.) are not explicitly driven by replayed `TESActionData`. They may rely on Havok graph transitions triggered by Speed changes alone, which might be unreliable.

5. **Sneak/Jump/Weapon disabled** — These are compile-time disabled (`kEnableProxySneakSync = false`), possibly because you haven't found the correct graph variable names yet.

6. **No actor state sync** — Unlike TiltedEvolution, F4T doesn't replicate `ActorState` flags (combat, sneaking, etc.), which Havok graphs use to gate state machine transitions.

---

## Recommended Investigation Steps

### 1. **Identify Correct Variable Names**

Use the Creation Kit's **Behavior Graph Editor** or debug logging to find FO4's actual graph variable names:

```cpp
// In your mock-up or debug build:
void DumpActorGraphVariables(RE::Actor* actor) {
    auto graphHolder = static_cast<RE::IAnimationGraphManagerHolder*>(actor);
    RE::BSTSmartPointer<RE::BSAnimationGraphManager> manager;
    if (graphHolder->GetAnimationGraphManagerImpl(manager) && manager) {
        // Iterate graph's variable cache and log all names/values
        // This requires reverse-engineering the graph structure
    }
}
```

**Current guesses** in your code:
- `Speed` ← likely correct (common in both engines)
- `IsSprinting` ← verify against FO4 graph
- `SneakState`, `JumpState`, `WeaponDrawn` ← not yet enabled; need to verify names

### 2. **Compare with TiltedEvolution's Descriptor Approach**

Instead of writing per-frame by name, consider:
- Build a **per-graph descriptor table** for FO4's humanoid graph (map indices to variable names)
- Bulk-write the cache like Tilted does
- This avoids repeated string lookups and ensures consistency

Example:

```cpp
struct F4AnimationDescriptor {
    std::unordered_map<std::string, std::size_t> nameToIndex;
    std::vector<std::string> floatNames { "Speed", "Direction", ... };
    std::vector<std::string> boolNames { "IsSprinting", ... };
};
```

### 3. **Add Actor State Replication**

Capture and sync the remote actor's `ActorState` flags:

```cpp
// In your telemetry:
struct RemotePlayerState {
    // ... existing fields ...
    uint32_t actorStateFlags1;  // Actor::actorState.flags1
    uint32_t actorStateFlags2;  // Actor::actorState.flags2
};

// When applying to proxy:
proxy->actorState.flags1 = remoteState.actorStateFlags1;
proxy->actorState.flags2 = remoteState.actorStateFlags2;
```

This signals to the graph FSM whether the actor is in combat, sneaking, etc.

### 4. **Implement Action Replay (Optional but Recommended)**

Instead of only setting locomotion variables, capture and replay discrete actions:

```cpp
struct RemoteActionEvent {
    uint32_t actionType;
    std::string eventName;
    RemotePlayerState animationVariablesSnapshot;
    // ... target, idleForm, etc. ...
};
```

Then on the proxy:
```cpp
// Apply graph state
proxy->LoadAnimationVariables(action.Variables);
// Trigger action
ForceProxyActionWith(proxy, action);
```

### 5. **Check Animation Graph Manager Lifecycle**

Ensure the proxy actor's animation graph manager remains valid:

```cpp
// Before every write:
if (!proxy->GetAnimationGraphManager() || 
    proxy->GetAnimationGraphManager()->IsInvalysis()) {
    return;  // Graph not ready
}
```

### 6. **Thread Safety**

If applicable, add locking (like TiltedEvolution):

```cpp
auto manager = proxy->GetAnimationGraphManager();
if (manager) {
    BSScopedLock lock(manager->lock);
    // Perform writes
}
```

---

## Implementation Roadmap for F4T

### Phase 1: Debug & Understand
- [ ] Dump all animation graph variable names and types for a FO4 humanoid actor
- [ ] Verify `Speed` and `IsSprinting` work by testing manually in your fake client
- [ ] Log graph manager availability and lifecycle

### Phase 2: Core Locomotion Fix
- [ ] Replicate `ActorState` flags in `RemotePlayerState`
- [ ] Apply flags to proxy before graph variable updates
- [ ] Test walking/running/sprinting on proxy

### Phase 3: Additional Variables
- [ ] Enable and test `Sneak`, `Jump`, `WeaponDrawn` once names are confirmed
- [ ] Add direction syncing (if not already done)

### Phase 4: Action Replay (Future)
- [ ] Capture `TESActionData` on local player
- [ ] Replay on remotes via `ForceAction` or equivalent
- [ ] Handle transitions (idle ↔ combat, equipping, etc.)

### Phase 5: Optimization (Later)
- [ ] Switch to descriptor-based bulk writes (if needed for performance)
- [ ] Profile graph variable write overhead

---

## File References

### Your Files
- `plugin/src/F4TProxyAnimationSync.cpp` — Current animation sync logic
- `plugin/src/F4TProxyAnimationSync.h` — Animation state tracking
- `plugin/src/F4TProxyActorController.cpp` — Proxy actor creation/update

### TiltedEvolution Equivalents
- [AnimationSystem.cpp](https://github.com/tiltedphoques/TiltedEvolution/blob/dev/Code/client/Systems/AnimationSystem.cpp) — Action replay
- [InterpolationSystem.cpp](https://github.com/tiltedphoques/TiltedEvolution/blob/dev/Code/client/Systems/InterpolationSystem.cpp) — Movement + graph sync
- [TESObjectREFR.cpp](https://github.com/tiltedphoques/TiltedEvolution/blob/dev/Code/client/Games/Skyrim/TESObjectREFR.cpp) — Save/load animation variables
- [AnimationGraphDescriptor_Master_Behavior.cpp](https://github.com/tiltedphoques/TiltedEvolution/blob/dev/Code/encoding/Structs/Skyrim/AnimationGraphDescriptor_Master_Behavior.cpp) — Variable index tables

---

## Summary

**TiltedEvolution's Key Advantage:**
- Pre-computed variable descriptors avoid repeated string lookups
- Action replay ensures discrete actions (combat, emotes) transition correctly
- Actor state replication signals the animation graph FSM
- Lock-protected access to graph manager

**Your Current Gaps:**
1. Variable names may be wrong for FO4 (need to verify)
2. No actor state sync
3. No action replay (continuous graph writes alone may not drive state machine correctly)
4. Per-frame string-based lookups (minor performance concern)

**Next Step:** Start with **Phase 1** above—dump and verify the actual FO4 graph variable names, then enable actor state replication. That should get basic locomotion working.