Commonwealth-Online-Public/docs/animation-architecture-alignment.md

Animation Architecture Alignment: TiltedEvolution → F4T

Executive Summary

This document maps TiltedEvolution's proven multiplayer animation and state synchronization architecture onto Fallout 4 Together's codebase. It identifies which concepts are applicable to FO4, what needs adaptation, and FO4-specific constraints.

Key Finding: TiltedEvolution's architecture is fundamentally compatible with FO4, but FO4 has stricter constraints around AI, velocity, and action replay mechanics.

---

High-Level Architecture Comparison

TiltedEvolution (Skyrim SE) - The Gold Standard

Local Player                Remote Server              Other Clients
     |                            |                           |
[Capture] - Save transform, - relay - [Receive] 
 + graph variables               |        + parse JSON
 + actor state flags             |        + update state
 + action events           [Broadcast]
     |                            |
[Send] -------- JSON line ------> |
            ~100ms intervals       |
                                   |
                    [Remote Actor Control]
                          |
                    [InterpolationSystem]
                   - Position lerp
                   - Direction lerp
                   - Graph variable load
                          |
                    [AnimationSystem]
                   - Action replay queue
                   - Actor state apply
                   - Discrete animation trigger
                          |
                   [Result: Smooth movement +
                    playing correct animations]

F4T Current (Pre-Refactor) - SetPosition Only

Local Player                Remote Server              Other Clients
     |                            |                           |
[Capture] - movement state  relay [Receive]
 + position              |        + parse JSON
 + flags (move/sprint)         [Broadcast]        + update RemotePlayerState
 + NO actor state        |
 + NO actions                    |
     |                    [Proxy Actor Control]
[Send] -------- JSON ---------> |
                                  |
                         [ProxyActorController]
                        - GetOrCreateSlot()
                        - ApplyRuntimeProxyTransform()
                           |
                           v
                    [SetPosition() only]
                    [+ Move(0, {0,0,0}, false)]
                    [+ SetLinearVelocityImpl()]
                           |
                    [Result: Smooth movement
                    but NO ANIMATIONS
                    (velocity doesn't trigger anim system)]

---

Component Mapping

1. Capture & Serialization

Concept	TiltedEvolution	F4T Current	F4T After Phase 2
Local state capture	CharacterService::RunLocalUpdates()	main.cpp:PollLocalPlayerTransform()	Same + add PollLocalPlayerActionState()
What's captured	Position, rotation, direction, animation variables snapshot, actor state flags, action events	Position, rotation, direction, movement flags, speed	Same as Tilted (extended)
Capture interval	100 ms	~50-100 ms	~100 ms
Storage	Movement { position, Variables, Direction, ... }	RemotePlayerState { x, y, z, angleZ, movementType, cellId, ... }	Extend with actorStateFlags1/2, ActionEvent[]

Action: Phase 2.3 adds action capture to main.cpp; Phase 2.1 extends struct.

2. Network Transmission

Concept	TiltedEvolution	F4T Current	F4T After Phase 2
Packet type	ClientReferencesMoveRequest (C++ struct)	JSON: {"type":"transform", "x",...}	Extend JSON with "actorStateFlags1", "actionEvents":[...]
Serialization	Binary (optimized for network)	JSON (text over TCP)	JSON (text over TCP, extended)
Frequency	~10 Hz (adaptive)	~10 Hz (throttled)	~10 Hz (throttled)
Thread	Game thread sends	Game thread sends	Game thread sends

Action: Phase 2.2 updates JSON parsing in F4TNetworking.cpp.

3. Remote State Storage

Concept	TiltedEvolution	F4T Current	F4T After Phase 2
Store	std::unordered_map<uint32 playerId, Character> in CharacterService	g_remotePlayers map + mutex	Same + extend struct
Thread	Network thread writes, game thread reads (snapshot copy)	Network thread writes, game thread reads (snapshot copy)	Same (thread-safe boundary maintained)
Per-player data	Multiple Character instances (one per remote)	One RemotePlayerState per remote	Extend with flags + action queue

Action: Phase 2.1 modifies struct; thread safety unchanged.

4. Proxy Actor Spawning & Lifecycle

Concept	TiltedEvolution	F4T Current	F4T After Phase 4
Spawn method	Actor::PlaceAtMe() on demand	Pre-placed pool in CK	Dynamic PlaceAtMe() (like Tilted)
Lifecycle	Create on connect, destroy on disconnect	Pool reuse; hold out of range when inactive	Pool reuse (same as current)
Count limit	~256 (depends on memory)	4 (pre-placed limit)	4-8 (dynamic pool size)
AI	Suppressed (SetRemote(true))	Suppressed (neutralize)	Suppressed (neutralize)
Puppet mode	Yes (movement external)	Yes (external via SetPosition)	Yes (external via SetPosition + velocity)

Action: Phase 4.1 replaces pool init with dynamic spawn logic.

5. Movement & Interpolation

Concept	TiltedEvolution	F4T Current	F4T After Phase 4
Position sync	ForcePosition() (Skyrim API)	SetPosition() (FO4 API)	Same (SetPosition works)
Rotation sync	Lerp + direct set	SetHeading() (snap)	SetHeading() + lerp (Phase 4.3 later)
Velocity	Natural from AI controller + character controller velocity	Manual injection via Move() + SetLinearVelocityImpl()	Same (proven approach)
Interpolation	InterpolationSystem lerps transform + graph vars	Lerp position, snap rotation	Enhance rotation lerp (Phase 7+)

Action: Phase 4.2 verifies velocity injection works on dynamic proxies.

6. Animation Variable Sync

Concept	TiltedEvolution	F4T Current	F4T After Phase 5
Method	Descriptor-based bulk write (by index)	String-based per-variable write	Descriptor-based bulk write (Phase 3 + 5)
Variables	Speed, Direction, IsSprinting, IsSneaking, + others	Speed (string fallback), IsSprinting	Same minimal set + indexed
Frequency	Once per action + continuous during interpolation	Every frame	Every frame (indexed, faster)
Thread safety	BSScopedLock on graph manager	No explicit locking	Add locking (Phase 5)
Variable persistence	Snapshot approach (graph state saved/loaded)	Continuous writes	Hybrid: snapshot for actions, continuous for locomotion

Action: Phase 3 builds descriptor class; Phase 5 refactors sync to use it.

7. Action Replay System

Concept	TiltedEvolution	F4T Current	F4T After Phase 5
Capture	Hook ActorMediator::PerformAction() → queue ActionEvent	Not implemented	Add PerformAction hook (Phase 2.3)
Storage	RemoteAnimationComponent::TimePoints (event queue per remote)	No action queue	RemoteActionComponent per proxy (Phase 5.1)
Replay	AnimationSystem::Update() pops + replays actions in order	No action replay	ApplyRemoteAction() call in slot update (Phase 5.2)
Content	{ Type, EventName, Variables, State1, State2, IdleForm, Target, ... }	N/A	Minimal set: { Type, EventName, Variables, State1, State2 }
Effect	Discrete animations (combat idles, emotes, equipping) play deterministically	Proxies stand still	Combat idles + transitions animate

Action: Phases 2.3, 5.1, 5.2 implement action pipeline.

8. Actor State Flags

Concept	TiltedEvolution	F4T Current	F4T After Phase 2
Capture	actor→actorState.flags1/flags2 on local player	Not captured	Add to state capture (Phase 2.3)
Network	Packed in ActionEvent.State1/State2	No	Add to transform packet JSON
Apply	Before action replay: actor→actorState.flags1 = remote.State1	No	Apply before animation sync (Phase 4.3)
Purpose	Signal animation graph FSM (combat mode, sneaking, etc.)	No effect	Enable graph-driven state transitions

Action: Phase 2.1 adds fields; Phase 2.3 captures; Phase 4.3 applies.

---

Data Flow Diagrams

TiltedEvolution Animation Sync (Reference)

PerformAction hook (local)
    ↓
Capture: {Type, EventName, Variables, State1, State2, Idle}
    ↓
Network: Send ActionEvent + Movement (animation vars)
    ↓
Remote receives (network thread)
    ↓
AnimationSystem::Update() (game thread)
    ↓
For each action in queue:
  - actor→actorState.flags1/2 = State1/State2
  - actor→LoadAnimationVariables(Variables)
  - ActorMediator::ForceAction(ActionData)
    ↓
Result: Correct animation plays

F4T Current Animation Sync (Broken)

PollLocalPlayerTransform (local)
    ↓
Capture: {position, angle, moving, sprinting}
    ↓
Network: Send transform (JSON)
    ↓
Remote receives (network thread)
    ↓
ProxyActorController::Update() (game thread)
    ↓
ApplyRuntimeProxyTransform()
  - SetPosition()
  - Move(0, {0,0,0}, false) → get controller
  - SetLinearVelocityImpl(calc_velocity)
    ↓
ApplyProxyAnimationFromRemoteState()
  - TrySetGraphFloat("Speed", value)  ← writes by name, slow
  - TrySetGraphBool("IsSprinting", value)
    ↓
Result: Movement works, but animations NEVER play
         (velocity is set, but graph doesn't react)

F4T After Phase 5 (Proposed)

Local Player
  ├─ PollLocalPlayerTransform()
  │  └─ Capture: {position, angle, velocity, movementFlags}
  │
  ├─ PollLocalPlayerActionState() [NEW - Phase 2.3]
  │  └─ Capture: {actor→actorState, PerformAction events}
  │
  └─ SendTransformPacket() [EXTENDED - Phase 2.2]
     └─ JSON: {position, angle, actorStateFlags1/2, actionEvents}

Network Thread
  └─ HandleTransformPacket()
     └─ UpdateRemotePlayer() [EXTENDED - Phase 2.1]
        └─ RemotePlayerState: {position, flags, action queue}

Game Thread (ProxyActorController::Update)
  ├─ GetRemotePlayerSnapshot()
  │
  ├─ For each remote:
  │  ├─ GetOrCreateSlot()
  │  ├─ RestoreSlotProxyForRemotePlayer() [First frame]
  │  │  └─ Snap transform
  │  │
  │  └─ UpdateRuntimeSlotForRemotePlayer() [Ongoing]
  │     ├─ ApplyRuntimeProxyTransform() [Phase 4]
  │     │  ├─ SetPosition()
  │     │  ├─ Move(0, {0,0,0}, false)
  │     │  └─ SetLinearVelocityImpl()
  │     │
  │     ├─ ApplyRemoteActionState() [NEW - Phase 4.3]
  │     │  └─ actor→actorState.flags1/2 = remote.flags
  │     │
  │     ├─ ReplayRemoteActions() [NEW - Phase 5.2]
  │     │  └─ For each action:
  │     │     ├─ actor→LoadAnimationVariablesFromDescriptor()
  │     │     └─ ForceProxyAction()
  │     │
  │     └─ ApplyProxyAnimationFromRemoteState() [REFACTORED - Phase 5.3]
  │        └─ LoadAnimationVariablesFromDescriptor() [indexed, bulk]

Result
  ├─ ✅ Smooth movement
  ├─ ✅ Character controller velocity set
  ├─ ✅ Locomotion animations (idle/walk/run/sprint)
  ├─ ✅ Action animations (combat, equips)
  ├─ ✅ Actor state affects graph FSM
  └─ ✅ Performance: O(1) indexed access, not O(n) string lookup

---

FO4-Specific Adaptations & Constraints

1. Character Controller Velocity

Aspect	Skyrim SE	FO4	F4T Approach
API exposure	High (character controller accessible)	Lower (not directly exposed)	Use Actor::Move() return value to get controller
Velocity injection	Direct via API	Via Move() + SetLinearVelocityImpl()	Same as current proven approach
Animation system watching velocity	Yes	Yes (verified in dev-log)	Confirmed working; use same method

Note: F4T already verified this works (see dev-log 2026-06-03). Use same velocity injection.

2. Actor State Flags

Aspect	Skyrim SE	FO4	F4T Approach
Flags struct	ActorState	ActorState	Same struct + same flags
Graph dependency	Yes (FSM gated by flags)	Yes (FSM gated by flags)	Apply before animation sync
Combat flag	inCombat	inCombat	Same
Sneak flag	sneaking	sneaking	Same

Note: F4T tried SetSneaking() API directly (disabled in code). Better approach: sync actor→actorState.flags1/2 bitfields. More reliable than API calls.

3. Action Replay

Aspect	Skyrim SE	FO4	F4T Approach
API	ActorMediator::ForceAction()	ActorMediator::ForceAction()	Same
Availability	Well-exposed in common headers	Exposed in CommonLibF4	Use same
Idle forms	Rich idle system with conditions	Rich idle system with conditions	Capture + replay
Combat actions	Power attacks, spells, shouts	Power attacks, guns, grenades	Defer to Phase 2+

Note: CommonLibF4 likely has equivalent ActorMediator and ForceAction() (verify in Phase 1.2 testing).

4. Animation Graph Manager

Aspect	Skyrim SE	FO4	F4T Approach
Manager lifecycle	Stable per actor	Stable per actor	Same assumptions
Variable cache	Accessible via GetBSAnimationGraph()	Accessible via GetAnimationGraphManagerImpl()	Use CommonLibF4 API (current code works)
Thread safety	Requires BSScopedLock	Requires BSScopedLock	Add explicit locking (Phase 5)
Descriptor key	Hash of graph type name	Hash of graph type name	Build descriptor once, cache

Note: F4T already uses the CommonLibF4 API successfully. No changes needed, just add locking.

5. Proxy Puppet Suppression

Aspect	Skyrim SE	FO4	F4T Approach
AI blocking	SetRemote(true)	No direct equivalent	Current: NeutralizeProxyActor() + flags (kept for Phase 4)
Movement blocking	External ForcePosition()	External SetPosition()	Same
Action suppression	Hook PerformAction return 0	Need to verify hook exists	Assume possible; test in Phase 1

Note: F4T currently suppresses AI via actor flags/hostile reset. Tilted uses a Remote flag. Both approaches work; F4T's is compatible.

---

Phase-by-Phase Integration Points

Phase 1: Research

• Verify FO4 variable indices match assumptions
• Confirm ActorMediator::ForceAction() exists in CommonLibF4
• Document any differences from Skyrim model

Phase 2: Protocol & State

• Extend JSON packets with actorStateFlags1/2, actionEvents
• Add action capture in PerformAction hook (or equivalent)
• Maintain backward compatibility

Phase 3: Descriptors

• Build F4AnimationDescriptor using discovered indices
• Implement bulk read/write methods
• Verify thread safety with BSScopedLock

Phase 4: Spawn & Velocity

• Switch to dynamic spawn (proof of concept: single proxy first)
• Verify velocity injection works on dynamic actors
• Apply actor state flags before animation sync

Phase 5: Action Replay

• Implement RemoteActionComponent action queue
• Refactor animation sync to use descriptors
• Ensure action replay doesn't conflict with continuous locomotion

Phase 6-7: Testing & Polish

• Verify animations play correctly
• Performance benchmark descriptor vs string-based
• Document FO4-specific adaptations

---

Success Criteria for Phase 1

• Document all float/bool/int variables for humanoid graph (variable names, likely indices)
• Confirm TiltedEvolution concepts apply to FO4 (actor state, action replay, descriptors)
• Verify Actor::Move() + SetLinearVelocityImpl() works on dynamic proxies
• No major architectural blockers identified
• Produce animation-architecture-alignment.md (this document)

---

References

TiltedEvolution Source

• AnimationSystem.cpp - Action replay
• CharacterService.cpp - State capture
• AnimationGraphDescriptor_Master_Behavior.cpp - Variable indices

F4T Source

• plugin/src/main.cpp - Local transform capture
• plugin/src/F4TNetworking.cpp - JSON serialization
• plugin/src/F4TProxyAnimationSync.cpp - Current animation sync (to be refactored)
• plugin/src/F4TProxyActorController.cpp - Proxy lifecycle + movement

CommonLibF4 Headers

• RE/H/ActorState.h - Actor state flags
• RE/H/BSAnimationGraphManager.h - Graph manager
• RE/H/IAnimationGraphManagerHolder.h - Graph holder interface
• RE/H/Actor.h - Actor class (Move, SetPosition, etc.)