"The Actor Monitor"
Distributed actor systems — Erlang processes, Akka actors, microservices — communicate by message passing. Debugging them is hard because causality is distributed: the effect of one actor’s message on another’s behavior may be mediated through chains of intermediate messages across many actors. A bug in the system may only manifest when a specific causal chain occurs.
ACTORCHESTRA monitors causal relationships across actors at runtime without modifying source code. The framework injects monitoring code at the bytecode level, intercepting message sends and receives to reconstruct the causal graph of the system’s execution. Each message carries a logical timestamp that orders events across actors, enabling the monitor to detect when causal sequences violate specified properties.
The specification language WALTZ expresses properties over multi-actor interactions. A WALTZ specification describes sequences of events across actors — “actor A sends message m, then actor B receives it and sends message n to actor C, and C must respond within k steps” — and the compiler converts these specifications into executable monitors that track the property’s satisfaction in real time.
The targeted code injection preserves the system’s behavior exactly. The monitoring is passive: it observes message traffic and maintains shadow state but never modifies message contents, delivery order, or timing. The system under test runs identically whether monitored or not, ensuring that the monitoring doesn’t introduce Heisenbugs.
Validation through Erlang case studies demonstrates violation detection in real systems — distributed protocols where race conditions or missing acknowledgments create causal chains that violate safety properties. The monitor catches these at runtime, providing the exact causal chain that led to the violation.
You can’t debug a distributed system by reading logs — by the time you read them, the causal order is lost. The monitor preserves the causality as it happens.
Write a comment