APIs, concepts, guides, and more
Sync Groups

Partition EtherCAT nodes into fault-contained sync groups so an optional section of the network can degrade, disconnect, and recover in place while the rest of the machine keeps running.

🔹 What is a Sync Group?

By default, the RMP treats the EtherCAT network as a single fault domain. Every cyclic process-data datagram carries an expected Working Counter (WKC): each node that successfully exchanges its process data increments the counter, so a received WKC lower than expected means some node did not participate. Out of the box, a sustained WKC mismatch on any node is fatal to the whole network within a few samples, because the RMP cannot know which data is still trustworthy.

A sync group changes that. Nodes assigned to a non-zero sync group are placed in their own cyclic datagram with its own expected Working Counter, so the RMP can evaluate their process-data health independently:

  • Group 0 is the default, mandatory group. Every node belongs to it unless configured otherwise, and its failure behavior is unchanged: a sustained WKC mismatch remains fatal to the network.
  • Non-zero groups are fault-tolerant. When a member node stops exchanging process data (left the Operational state, cable pulled, powered off), the group is marked degraded or absent, its data is flagged invalid, and the rest of the network keeps running: no other group's data is invalidated and nothing escalates to a network fault.

In EtherCAT terms, each sync group becomes a Master Sync Unit in the generated ENI file, giving the group its own process-data / WKC boundary.

🔹 Why use Sync Groups?

Use sync groups when part of your network is optional or removable and the rest must keep running when that part fails or is serviced:

  • Removable tooling or end-of-arm fixtures on a robot.
  • Modular I/O sections that operators may disconnect for maintenance.
  • Auxiliary devices (vision triggers, indicator I/O) whose loss should not stop motion on the main axes.
  • Bench and commissioning setups where nodes come and go.

What you get, compared to the default all-or-nothing behavior:

Event on a grouped node Without sync groups With sync groups
Node drops out of OP Network fatal within a few samples Group DEGRADED; other groups keep running with valid data
Cable pulled / node powered off Network fatal Group ABSENT; the network survives (see the containment note below)
Bringing the node back Full network restart (tens of seconds, all axes down) NetworkSyncGroupRecover, typically tens to a few hundred milliseconds, no restart

Containment is logical, not physical
Sync groups contain node-dropout faults: a grouped node can leave OP, be commanded away, or stop answering without invalidating any other group's data or faulting the network. Tolerance covers Working-Counter deficits only, because a dropout can only reduce the counter; a WKC surplus (more responders than the configuration expects) is a network-integrity fault and remains fatal, exactly as on an ungrouped network. All groups still share one physical Ethernet frame per cycle, so a physical disruption of the cable plant is felt bus-wide while it lasts. Measured on a bench rig: unplugging a grouped node cost zero to one network-wide cyclic frame; replugging it cost 100 to 160 consecutive frames while the upstream port reopened and the link retrained. A lost frame stales every group's data for that cycle, which can trip a tightly tuned following-error limit on an unrelated axis. SDO-commanded state drops and in-place recovery do not disturb the shared frame; only physical connect/disconnect events do.

Availability, not safety
Sync group tolerance is an availability feature: it keeps the machine running around a failed optional section. It is not a safety-rated detection mechanism. For safety-rated applications, the safety function must remain on FSoE or other safety-rated channels.

🔹 Order of Operations

Sync group membership is declared before ENI generation and baked into the ENI file. The workflow is: discover, configure, regenerate, restart.

1. Start the network (discovery)

Start the network normally (for example with NetworkStart()) so all nodes are discovered and NetworkNode objects exist. See get-started-first-steps_start-the-network for a walkthrough.

2. Configure the NetworkNodes

Assign each optional node to a non-zero sync group with ConfigSet. The SyncGroup member of the node's Config structure selects the group (0 is the default group). Nodes you do not configure stay in group 0.

Staged vs. actual
ConfigSet stages a desired configuration that is consumed at ENI-generation time. It is not runtime state: network discovery re-zeroes staged configs on every restart by design. After a restart, the loaded ENI is the source of truth and a node's actual group is reported by the SyncGroupId field of StatusGet, not by ConfigGet.

3. Regenerate the ENI

Call NetworkEniGenerate (or use rsiconfig, or the GUI paths described in ENI File (EtherCAT.xml)). ENI generation reads the staged node configs and emits the corresponding Master Sync Unit entries, so each group gets its own cyclic datagram, expected Working Counter, and an RsiSyncGroup metadata element that the EtherCAT master reads at startup.

4. Restart the network

Call NetworkShutdown() followed by NetworkStart(). The master parses the sync group metadata from the ENI, populates the group status array, and reports each node's actual membership in Status.SyncGroupId.

Because membership travels in the ENI, it survives subsequent network restarts; you only repeat steps 2 and 3 when you want to change the grouping.

🔹 Sync Group States

Each group runs a small state machine, reported as the State field of NetworkSyncGroupStatusGet (see RSINetworkSyncGroupState):

State Meaning
UNUSED Group slot is not in use (no nodes assigned).
OPERATIONAL All member nodes are exchanging process data; the group WKC matches its expected value.
DEGRADED Members are responding on the wire, but at least one is not Operational (or only some members are responding). The WKC mismatch is tolerated and counted.
RECOVERING NetworkSyncGroupRecover is in progress for this group.
ABSENT No member of the group is responding on the wire (physically disconnected or unpowered).

Notes on the transitions:

  • DEGRADED and ABSENT are detected from the Working Counter within a few network samples.
  • A group can return to OPERATIONAL autonomously if its members re-enter OP on their own (some simple devices never demote their state during a short cable disconnect); otherwise call NetworkSyncGroupRecover.
  • If a recovery attempt fails (an exception, or a logged error when exceptions are disabled), the group's prior state is restored so the failure remains visible.

🔹 Monitoring Sync Groups

Group status

NetworkSyncGroupCountGet returns the number of group slots populated from the ENI at startup (at least 1 once the network is operational). NetworkSyncGroupStatusGet returns a NetworkSyncGroupStatus snapshot per group:

Field Description
Id Sync group id (0 = default group).
State Current group state, see the table above.
ExpectedWkc Expected Working Counter for the group's cyclic datagram.
ActualWkc Most recent actual Working Counter. Less than ExpectedWkc while a member is out of OP.
NodeCount Number of nodes assigned to this group.
DataValid true when the group's cyclic data is trustworthy (WKC matches). Cleared on a tolerated mismatch; set again when the WKC matches.
ToleratedWkcCount Cumulative count of WKC mismatches tolerated for this group since network start.
OperationalNodeCount Member nodes currently in OP. Equals NodeCount when healthy.
NonOperationalNodeMask Bitmask (by node index) of member nodes not in OP. Use it to identify exactly which node dropped in a multi-node group.

Timing characteristics to be aware of:

  • State, ActualWkc, and DataValid are authoritative: the master maintains them every cycle from the Working Counter, so they flip within a few samples of the fault. Treat them as the immediate degrade signal.
  • OperationalNodeCount and NonOperationalNodeMask are derived from per-node AL-status reads refreshed about once per second while the group is degraded, so they can lag by a second or two. A fresh DEGRADED state with a zero mask means "not yet attributed", not "all members healthy".
  • ToleratedWkcCount is cumulative per network session. To measure a single event, snapshot it before and after and take the delta.

Per-node status

StatusGet reports each node's actual membership and reachability:

  • SyncGroupId: the group this node belongs to, from the loaded ENI.
  • Present: true if the node responded to the master's most recent AL-status read. When false, the node's AlStatus is stale and should not be trusted.

Recording group status

The group status fields are recordable and readable by address via AddressGet using the NETWORK_SYNC_GROUP_* members of RSIControllerAddressType (state, expected/actual WKC, data valid, tolerated WKC count, operational node count, and non-operational node mask). Pass the group id as the object index. This lets the Recorder capture sample-exact group state transitions alongside your motion signals.

A node's group membership is also readable by address: NETWORK_NODE_SYNC_GROUP_ID takes a node index (not a group id) as the object index and returns the sync group id that node was assigned at ENI generation (0 = default group), so a recording can correlate per-node data with the right group's status.

DataValid does not reveal SafeOp
A node commanded to SafeOp keeps producing inputs, so its group can read OPERATIONAL with DataValid=true while its outputs are not being applied. Group status tells you the process data exchange is healthy; check the node's AL status when you need to know its exact EtherCAT state.

🔹 Recovering a Sync Group

NetworkSyncGroupRecover brings a degraded non-zero group back to Operational without restarting the network:

  1. For each member node that is not in OP, it acknowledges any AL error and drives the node through the EtherCAT state machine (PreOp/SafeOp to Op). Members already in OP are skipped.
  2. It then waits for the master to confirm the group's Working Counter matches again (DataValid), which is the recovery-success signal.

Group 0 cyclic I/O continues untouched throughout; recovery typically completes in tens to a few hundred milliseconds (nodes that are the Distributed Clocks reference take the longest because DC resynchronization dominates).

Recovery is deliberately conservative and raises an error instead of guessing:

  • Group 0 cannot be recovered; it is the mandatory default group.
  • The network must be OPERATIONAL.
  • An ABSENT group cannot be recovered; reconnect the hardware first. Once members respond again (group returns to DEGRADED), recovery proceeds normally.
  • A node that lost its ESC configuration (for example, it was power cycled) cannot be reached by recovery: it no longer answers at its configured station address. The error message identifies this case (an unconfigured node visible on the wire) and a full network restart (NetworkShutdown() + NetworkStart()) is required to reclaim it.
  • A group that is already RECOVERING cannot be recovered again until the in-progress recovery finishes.

The call blocks until it succeeds or fails: the timeout parameter is a wall-clock deadline for each AL transition of each member node, so the worst case is roughly (member nodes) x 2 x timeout, plus a fixed 2 s Working Counter confirmation wait (each individual service-channel operation can overshoot a deadline by up to twice the SDO timeout under contention). It does not hold the controller lock while polling: different groups can be recovered concurrently from different threads, while a second call for the same group fails fast with an error until the first returns (that protection is per process). While any recovery is in flight, Reset and Refresh raise an error instead of deleting the node objects the recovery is using.

A practical rule of thumb: recovery handles everything that kept its ESC configuration (commanded state drops, sync manager watchdog trips, cable disconnect/reconnect); a power cycle needs a restart.

🔹 Axis Faults from Sync Groups

If a sync group contains servo drives, the axes on those drives must not keep trusting frozen feedback. The firmware asserts a configurable per-axis action while the group containing the axis's node reports DEGRADED or ABSENT:

  • Configure it with SyncGroupFaultActionSet / SyncGroupFaultActionGet. The default action is RSIActionABORT.
  • The fault source reported by SourceGet is RSISourceSYNC_GROUP_FAULT.
  • The fault latches until ClearFaults. Clearing while the group is still down re-latches it, so an axis cannot be re-enabled against a dead group. RECOVERING and OPERATIONAL do not assert. Because RECOVERING does not assert, an axis can be cleared and re-enabled while members are still walking the AL states; their outputs are not applied yet, so the enable can trip an immediate following error. Wait for NetworkSyncGroupRecover to return before re-enabling.
  • Applications can wait on the corresponding RSIEventTypeSYNC_GROUP_FAULT interrupt (see InterruptWait) instead of polling group status.

Axes on the default group (group 0) are excluded from this fault: group 0 failures are network-wide failures owned by the long-standing wrong-Working-Counter escalation, which stops the network (and with it every axis) within a few cycles when the problem persists. The exclusion keeps behavior on networks without sync groups identical to releases before sync groups existed: a transient single-cycle Working Counter mismatch is log-only and never aborts axes.

🔹 Choosing Group Membership

Group membership is both a fault-containment and a fault-blast-radius decision:

Co-grouped nodes share a fate
All members of a group share one cyclic datagram. While any member's failure makes the group's WKC mismatch, the master stops copying inputs for the entire group: healthy members keep their last-good values, frozen, with DataValid=false. Nodes that must keep fresh data through a peer's failure belong in different groups.

Additional guidance:

  • Keep the machine's core in group 0. Group 0 is the only group whose failure stops the network, which is exactly what you want for mandatory axes and I/O.
  • Topology matters for physical disconnects. EtherCAT is a line topology: unplugging a mid-line node also severs everything downstream of it. Place removable grouped sections at the end of the line (or on their own junction/coupler port) so disconnecting them cannot take mandatory nodes with them. Severing mandatory nodes is a network-fatal event by design.
  • Keep the Distributed Clocks reference in group 0. The DC reference clock is normally the first DC-capable node on the network. Assigning it to a removable group is not recommended (and not currently validated on all platforms): plan your node order so a group-0 node is the DC reference.
  • Prefer orderly removal. Disconnecting a node (or closing its upstream port) before powering it off is the cleanest sequence and is fully contained. Surprise power-offs of a connected node disturb the physical layer; the RMP hardens its DC clock discipline against this, but orderly removal removes the risk entirely.
  • Plan reconnects for a tolerant moment. Physically replugging a cable is the most disruptive event on the shared frame (link retraining can cost 100+ consecutive cycles bus-wide, see the containment note above). Where that matters, reconnect during a pause in motion, or open the error limits on tightly tuned axes for the reconnection window.

🔹 Limitations

  • Group state is Working-Counter based. Members that exchange process data without sync managers (for example, a coupler such as the EK1100 plus simple digital terminals) keep answering cyclic datagrams even in PreOp, so a group made up entirely of such devices can still read OPERATIONAL after an SDO-commanded state drop. Servo drives and other SM-based devices stop contributing WKC when they leave OP and are detected reliably. Physical disconnects are always detected (ABSENT). When in doubt, check per-node AL status.
  • Containment is not physical-frame isolation. All groups share one cyclic Ethernet frame, so physically unplugging or replugging a grouped node can cost whole frames network-wide while the link renegotiates (measured: zero to one frame on unplug, 100 to 160 consecutive frames on replug). The guarantee is that grouped-node dropout faults (state changes and WKC deficits) never invalidate other groups or fault the network; it is not a guarantee that the shared physical layer is undisturbed, and a WKC surplus stays network-fatal by design. See the containment note near the top of this page.
  • Membership is configuration-time only. Changing groups requires an ENI regeneration and a network restart. Groups cannot be reassigned while the network runs.
  • Groups must be present at startup. All nodes described by the ENI, grouped or not, must be connected for the network to start. Starting with a group physically absent is not supported.
  • No dynamic reconfiguration. Sync groups provide graceful degradation and in-place recovery, not on-the-fly topology changes. Attaching a group that was absent at startup, reattaching a node elsewhere in the topology, or re-initializing a single node while the network runs all require a full network restart.
  • A power-cycled node needs a restart to rejoin. In-place recovery covers nodes that kept their ESC configuration; re-initializing a single node from scratch while the network runs is not yet supported.
  • Only the first 64 nodes can be grouped. Group membership travels in 64-bit node-index masks, so a non-zero sync group on a node at index 64 or higher is rejected at ENI generation with RSINetworkEniResultSYNC_GROUP_NODE_INDEX_UNSUPPORTED.
  • Only single-sync-unit devices can be grouped. A group's own cyclic datagram carries slave sync unit 0 only, so part of a multi-sync-unit device's process data would remain in the default datagram and dropping the node would still fault group 0. Such a configuration (for example, grouping an EL6910 TwinSAFE PLC) is rejected at ENI generation with RSINetworkEniResultSYNC_GROUP_MULTI_SYNC_UNIT_UNSUPPORTED.
  • The number of sync groups is bounded; ConfigSet validates the group id and reports the supported range in its error message.

📜 Sample Code

The following samples configure a node into sync group 1, monitor the group, and recover it after a fault.

  • C# ⚙️

    Configure a node into a sync group

    // 1. Start the network so nodes are discovered.
    controller.NetworkStart();
    // 2. Stage the group assignment on the optional/removable node.
    RapidCodeNetworkNode node = controller.NetworkNodeGet(nodeIndex);
    var config = node.ConfigGet();
    config.SyncGroup = 1; // 0 = default (mandatory) group
    node.ConfigSet(config);
    // 3. Regenerate the ENI; generation consumes the staged configs.
    RSINetworkEniResult result = controller.NetworkEniGenerate();
    if (result != RSINetworkEniResult.RSINetworkEniResultSUCCESS)
    Console.WriteLine(controller.NetworkEniGenerateOutputGet());
    // 4. Restart the network on the new ENI.
    controller.NetworkShutdown();
    controller.NetworkStart();
    // The node's ACTUAL membership now comes from the loaded ENI.
    Console.WriteLine($"Node {nodeIndex} is in sync group " +
    $"{node.StatusGet().SyncGroupId}");

    Monitor the groups

    int groupCount = controller.NetworkSyncGroupCountGet();
    for (int groupId = 0; groupId < groupCount; groupId++)
    {
    var status = controller.NetworkSyncGroupStatusGet(groupId);
    Console.WriteLine($"Group {status.Id}: {status.State} " +
    $"WKC {status.ActualWkc}/{status.ExpectedWkc} " +
    $"DataValid={status.DataValid} " +
    $"OpNodes {status.OperationalNodeCount}/{status.NodeCount}");
    }

    Recover a degraded group

    var status = controller.NetworkSyncGroupStatusGet(1);
    if (status.State == RSINetworkSyncGroupState.RSINetworkSyncGroupStateDEGRADED)
    {
    // Drives every non-OP member back to OP. No network restart;
    // group-0 cyclic I/O continues throughout.
    controller.NetworkSyncGroupRecover(1);
    }
    // An ABSENT group must be physically reconnected first;
    // NetworkSyncGroupRecover raises a descriptive error otherwise.

    Configure the axis fault action

    // Default is ABORT; pick the stopping action your machine needs.
    axis.SyncGroupFaultActionSet(RSIAction.RSIActionABORT);
    // After the group is recovered, clear the latched fault.
    // (ClearFaults while the group is still down will re-latch.)
    axis.ClearFaults();

  • Python 🐍

    Configure a node into a sync group

    controller.NetworkStart() # 1. discover the network
    node = controller.NetworkNodeGet(node_index)
    config = node.ConfigGet()
    config.SyncGroup = 1 # 2. stage it (0 = default group)
    node.ConfigSet(config)
    result = controller.NetworkEniGenerate() # 3. consumes the staged configs
    controller.NetworkShutdown() # 4. restart on the new ENI
    controller.NetworkStart()
    print(f"Node {node_index} is in sync group "
    f"{node.StatusGet().SyncGroupId}") # actual membership, from the ENI

    Monitor and recover

    for group_id in range(controller.NetworkSyncGroupCountGet()):
    status = controller.NetworkSyncGroupStatusGet(group_id)
    print(f"Group {status.Id}: state={status.State} "
    f"WKC {status.ActualWkc}/{status.ExpectedWkc} "
    f"DataValid={status.DataValid}")
    controller.NetworkSyncGroupRecover(1) # after members are back on the wire