Tutorials > Customization > SDK > Parsing Lists of Objects in Live Watch Plugins

Parsing Lists of Objects in Live Watch Plugins

April 4, 2023 live watch, sdk, stm32

This tutorial shows how to use the Live Watch plugins to discover and show all variables of a particular type, whether created statically or dynamically. We will extend a basic Live Watch plugin from this tutorial that locates the g_Counter variable and creates a custom Live Watch node for it. Instead of looking for a particular variable, we will create the nodes for all variables of the SampleCounter type, and even discover dynamically allocated instances by traversing linked lists.

Before you begin, clone this git tag, or simply follow our previous tutorial to get a basic live watch plugin working.

Open the SampleLiveWatchExtension project from this tutorial. Locate the CounterListNode.GetChildren() method that creates a node for g_Counter, and replace it with the code shown below:

var result = new List<ILiveWatchNode>();
 
foreach(var v in _Engine.Symbols.TopLevelVariables)
{
	if (v.RawType.Resolved == "SampleCounter")
	{
		result.Add(new CounterNode(_Engine, v));
	}
}
 
return result.ToArray();

Note how we are iterating through all global/static variables, checking their resolved type (i.e. resolving typedefs), and creating nodes for all variables that have a matching type.

Press F5 to start debugging the plugin. When the second instance of Visual Studio launches, open the SampleLiveWatchProject project in it, rename g_Counter to g_Counter1 and add a g_Counter2 variable next to it. Make sure you assign the Name field for both of them:
Start debugging the embedded project. Note how the Live Watch window now shows both Test counter #1 and Test Counter #2: You can use the same technique to discover and present objects of various types. The GetChildren() method is normally called only once when the node is expanded for the first time, so you can easily iterate through all global variables in it, find the relevant ones, and create meaningful nodes for them.

Now we will show how to handle dynamic lists. Add a “SampleCounter *Next” field to the SampleCounter struct, and create a few more instances of it dynamically:

    g_Counter1.Next = (SampleCounter *)malloc(sizeof(SampleCounter));
    g_Counter1.Next->Name = "Dynamic counter #1";
 
    g_Counter1.Next->Next = (SampleCounter *)malloc(sizeof(SampleCounter));
    g_Counter1.Next->Next->Name = "Dynamic counter #2";
 
    g_Counter1.Next->Next->Next = NULL;

We will now modify the CounterListNode class to dynamically parse the linked lists of counters, and show them in the child list. In order to do that, we would need to:

Remember the list of statically defined counters, so that we could traverse the linked lists starting at them.
Locate the definition of the SampleCounter type, so that we could dynamically create pinned variables of that type.
Get the exact offset and size of the Next field, so that we could read it in order to traverse the lists.

Update the GetChildren() method as shown below:

List<CounterNode> _StaticCounters = new List<CounterNode>();
 
IPinnedVariableStructType _CounterStruct;
IPinnedVariableStructMember _NextField;
 
public ILiveWatchNode[] GetChildren(LiveWatchChildrenRequestReason reason)
{
	var result = new List<ILiveWatchNode>();
 
	foreach (var v in _Engine.Symbols.TopLevelVariables)
	{
		if (v.RawType.Resolved == "SampleCounter")
			_StaticCounters.Add(new CounterNode(_Engine, v));
	}
 
	_CounterStruct = _Engine.Symbols.LookupType("SampleCounter") as IPinnedVariableStructType;
	_NextField = _CounterStruct?.LookupMember("Next", false);
 
	return result.ToArray();
}

Note that as GetChildren() only gets called once per node, it doesn’t parse the dynamic lists, but only loads the data needed to parse them later.

Add a “public ulong Address => _Counter.Address;” property to the CounterNode class, so that we could get the addresses from the list of CounterNode-s:

The most straight-forward implementation of dynamic list parsing would be to just recursively read the Next field of each counter, starting at the top-level nodes, and create instances of CounterNode as needed. However, it would create 2 performance issues:

Reading each pointer separately would introduce considerable delay due to latency.
Creating the node objects at each refresh would cause unnecessary updates of the Live Watch window.

We will address these issues by maintaining a cache of dynamically watched counters. For each target memory address that contains a counter, we will keep:

A Live Variable corresponding to its Next field, so that VisualGDB can read it ahead of time, together with other live variables.
A Live Watch Node (actually, our own CounterNode class) corresponding to that variable.

To do this, define a WatchedCounter class inside the CounterListNode class:

class WatchedCounter
{
	public readonly ILiveVariable NextField;
	public readonly CounterNode Node;
 
	public WatchedCounter(CounterListNode list, ulong addr)
	{
		NextField = list._Engine.Memory.CreateLiveVariable(addr + list._NextField.Offset, list._NextField.Size, $"[{addr:x8}]");
		var tv = list._Engine.Symbols.CreateTypedVariable(addr, list._CounterStruct);
 
		if (tv != null)
			Node = new CounterNode(list._Engine, tv);
	}
 
	public bool IsValid => NextField != null && Node != null;
}

Update the UpdateState() method as shown below:

        public LiveWatchNodeState UpdateState(LiveWatchUpdateContext context)
        {
            List<ILiveWatchNode> children = new List<ILiveWatchNode>();
            children.AddRange(_StaticCounters);
 
            if (_NextField != null)
            {
                foreach (var head in _StaticCounters)
                {
                    ulong addr, nextAddr;
                    for (addr = head.Address; addr != 0; addr = nextAddr)
                    {
                        if (!_DynamicCounters.TryGetValue(addr, out WatchedCounter counter))
                            _DynamicCounters[addr] = counter = new WatchedCounter(this, addr);
 
                        if (!counter.IsValid)
                            break;
 
                        if (addr != head.Address) 
                            children.Add(counter.Node);
 
                        nextAddr = counter.NextField.GetValue().ToUlong();
                    }
                }
            }
 
            return new LiveWatchNodeState
            {
                Value = "Changed",
                NewChildren = children.Count == 0 ? null : children.ToArray(),
            };
        }

This traverses the lists starting at each statically discovered instance (_StaticCounters), creates the WatchedCounter instances for each item in the list (including the very first ones), and returns the corresponding nodes via the NewChildren field.
Note that if the node was never expanded, its GetChildren() method would never get called, so _NextField would be null and no lists will be traversed. If this is the case, we set NewChildren to null, to get VisualGDB a chance to call GetChildren() when the node is first expanded.

Run the embedded program and see how the dynamically discovered counters are now also shown in the list:
Try collapsing the “Counters” node in the Live Watch view. Note how despite not needing to parse the counter lists, VisualGDB still tries to read 4 live variables at each refresh cycle. These are the variables for the Next field held by the WatchedCounter instances:

We can prevent them from being updated if the Counters node is fully scrolled out or suspended by implementing the SetSuspendState() method in CounterListNode:

public void SetSuspendState(LiveWatchNodeSuspendState state)
{
	lock (_DynamicCounters)
		foreach (var c in _DynamicCounters.Values)
			if (c.IsValid)
				c.NextField.SuspendUpdating = state.SuspendRegularUpdates || !state.IsExpanded;
}

You would also need to wrap the if (_NextField != null) {…} block with another “lock (_DynamicCounters) {}” to avoid race conditions between SetSuspendState() and UpdateState(). The lock() syntax is a shorthand for acquiring a mutex associated with _DynamicCounters at the beginning, and releasing it at the end of the block. You could also use a mutex explicitly, or lock on any other .Net object, as long as both methods lock on the same object, and no other method tries to lock on it for a different purpose.

Now collapsing the Counters node will automatically prevent the list pointers from being updated, freeing the memory access bandwidth for other tasks:One last improvement would be to automatically remove WatchedCounter instances for the counters that are no longer accessible via any of the lists. This can be done by maintaining a generation counter in CounterListNode, incrementing it at each call to UpdateState() remembering the last update generation for each WatchedCounter and cleaning up the ones that were not updated recently:
lock (_DynamicCounters) foreach (var kv in _DynamicCounters.ToArray()) if (kv.Value.Generation != _Generation) { kv.Value.NextField?.Dispose(); _DynamicCounters.Remove(kv.Key); }
lock (_DynamicCounters) foreach (var kv in _DynamicCounters.ToArray()) if (kv.Value.Generation != _Generation) { kv.Value.NextField?.Dispose(); _DynamicCounters.Remove(kv.Key); }

You can find a detailed documentation on the Live Watch plugin API here, and the final version of the code shown in this tutorial series in this repository.