{"id":8109,"date":"2023-04-04T10:54:59","date_gmt":"2023-04-04T17:54:59","guid":{"rendered":"https:\/\/visualgdb.com\/w\/?p=8109"},"modified":"2023-04-04T11:00:20","modified_gmt":"2023-04-04T18:00:20","slug":"parsing-lists-of-objects-in-live-watch-plugins","status":"publish","type":"post","link":"https:\/\/visualgdb.com\/tutorials\/extensibility\/livewatch\/lists\/","title":{"rendered":"Parsing Lists of Objects in Live Watch Plugins"},"content":{"rendered":"<p>This tutorial shows how to use the Live Watch plugins to discover and show all variables of a particular type, whether created statically or dynamically. We will extend a basic Live Watch plugin from <a href=\"https:\/\/visualgdb.com\/tutorials\/extensibility\/livewatch\/nodes\/\">this tutorial<\/a> that locates the <strong>g_Counter<\/strong> variable and creates a custom Live Watch node for it. Instead of looking for a particular variable, we will create the nodes for all variables of the <strong>SampleCounter<\/strong> type, and even discover dynamically allocated instances by traversing linked lists.<\/p>\n<p>Before you begin, clone <a href=\"https:\/\/github.com\/sysprogs\/VisualGDBExtensibilityExamples\/tree\/tutorials\/LiveWatchPlugin\/counter-done\">this git tag<\/a>, or simply follow our <a href=\"https:\/\/visualgdb.com\/tutorials\/extensibility\/livewatch\/nodes\/\">previous tutorial<\/a> to get a basic live watch plugin working.<\/p>\n<ol>\n<li>Open the <strong>SampleLiveWatchExtension<\/strong> project from <a href=\"https:\/\/visualgdb.com\/tutorials\/extensibility\/livewatch\/nodes\/\">this tutorial<\/a>. Locate the <strong>CounterListNode.GetChildren()<\/strong> method that creates a node for <strong>g_Counter<\/strong>, and replace it with the code shown below:\n<pre class=\"\">var result = new List&lt;ILiveWatchNode&gt;();\r\n\r\nforeach(var v in _Engine.Symbols.TopLevelVariables)\r\n{\r\n\tif (v.RawType.Resolved == \"SampleCounter\")\r\n\t{\r\n\t\tresult.Add(new CounterNode(_Engine, v));\r\n\t}\r\n}\r\n\r\nreturn result.ToArray();<\/pre>\n<p><a href=\"https:\/\/visualgdb.com\/w\/wp-content\/uploads\/2023\/04\/02-code.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-8111\" src=\"https:\/\/visualgdb.com\/w\/wp-content\/uploads\/2023\/04\/02-code.png\" alt=\"\" width=\"1226\" height=\"843\" \/><\/a>Note how we are iterating through all global\/static variables, checking their resolved type (i.e. resolving typedefs), and creating nodes for all variables that have a matching type.<\/li>\n<li>Press F5 to start debugging the plugin. When the second instance of Visual Studio launches, open the <strong>SampleLiveWatchProject<\/strong> project in it, rename <strong>g_Counter<\/strong> to <strong>g_Counter1<\/strong> and add a <strong>g_Counter2<\/strong> variable next to it. Make sure you assign the <strong>Name<\/strong> field for both of them: <a href=\"https:\/\/visualgdb.com\/w\/wp-content\/uploads\/2023\/04\/01-multi.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-8110\" src=\"https:\/\/visualgdb.com\/w\/wp-content\/uploads\/2023\/04\/01-multi.png\" alt=\"\" width=\"1226\" height=\"843\" \/><\/a><\/li>\n<li>Start debugging the embedded project. Note how the <strong>Live Watch<\/strong> window now shows both <strong>Test counter #1<\/strong> and <strong>Test Counter #2<\/strong>:<a href=\"https:\/\/visualgdb.com\/w\/wp-content\/uploads\/2023\/04\/03-staticlist.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-8112\" src=\"https:\/\/visualgdb.com\/w\/wp-content\/uploads\/2023\/04\/03-staticlist.png\" alt=\"\" width=\"1226\" height=\"843\" \/><\/a> You can use the same technique to discover and present objects of various types. The <a href=\"https:\/\/visualgdb.com\/reference\/extensibility\/html\/M_VisualGDBExtensibility_LiveWatch_ILiveWatchNode_GetChildren.htm\">GetChildren()<\/a> method is normally called only once when the node is expanded for the first time, so you can easily iterate through all global variables in it, find the relevant ones, and create meaningful nodes for them.<\/li>\n<li>Now we will show how to handle dynamic lists. Add a &#8220;<strong>SampleCounter *Next<\/strong>&#8221; field to the <strong>SampleCounter<\/strong> struct, and create a few more instances of it dynamically:\n<pre class=\"\">    g_Counter1.Next = (SampleCounter *)malloc(sizeof(SampleCounter));\r\n    g_Counter1.Next-&gt;Name = \"Dynamic counter #1\";\r\n\r\n    g_Counter1.Next-&gt;Next = (SampleCounter *)malloc(sizeof(SampleCounter));\r\n    g_Counter1.Next-&gt;Next-&gt;Name = \"Dynamic counter #2\";\r\n\r\n    g_Counter1.Next-&gt;Next-&gt;Next = NULL;<\/pre>\n<p><a href=\"https:\/\/visualgdb.com\/w\/wp-content\/uploads\/2023\/04\/04-dynamic.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-8113\" src=\"https:\/\/visualgdb.com\/w\/wp-content\/uploads\/2023\/04\/04-dynamic.png\" alt=\"\" width=\"1226\" height=\"843\" \/><\/a><\/li>\n<li>We will now modify the <strong>CounterListNode<\/strong> class to dynamically parse the linked lists of counters, and show them in the child list. In order to do that, we would need to:\n<ul>\n<li>Remember the list of statically defined counters, so that we could traverse the linked lists starting at them.<\/li>\n<li>Locate the definition of the <strong>SampleCounter<\/strong> type, so that we could <a href=\"https:\/\/visualgdb.com\/reference\/extensibility\/html\/M_VisualGDBExtensibility_LiveWatch_ILiveSymbolParser_CreateTypedVariable.htm\">dynamically create pinned variables<\/a> of that type.<\/li>\n<li>Get the <a href=\"https:\/\/visualgdb.com\/reference\/extensibility\/html\/M_VisualGDBExtensibility_LiveWatch_IPinnedVariableStructType_LookupMember.htm\">exact offset and size<\/a> of the <strong>Next<\/strong> field, so that we could read it in order to traverse the lists.<\/li>\n<\/ul>\n<p>Update the <strong>GetChildren()<\/strong> method as shown below:<\/p>\n<pre class=\"\">List&lt;CounterNode&gt; _StaticCounters = new List&lt;CounterNode&gt;();\r\n\r\nIPinnedVariableStructType _CounterStruct;\r\nIPinnedVariableStructMember _NextField;\r\n\r\npublic ILiveWatchNode[] GetChildren(LiveWatchChildrenRequestReason reason)\r\n{\r\n\tvar result = new List&lt;ILiveWatchNode&gt;();\r\n\r\n\tforeach (var v in _Engine.Symbols.TopLevelVariables)\r\n\t{\r\n\t\tif (v.RawType.Resolved == \"SampleCounter\")\r\n\t\t\t_StaticCounters.Add(new CounterNode(_Engine, v));\r\n\t}\r\n\r\n\t_CounterStruct = _Engine.Symbols.LookupType(\"SampleCounter\") as IPinnedVariableStructType;\r\n\t_NextField = _CounterStruct?.LookupMember(\"Next\", false);\r\n\r\n\treturn result.ToArray();\r\n}\r\n\r\n<\/pre>\n<p><a href=\"https:\/\/visualgdb.com\/w\/wp-content\/uploads\/2023\/04\/05-list.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-8114\" src=\"https:\/\/visualgdb.com\/w\/wp-content\/uploads\/2023\/04\/05-list.png\" alt=\"\" width=\"1226\" height=\"843\" \/><\/a>Note that as GetChildren() only gets called once per node, it doesn&#8217;t parse the dynamic lists, but only loads the data needed to parse them later.<\/li>\n<li>Add a &#8220;<strong>public ulong Address =&gt; _Counter.Address;<\/strong>&#8221; property to the <strong>CounterNode<\/strong> class, so that we could get the addresses from the list of <strong>CounterNode<\/strong>-s:<a href=\"https:\/\/visualgdb.com\/w\/wp-content\/uploads\/2023\/04\/06-addr.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-8115\" src=\"https:\/\/visualgdb.com\/w\/wp-content\/uploads\/2023\/04\/06-addr.png\" alt=\"\" width=\"1226\" height=\"843\" \/><\/a><\/li>\n<li>The most straight-forward implementation of dynamic list parsing would be to just recursively read the <strong>Next<\/strong> field of each counter, starting at the top-level nodes, and create instances of <strong>CounterNode<\/strong> as needed. However, it would create 2 performance issues:\n<ul>\n<li>Reading each pointer separately would introduce considerable delay due to latency.<\/li>\n<li>Creating the node objects at each refresh would cause unnecessary updates of the Live Watch window.<\/li>\n<\/ul>\n<p>We will address these issues by maintaining a cache of dynamically watched counters. For each target memory address that contains a counter, we will keep:<\/p>\n<ul>\n<li>A <a href=\"https:\/\/visualgdb.com\/reference\/extensibility\/html\/T_VisualGDBExtensibility_LiveWatch_ILiveVariable.htm\">Live Variable<\/a> corresponding to its <strong>Next<\/strong> field, so that VisualGDB can read it ahead of time, together with other live variables.<\/li>\n<li>A <a href=\"https:\/\/visualgdb.com\/reference\/extensibility\/html\/T_VisualGDBExtensibility_LiveWatch_ILiveWatchNode.htm\">Live Watch Node<\/a> (actually, our own <strong>CounterNode<\/strong> class) corresponding to that variable.<\/li>\n<\/ul>\n<p>To do this, define a <strong>WatchedCounter<\/strong> class inside the <strong>CounterListNode<\/strong> class:<\/p>\n<pre class=\"\">class WatchedCounter\r\n{\r\n\tpublic readonly ILiveVariable NextField;\r\n\tpublic readonly CounterNode Node;\r\n\r\n\tpublic WatchedCounter(CounterListNode list, ulong addr)\r\n\t{\r\n\t\tNextField = list._Engine.Memory.CreateLiveVariable(addr + list._NextField.Offset, list._NextField.Size, $\"[{addr:x8}]\");\r\n\t\tvar tv = list._Engine.Symbols.CreateTypedVariable(addr, list._CounterStruct);\r\n\r\n\t\tif (tv != null)\r\n\t\t\tNode = new CounterNode(list._Engine, tv);\r\n\t}\r\n\r\n\tpublic bool IsValid =&gt; NextField != null &amp;&amp; Node != null;\r\n}\r\n\r\n<\/pre>\n<\/li>\n<li>Update the <strong>UpdateState()<\/strong> method as shown below:\n<pre class=\"\">        public LiveWatchNodeState UpdateState(LiveWatchUpdateContext context)\r\n        {\r\n            List&lt;ILiveWatchNode&gt; children = new List&lt;ILiveWatchNode&gt;();\r\n            children.AddRange(_StaticCounters);\r\n\r\n            if (_NextField != null)\r\n            {\r\n                foreach (var head in _StaticCounters)\r\n                {\r\n                    ulong addr, nextAddr;\r\n                    for (addr = head.Address; addr != 0; addr = nextAddr)\r\n                    {\r\n                        if (!_DynamicCounters.TryGetValue(addr, out WatchedCounter counter))\r\n                            _DynamicCounters[addr] = counter = new WatchedCounter(this, addr);\r\n\r\n                        if (!counter.IsValid)\r\n                            break;\r\n\r\n                        if (addr != head.Address) \r\n                            children.Add(counter.Node);\r\n\r\n                        nextAddr = counter.NextField.GetValue().ToUlong();\r\n                    }\r\n                }\r\n            }\r\n\r\n            return new LiveWatchNodeState\r\n            {\r\n                Value = \"Changed\",\r\n                NewChildren = children.Count == 0 ? null : children.ToArray(),\r\n            };\r\n        }<\/pre>\n<p>This traverses the lists starting at each statically discovered instance (<strong>_StaticCounters<\/strong>), creates the <strong>WatchedCounter<\/strong> instances for each item in the list (including the very first ones), and returns the corresponding nodes\u00a0 via the <a href=\"https:\/\/visualgdb.com\/reference\/extensibility\/html\/F_VisualGDBExtensibility_LiveWatch_LiveWatchNodeState_NewChildren.htm\">NewChildren<\/a> field.<br \/>\nNote that if the node was never expanded, its <strong>GetChildren()<\/strong> method would never get called, so <strong>_NextField<\/strong> would be <strong>null<\/strong> and no lists will be traversed. If this is the case, we set <strong>NewChildren<\/strong> to <strong>null<\/strong>, to get VisualGDB a chance to call <strong>GetChildren()<\/strong> when the node is first expanded.<a href=\"https:\/\/visualgdb.com\/w\/wp-content\/uploads\/2023\/04\/07-dyn.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-8116\" src=\"https:\/\/visualgdb.com\/w\/wp-content\/uploads\/2023\/04\/07-dyn.png\" alt=\"\" width=\"1226\" height=\"843\" \/><\/a><\/li>\n<li>Run the embedded program and see how the dynamically discovered counters are now also shown in the list:<a href=\"https:\/\/visualgdb.com\/w\/wp-content\/uploads\/2023\/04\/08-dynlist-1.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-8124\" src=\"https:\/\/visualgdb.com\/w\/wp-content\/uploads\/2023\/04\/08-dynlist-1.png\" alt=\"\" width=\"1226\" height=\"843\" \/><\/a><\/li>\n<li>Try collapsing the &#8220;<strong>Counters<\/strong>&#8221; node in the Live Watch view. Note how despite not needing to parse the counter lists, VisualGDB still tries to read 4 live variables at each refresh cycle. These are the variables for the <strong>Next<\/strong> field held by the <strong>WatchedCounter<\/strong> instances:<a href=\"https:\/\/visualgdb.com\/w\/wp-content\/uploads\/2023\/04\/09-4vars-1.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-8125\" src=\"https:\/\/visualgdb.com\/w\/wp-content\/uploads\/2023\/04\/09-4vars-1.png\" alt=\"\" width=\"1226\" height=\"843\" \/><\/a><\/li>\n<li>We can prevent them from being updated if the <strong>Counters<\/strong> node is fully scrolled out or suspended by implementing the <strong>SetSuspendState()<\/strong> method in <strong>CounterListNode<\/strong>:\n<pre class=\"\">public void SetSuspendState(LiveWatchNodeSuspendState state)\r\n{\r\n\tlock (_DynamicCounters)\r\n\t\tforeach (var c in _DynamicCounters.Values)\r\n\t\t\tif (c.IsValid)\r\n\t\t\t\tc.NextField.SuspendUpdating = state.SuspendRegularUpdates || !state.IsExpanded;\r\n}<\/pre>\n<p>You would also need to wrap the <strong>if (_NextField != null) {&#8230;}<\/strong> block with another &#8220;<strong>lock (_DynamicCounters) {}<\/strong>&#8221; to avoid race conditions between <strong>SetSuspendState()<\/strong> and <strong>UpdateState()<\/strong>. The <strong>lock()<\/strong> syntax is a shorthand for acquiring a mutex associated with <strong>_DynamicCounters<\/strong> at the beginning, and releasing it at the end of the block. You could also use a mutex explicitly, or lock on any other .Net object, as long as both methods lock on the same object, and no other method tries to lock on it for a different purpose.<a href=\"https:\/\/visualgdb.com\/w\/wp-content\/uploads\/2023\/04\/10-suspend.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-8119\" src=\"https:\/\/visualgdb.com\/w\/wp-content\/uploads\/2023\/04\/10-suspend.png\" alt=\"\" width=\"1226\" height=\"843\" \/><\/a><\/li>\n<li>Now collapsing the <strong>Counters<\/strong> node will automatically prevent the list pointers from being updated, freeing the memory access bandwidth for other tasks:<a href=\"https:\/\/visualgdb.com\/w\/wp-content\/uploads\/2023\/04\/11-zero.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-8120\" src=\"https:\/\/visualgdb.com\/w\/wp-content\/uploads\/2023\/04\/11-zero.png\" alt=\"\" width=\"1226\" height=\"843\" \/><\/a>One last improvement would be to automatically remove <strong>WatchedCounter<\/strong> instances for the counters that are no longer accessible via any of the lists. This can be done by maintaining a generation counter in <strong>CounterListNode<\/strong>, incrementing it at each call to <strong>UpdateState()<\/strong> remembering the last update generation for each <strong>WatchedCounter<\/strong> and cleaning up the ones that were not updated recently:\n<pre class=\"\">lock (_DynamicCounters)\r\n\tforeach (var kv in _DynamicCounters.ToArray())\r\n\t\tif (kv.Value.Generation != _Generation)\r\n\t\t{\r\n\t\t\tkv.Value.NextField?.Dispose();\r\n\t\t\t_DynamicCounters.Remove(kv.Key);\r\n\t\t}<\/pre>\n<p><a href=\"https:\/\/visualgdb.com\/w\/wp-content\/uploads\/2023\/04\/12-remove-2.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-8127\" src=\"https:\/\/visualgdb.com\/w\/wp-content\/uploads\/2023\/04\/12-remove-2.png\" alt=\"\" width=\"1226\" height=\"843\" \/><\/a><\/li>\n<\/ol>\n<p>You can find a detailed documentation on the Live Watch plugin API <a href=\"https:\/\/visualgdb.com\/reference\/extensibility\/html\/N_VisualGDBExtensibility_LiveWatch.htm\">here<\/a>, and the final version of the code shown in this tutorial series in <a href=\"https:\/\/github.com\/sysprogs\/VisualGDBExtensibilityExamples\/tree\/master\/LiveWatchPlugins\/LiveWatchPluginTutorial\">this repository<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>This tutorial shows how to use the Live Watch plugins to discover and show all variables of a particular type,<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[26],"tags":[219,110,61],"_links":{"self":[{"href":"https:\/\/visualgdb.com\/w\/wp-json\/wp\/v2\/posts\/8109"}],"collection":[{"href":"https:\/\/visualgdb.com\/w\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/visualgdb.com\/w\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/visualgdb.com\/w\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/visualgdb.com\/w\/wp-json\/wp\/v2\/comments?post=8109"}],"version-history":[{"count":5,"href":"https:\/\/visualgdb.com\/w\/wp-json\/wp\/v2\/posts\/8109\/revisions"}],"predecessor-version":[{"id":8132,"href":"https:\/\/visualgdb.com\/w\/wp-json\/wp\/v2\/posts\/8109\/revisions\/8132"}],"wp:attachment":[{"href":"https:\/\/visualgdb.com\/w\/wp-json\/wp\/v2\/media?parent=8109"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/visualgdb.com\/w\/wp-json\/wp\/v2\/categories?post=8109"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/visualgdb.com\/w\/wp-json\/wp\/v2\/tags?post=8109"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}