WPF : Dynamic Search Driven List Results
At work at the moment I am working on a way way cool customisable search that basically allows user to pick source entities and related entities and then pick what fields they would like to show, this is similar to Microsoft’s CRM Dynamics product, which allows very very sophisticated searches to be produces, by the use of a nice search UI.
Here is what CRM looks like. The search we actually built at work is even better than this, but for NDA reasons I can not show.
From our search we create a strongly typed Query object, which is sent across a WCF service boundary and when received at the other end, in converted to dynamic SQL, and is the run against a SQL database. Yes that right we created our own LINQ effectively. Its call GNRSQL.
Anyway as a side effect from being able to search for anything from anywhere, we also needed the ability for our search results grid to dynamically adjust to any results set. To compound this our results objects are hierarchical in nature, and we are using the Infragistics XamDataGrid, which wants to show things in a hierarchical manner. Which is not what we wanted, so we needed to flatten the results from a hierarchy to a flat structure.
Your 1st thought might be, oh just send a DataTable across the WCF boundary, this is not a good idea, the serialization of DataTable(s) and WCF is very strange, and also incredibly heavy. Basically the namespaces get messed up and so do some of the internal field names, and my god the serialization is so far from pretty.
So we had to try some other things.
My next idea was to use IEnumerable and use LINQ to pluck out only those properties that we wanted to show based on the actually results obtained. Our results contained metadata that we could use to construct a LINQ query which would return a flattened IEnumerable of anonymous types, that had only the properties we wanted. Choosing of properties was driven by examining the actual results metadata. And all this has to happen at runtime. Doing this at design time is dead simple we can just do something like (assuming we have a list of results already)
var newFlattenedResults = (from x in results select new { ID = x.IDField, DateOfOrder=x.OrderDate });
But how could you create something that could be used in this manner, and tailored to suit the properties returned by the search, but done at runtime. The rest of this post will show you how.
Dynamic Assemblies
.NET comes equipped with the ability to produce dynamic assemblies on the fly at runtime. So lets start there and have a look at a helper class for making this process easy.
1: using System.Reflection;
2: using System.CodeDom.Compiler;
3: using Microsoft.CSharp;
4:
5: using System;
6: using System.Collections.Generic;
7: using System.Text;
8:
9: namespace Dyno
10: {
11: /// <summary>
12: /// Utlilty class. Compliles and runs assemblies
13: /// on the fly, with the option to invoke a method
14: /// and return the results from the chosen
15: /// method
16: /// </summary>
17: public class DynamicCompiler
18: {
19: #region Public Methods
20: /// <summary>
21: /// Compiles the code and either returns the compiled code as a Type
22: /// or creates compiled code as a Type, and invokes the created Types
23: /// method. Where the method is picked by the user
24: /// </summary>
25: /// <typeparam name="T">Type of T to use</typeparam>
26: /// <param name="code">code to compile</param>
27: /// <param name="nameSpace">namespace to use for compiled code</param>
28: /// <param name="classToLoad">class name</param>
29: /// <param name="methodToRun">method to invoke (optional)</param>
30: /// <param name="ShouldImvokeMethod">true if you want to
31: /// invoke the method</param>
32: /// <returns>Type of T to use</returns>
33: public T ComplileAndRun<T>(String code, String nameSpace,
34: String classToLoad, string methodToRun, Boolean ShouldImvokeMethod)
35: {
36: try
37: {
38: String lcCode = code;
39:
40: var provider = new CSharpCodeProvider(
41: new Dictionary<String, String>()
42: { { "CompilerVersion", "v3.5" } });
43:
44: CompilerParameters parameters = new CompilerParameters();
45:
46: // Start by adding any referenced assemblies
47: parameters.ReferencedAssemblies.Add("System.dll");
48: parameters.ReferencedAssemblies.Add(
49: typeof(Demo.Data.Person).Assembly.Location);
50: parameters.ReferencedAssemblies.Add(
51: typeof(System.Linq.Enumerable).Assembly.Location);
52:
53:
54: // Load the resulting assembly into memory
55: parameters.GenerateInMemory = true;
56: // Now compile the whole thing
57: //Must create a fully functional assembly as the code string
58: CompilerResults compiledCode =
59: provider.CompileAssemblyFromSource(parameters, lcCode);
60:
61: if (compiledCode.Errors.HasErrors)
62: {
63: String errorMsg = String.Empty;
64: errorMsg = compiledCode.Errors.Count.ToString() +
65: " n Dynamically generated code threw an error. n Errors:";
66:
67: for (int x = 0; x < compiledCode.Errors.Count; x++)
68: {
69: errorMsg = errorMsg + "rnLine: " +
70: compiledCode.Errors[x].Line.ToString() + " - " +
71: compiledCode.Errors[x].ErrorText;
72: }
73:
74: throw new Exception(errorMsg);
75: }
76:
77: Assembly assembly = compiledCode.CompiledAssembly;
78:
79: // Retrieve an obj ref – generic type only
80: object instance = assembly.CreateInstance(
81: nameSpace + "." + classToLoad);
82:
83: //invoke the method if needs be, and get results
84: //back from method invocation
85: if (ShouldImvokeMethod)
86: {
87: if (instance == null)
88: return default(T);
89:
90: T result = (T)instance.GetType().InvokeMember(
91: methodToRun, BindingFlags.InvokeMethod,
92: null, instance, new object[0]);
93:
94: return result;
95: }
96: else
97: {
98: return (T)instance;
99: }
100: }
101: catch (Exception ex)
102: {
103: Console.WriteLine(String.Format(
104: "An exception occurred {0}", ex.Message));
105: return default(T);
106: }
107: }
108: #endregion
109: }
110: }
So using this helper class we can construct an in memory assembly and call a method within an object within it, or simply return the newly created object within the dynamic assembly.
So lets continue our journey, and look at a small XAML app, where there is a ListView that starts with the following columns
Where the XAML for this screen looks like this
1: <Window x:Class="DynamicLINQ.Window1"
2: xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
3: xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
4: WindowStartupLocation="CenterScreen"
5: Title="Window1" Height="300" Width="700">
6: <DockPanel LastChildFill="True">
7: <StackPanel DockPanel.Dock="Top" Orientation="Horizontal"
8: HorizontalAlignment="Stretch" Background="CornflowerBlue">
9: <Button x:Name="ShowAll" Content="Show All" Margin="5"
10: Click="ShowAll_Click"/>
11: <Button x:Name="ShowSome" Content="Show Some Columns" Margin="5"
12: Click="ShowSome_Click"/>
13: </StackPanel>
14:
15:
16: <ListView x:Name="lvItems" >
17: <ListView.View>
18: <GridView>
19: <GridViewColumn Header="Age"
20: DisplayMemberBinding="{Binding Age}" />
21: <GridViewColumn Header="FirstName"
22: DisplayMemberBinding="{Binding FirstName}" />
23: <GridViewColumn Header="MiddleName"
24: DisplayMemberBinding="{Binding MiddleName}" />
25: <GridViewColumn Header="LastName"
26: DisplayMemberBinding="{Binding LastName}" />
27: <GridViewColumn Header="LastName"
28: DisplayMemberBinding="{Binding LastName}" />
29: <GridViewColumn Header="ID"
30: DisplayMemberBinding="{Binding ID}" Width="230" />
31: <GridViewColumn Header="DOB"
32: DisplayMemberBinding="{Binding Dob}" Width="130" />
33: </GridView>
34: </ListView.View>
35: </ListView>
36:
37:
38: </DockPanel>
39: </Window>
You can see that the initial columns for the displayed results are static at this point, and are showing the results of being bound to a List<Person> objects, where a Person looks like this
1: using System;
2: using System.Collections.Generic;
3: using System.Linq;
4: using System.Text;
5: using System.ComponentModel;
6:
7: namespace Demo.Data
8: {
9: public class Person : INotifyPropertyChanged
10: {
11: #region Data
12: public Int32 age;
13: public String firstName;
14: public String middleName;
15: public String lastName;
16: public AdditionalData personData;
17: #endregion
18:
19: #region Public Properties
20: public Int32 Age
21: {
22: get { return age; }
23: set
24: {
25: age = value;
26: NotifyPropertyChanged("Age");
27: }
28: }
29:
30: public String FirstName
31: {
32: get { return firstName; }
33: set
34: {
35: firstName = value;
36: NotifyPropertyChanged("FirstName");
37: }
38: }
39:
40: public String MiddleName
41: {
42: get { return middleName; }
43: set
44: {
45: middleName = value;
46: NotifyPropertyChanged("MiddleName");
47: }
48: }
49:
50: public String LastName
51: {
52: get { return lastName; }
53: set
54: {
55: lastName = value;
56: NotifyPropertyChanged("LastName");
57: }
58: }
59:
60: public AdditionalData PersonData
61: {
62: get { return personData; }
63: set
64: {
65: personData = value;
66: NotifyPropertyChanged("PersonData");
67: }
68: }
69: #endregion
70:
71: #region INotifyPropertyChanged region
72: public event PropertyChangedEventHandler PropertyChanged;
73:
74: private void NotifyPropertyChanged(String info)
75: {
76: if (PropertyChanged != null)
77: {
78: PropertyChanged(this, new PropertyChangedEventArgs(info));
79: }
80: }
81: #endregion
82: }
83:
84: public class AdditionalData : INotifyPropertyChanged
85: {
86: #region Data
87: private Guid id;
88: private DateTime dob;
89: #endregion
90:
91: #region Public Properties
92: public Guid ID
93: {
94: get { return id; }
95: set
96: {
97: id = value;
98: NotifyPropertyChanged("Id");
99: }
100: }
101:
102: public DateTime Dob
103: {
104: get { return dob; }
105: set
106: {
107: dob = value;
108: NotifyPropertyChanged("Dob");
109: }
110: }
111: #endregion
112:
113: #region INotifyPropertyChanged region
114: public event PropertyChangedEventHandler PropertyChanged;
115:
116: private void NotifyPropertyChanged(String info)
117: {
118: if (PropertyChanged != null)
119: {
120: PropertyChanged(this, new PropertyChangedEventArgs(info));
121: }
122: }
123: #endregion
124: }
125:
126: }
So now let us see what would happen if we simulated some other search results coming back, from somewhere else, that should alter what is shown in the search results ListView.
1: private void ShowSome_Click(object sender, RoutedEventArgs e)
2: {
3: DataLoader dl = new DataLoader();
4:
5: //show some columns from Dynamically compiled LINQ
6: //query
7: IEnumerable results =
8: dl.ObtainFlattenedResults(lvItems,
9: people, WritePropertyType.Some);
10:
11:
12: lvItems.ItemsSource = results;
13:
14: }
Where people is simply a list of Person objects set up with some Person objects in it (obviously I am faking the search results part, for the sake of this article, the Person objects, would actually be some search results or something coming from the results of a search, but for this example it doesn’t matter, as I am just trying to show you how to dynamically work with LINQ at runtime)
1: private List<Person> people = new List<Person>();
You can see from the above code that we use a method called ObtainFlattenedResults() which is available in the following helper class
1: using System;
2: using System.Collections;
3: using System.Collections.Generic;
4: using System.Linq;
5: using System.Reflection;
6: using System.Text;
7:
8: using Demo.Data;
9: using System.Windows.Controls;
10: using System.Windows.Data;
11:
12:
13: namespace Dyno
14: {
15:
16: public enum WritePropertyType { All = 1, Some };
17:
18: public class EnumerableResultsCreator
19: {
20: #region Data
21: private readonly String generatedNamespace = "DynamicLINQ";
22: private readonly String generatedClassName = "ResultsWriter";
23: private readonly String generatedMethod = "GetResults";
24: private List<Person> originalResults = null;
25: private WritePropertyType currentPropertyType =
26: WritePropertyType.All;
27: #endregion
28:
29: #region Ctor
30: public EnumerableResultsCreator()
31: {
32:
33: }
34: #endregion
35:
36: #region Private Methods
37:
38: /// <summary>
39: /// Writes out code for class to dynamically compile
40: /// </summary>
41: private String WriteDynamicClass(WritePropertyType currentPropertyType)
42: {
43: StringBuilder code = new StringBuilder();
44: code.AppendLine(" using System; ");
45: code.AppendLine(" using System.Collections.Generic;");
46: code.AppendLine(" using System.Collections.ObjectModel;");
47: code.AppendLine(" using System.Linq;");
48: code.AppendLine(" using System.Text;");
49: //Where the DataLayer objects live
50: code.AppendLine(" using Demo.Data; ");
51: code.AppendFormat(" namespace {0}", generatedNamespace);
52: code.AppendLine();
53: code.AppendLine(" {");
54: code.AppendFormat(" public class {0}", generatedClassName);
55: code.AppendLine();
56: code.AppendLine(" {");
57: code.AppendFormat("{0}",
58: this.WriteCodeProperties(currentPropertyType));
59: code.AppendLine();
60: code.Append(" }");
61: code.AppendLine();
62: code.Append(" }");
63: code.AppendLine();
64: return code.ToString();
65: }
66:
67:
68: /// <summary>
69: /// Either writes a LINQ query string that returns an anomomous
70: /// set of object with ALL properties, or just some of the properties
71: /// </summary>
72: private String WriteCodeProperties(WritePropertyType currentPropertyType)
73: {
74: StringBuilder builder = new StringBuilder();
75: builder.AppendFormat(
76: "n public Object {0}(List<Person> results)",
77: generatedMethod);
78: builder.AppendLine("{");
79:
80:
81: switch (currentPropertyType)
82: {
83: case WritePropertyType.All:
84: //OK this is a static query that could have been done in standard LINQ
85: //but this techniqe can be used to create dyanamically created
86: //LINQ queries at runtime
87: builder.AppendLine("var x =(from r in results select new { ");
88: builder.AppendLine(" Age = r.Age,FirstName = r.FirstName,");
89: builder.AppendLine(" MiddleName=r.MiddleName, LastName = r.LastName,");
90: builder.AppendLine(" ID = r.PersonData.ID, Dob = r.PersonData.Dob });");
91: builder.AppendLine("return x;");
92: builder.AppendLine("}");
93: break;
94: case WritePropertyType.Some:
95: //OK this is a static query that could have been done in standard LINQ
96: //but this techniqe can be used to create dyanamically created
97: //LINQ queries at runtime
98: builder.AppendLine("var x =(from r in results select new { ");
99: builder.AppendLine(" Age = r.Age,FirstName = r.FirstName,");
100: builder.AppendLine(" LastName = r.LastName});");
101: builder.AppendLine("return x;");
102: builder.AppendLine("}");
103: break;
104: }
105:
106:
107: return builder.ToString();
108:
109: }
110:
111: /// <summary>
112: /// Creates the new columns for the Dynamically retrieved
113: /// results
114: /// </summary>
115: private void CreateListViewColumns(IEnumerable result, ListView lv)
116: {
117: GridView gv = new GridView();
118:
119: IEnumerator e = result.GetEnumerator();
120: while (e.MoveNext())
121: {
122: Type t = e.Current.GetType();
123: PropertyInfo[] infos = t.GetProperties();
124:
125: foreach (var item in infos)
126: {
127: GridViewColumn column = new GridViewColumn();
128: column.Header = item.Name;
129: Binding binding = new Binding(item.Name);
130:
131: column.DisplayMemberBinding = binding;
132: gv.Columns.Add(column);
133: }
134: break;
135: }
136:
137: lv.View = gv;
138: }
139:
140:
141: #endregion
142:
143: #region Public Method
144: /// <summary>
145: /// Create the dynamic Type invoke its method
146: /// and get the results
147: /// </summary>
148: public IEnumerable ObtainFlattenedResults(
149: ListView lv,
150: List<Person> originalResults,
151: WritePropertyType currentPropertyType)
152: {
153: if (originalResults.Count == 0)
154: return null;
155:
156: this.originalResults = originalResults;
157:
158: //Create the DynamicCompiler
159: DynamicCompiler compiler = new DynamicCompiler();
160:
161: //Get the newly compiled object
162: Object dynaClass = compiler.ComplileAndRun<Object>(
163: this.WriteDynamicClass(currentPropertyType),
164: generatedNamespace, generatedClassName, "", false);
165:
166: //invoke its method to get the result
167: IEnumerable result =
168: (IEnumerable)dynaClass.GetType().InvokeMember(
169: generatedMethod, BindingFlags.InvokeMethod,
170: null, dynaClass, new object[1]
171: { this.originalResults });
172:
173: //create new ListView columns
174: CreateListViewColumns(result, lv);
175:
176:
177: return result;
178:
179: }
180: #endregion
181:
182: }
183: }
So what this class does, is construct a new code file, that is compiled using the DynamicCompiler helper class that I showed you earlier. What we then do is get an actual instance of an object back from the dynamic compilation process, which we can then use to invoke a method on.
Here is what the compiled code will look like
1: using System;
2: using System.Collections.Generic;
3: using System.Collections.ObjectModel;
4: using System.Linq;
5: using System.Text;
6: using Demo.Data;
7: namespace DynoGen
8: {
9: public class ResultsWriter
10: {
11:
12: public Object GetResults(List<Person> results)
13: {
14: var x = (from r in results
15: select new
16: {
17: Age = r.Age,
18: FirstName = r.FirstName,
19: LastName = r.LastName
20: });
21: return x;
22: }
23:
24: }
25: }
But Beware
As we are effectivly creating a new dynamically created Dll in memory in our current AppDomain this could lead to the current AppDomain getting bigger and bigger. So it is a good idea to do all this in a new AppDomain, but how do we do that?
Well that is actually pretty simple, we can do this
1: using System;
2: using System.Collections.Generic;
3: using System.Linq;
4: using System.Text;
5:
6: using Demo.Data;
7: using System.Windows.Controls;
8: using System.Collections;
9: using System.IO;
10: using System.Runtime.Remoting;
11: using System.Reflection;
12:
13: namespace DynamicLINQ
14: {
15: /// <summary>
16: /// Obtains dynamic results using a seperate AppDomain that is
17: /// unloaded when the work is finished with the Dyanamic Assembly
18: /// </summary>
19: public class DataLoader
20: {
21: public IEnumerable ObtainFlattenedResults(
22: ListView lv,
23: List<Person> originalResults,
24: WritePropertyType currentPropertyType)
25: {
26:
27: //Create new app domain
28: AppDomain domain = AppDomain.CreateDomain("ResultsDomain");
29: IEnumerable results = null;
30: Object instance = null;
31:
32:
33: try
34: {
35:
36: //load search assembly
37: domain.Load(File.ReadAllBytes(String.Format(@"{0}{1}",
38: Environment.CurrentDirectory,
39: "Dyno.dll")));
40:
41: //Get the EnumerableResultsCreator instance
42: ObjectHandle instanceHandle = domain.CreateInstance("Dyno",
43: String.Format("{0}.{1}"
44: , "Dyno"
45: , "EnumerableResultsCreator"));
46:
47: if (instanceHandle != null)
48: instance = instanceHandle.Unwrap();
49:
50: //call method to get IEnumerable results
51: if (instance != null)
52: results = instance.GetType().InvokeMember
53: ("ObtainFlattenedResults",
54: BindingFlags.InvokeMethod,
55: null,
56: instance,
57: new object[3] {
58: lv
59: , originalResults
60: , currentPropertyType }
61: ) as IEnumerable;
62: }
63: catch (Exception e)
64: {
65: Console.WriteLine("Problem constructing the " +
66: "EnumerableResultsCreator instance");
67: }
68: finally
69: {
70: AppDomain.Unload(domain);
71: }
72: return results;
73: }
74: }
75: }
This takes care of managing a new AppDomain and gets the results from the dynamic assembly, and then unloads the new AppDomain. This ensures the primary applications AppDomain doesn’t get any unwanted code in it.
So you can see that we are creating an actual class that accepts a List<Person> (the fake search results) and uses some LINQ to grab out ONLY the properties we want to use all rolled up in an anonymous which is selected as a single flattened search result for the search results grid. Remember this is all happening at RUNTIME, so we can do what we like, we can manipulate the results as much as we like, as the code we compile is just a String.
The other thing that happens is that the WPF ListView columns are altered to only show the necessary columns based on the newly flattened search results.
Some grid works nicer with anonymous types, though so far the WPF Toolkit DataGrid failed, and so does the ListView, that is why we need to do the trick with re-creating the columns based on the new results. At work we are using the Infragistics xamDataGrid, and it works straight out of the can with anonymous types, we do not have to mess about with columns at all, it just knows what to do.
Here is the results
Showing ALL Columns Of Results
Which is done using the following logic
1: case WritePropertyType.All:
2: //OK this is a static query that could have been done in standard LINQ
3: //but this techniqe can be used to create dyanamically created
4: //LINQ queries at runtime
5: builder.AppendLine("var x =(from r in results select new { ");
6: builder.AppendLine(" Age = r.Age,FirstName = r.FirstName,");
7: builder.AppendLine(" MiddleName=r.MiddleName, LastName = r.LastName,");
8: builder.AppendLine(" ID = r.PersonData.ID, Dob = r.PersonData.Dob });");
9: builder.AppendLine("return x;");
10: builder.AppendLine("}");
Showing Selected Columns Of Results (see we only get some of the columns)
Which is done using the following logic
1: case WritePropertyType.Some:
2: //OK this is a static query that could have been done in standard LINQ
3: //but this techniqe can be used to create dyanamically created
4: //LINQ queries at runtime
5: builder.AppendLine("var x =(from r in results select new { ");
6: builder.AppendLine(" Age = r.Age,FirstName = r.FirstName,");
7: builder.AppendLine(" LastName = r.LastName});");
8: builder.AppendLine("return x;");
9: builder.AppendLine("}");
10: break;
Here is a small demo project
Now I know this is a very specific business case, and not many people will do this exactly, but it did lead to some very interesting code, that I thought I could share with you, it does show you how to work with dynamically compiled assemblies.
IMPORTANT NOTE
For this example I could have used the Dynamic LINQ library Dll, which is discussed over at Scott Guthrie’s blog.
However this technique is a powerful one, and could be used for a great many use cases, where you need to run some dynamic code that you only know about at runtime. It just happens to be I chose to use LINQ as the demonstration tool, at work we actually use the Dynamic LINQ library Dll, but the rest is the same.
Josh Smith said
am April 5 2009 @ 4:19 pm
Fun stuff!!!
Thanks for sharing this, Sacha.
Josh
brad said
am April 5 2009 @ 4:45 pm
Instead of all that dynamic c# stuff, i think you could create an expression tree (the backbone of linq) and then compile that tree and execute it. If you would like a quick example, let me know.
sacha said
am April 5 2009 @ 7:00 pm
Thanks Josh.
sacha said
am April 5 2009 @ 7:03 pm
Brad
I think you are missing the point, the LINQ stuff has to be decided and worked out at runtime, as I do not know what the expression tree will look like until a search is run (runtime). See what I mean.
What would the expression tree look like, what properties would it include, these would change at runtime depending on metadata.
I fully understand how LINQ works, it just doesnt work here.
On the other hand Entity SQL (which allows dynamic expressions to be created) also uses strings.
And the extensions that were released out of bands (Dynamic Library as seen on Scott Guthries blog) also only allowed a string for a Where() extension method.
Of of which are not good enough.
I needed to create a LINQ call at RUNTIME, which I knew nothing about at design time.
Ben said
am April 5 2009 @ 7:24 pm
Interesting article.
I’m curious does this method lend itself to memory leaks though, if code is being compiled dynamically each time a button is clicked?
sacha said
am April 6 2009 @ 7:46 am
Ben
Yeah you would not want to use this approach all over the place. But we have used it evry sparingly, and have seen no adverse effects from this approach, bith in terms of memory usage and time
Georg Begerow said
am April 6 2009 @ 2:48 pm
Realy nice article.
We used a similar approach some years ago (with .Net 1.1
) to compile user defined filters. This filters searched a collection of many thousand items in memory for matching items on multiple criteria for every logedin user, so they had to be fast. The application is still running on .Net 1.1 and still fast enough.
One point we learned the hard way, was not to hardcode the filter source code in the application. We finaly used a combination of a filter base class together with a homebrewn template engine. Nowadays i would suggest using T4. This way you can easily generate and debug the generated code outside your application.
Georg
sacha said
am April 6 2009 @ 3:00 pm
Georg
Yeah we actually create proper .NET types for our query, this dynamic stuff is just to trim the result set so to speak.
Gad you like it though.
Jean-Philippe Leconte said
am April 6 2009 @ 3:10 pm
I fail to see why you couldn’t use the Dynamic LINQ library. The only thing I used that is not provided by it is Join, and it can easily be implemented over the existing Dynamic LINQ code.
CodeDOM is an interesting approach, but the Dynamic LINQ does it without “leaking” new code in your AppDomain (unless you use a Select that returns a new “anonymous” struct, which makes Dynamic LINQ emit a dynamic class).
sacha said
am April 6 2009 @ 3:21 pm
Jean-Philippe
I have used the Dynamic LINQ API before but I thought it was only able to be used to create Func that could be used on Where() extension methods where we already had a IEnumerable already.
I could be wrong, have you got an example that would do what I am doing here returning an anonmous type using the Dynamic API.
I will give it a go, I have had another think about this, and yeah it may actually be possible using Dynamic LINQ after all.
Jean-Philippe Leconte said
am April 6 2009 @ 4:00 pm
To return an anonymous type using Dynamic LINQ, just use the Select ( string selector ) extension, which supports the new keyword.
from r in results
select new
{
Age = r.Age,
FirstName = r.FirstName,
LastName = r.LastName
};
becomes
results.Select ( “new ( Age, FirstName, LastName )” );
to rename properties, you can use the “as” keyword :
results.Select ( “new ( Age, FirstName + \” \” + LastName as Name )” );
sacha said
am April 6 2009 @ 4:16 pm
Jean-Philippe
I think this approach will actually work after all, still I have now given folks another option to think about.
Its all good.
Pablo Alarcón said
am April 13 2009 @ 10:37 am
As always a great post Sacha, thanks a lot for contributing that much.
As Ben has previously said, I expect a little “memory leak” each time a dynamic assembly is emitted.
When emitting the class you’re generating a new Assembly and loading it into the current AppDomain, each time. Assemblies cannot be Unloaded, you have to Unload the AppDomain they reside in, so here it comes the “memory leak”.
What I did in the past is to explicitely create an AppDomain where to inject dynamic assemblies and Unload it after execution. There were some difficulties, speaking from memory you had to use “inter-AppDomain remoting” to invoke the emitted methods, Dynamic class must derive from MarshalByRefObject and so on.
This was a typical problem with “Plug-In” scenarios in .NET 1.1, Google has info for sure ( as for everything
)
sacha said
am April 13 2009 @ 5:20 pm
Pablo,
This post is actually a trimmed down post of what we do at work, we actually do unload the AppDomain.
We use something simliar to this
string fileName = @”C:\theFolder\printlib.dll”;
string folderName =@”C:\theFolder “;
Assembly assemb = Assembly.LoadFile(fileName);
AppDomain otherDomain = AppDomain.CreateDomain(”OtherDomain”, null,
folderName , null, true);
object t = otherDomain.CreateInstance(assemb.FullName,”PrintLib”);
AppDomain.Unload(otherDomain);
Thanks for your comments
Pablo Alarcón said
am April 14 2009 @ 1:50 pm
Cool, but when publishing code for others to learn it’s safer to cover every “don’t this in production code” bit known, even if it’s just a little mention to avoid polluting the example
sacha said
am April 14 2009 @ 1:53 pm
Pablo
Fair point, Ill ammend the article shortly
sacha said
am April 14 2009 @ 3:38 pm
Pablo
Sure that was how I took it, no worries
sacha said
am April 18 2009 @ 10:33 am
Pablo I have now updated the demo code to show how to do all this in new AppDomain
sachabarber.net » WPF : Dynamic Search Driven List Results (Now uses AppDomain) said
am April 18 2009 @ 10:33 am
[...] A couple of posts ago I showed you all how to use CodeDom/CompilerServices and LINQ to create a dynamically created dll. A few people stated that this should have been in a new AppDomain (and it was already this way at work), so I have updated the example post to show you how to create a dynamic assembly in a new AppDomain. Have a look at the original post here : http://sachabarber.net/?p=496 [...]
Pablo Alarcón said
am April 20 2009 @ 8:24 am
Great, now let’s hope Google shows this as an example to do CodeDom emitting and less people suffers that common memory leak problem.
sacha said
am April 20 2009 @ 12:25 pm
Yeah right on Pablo. Thanks for your encouragement.
tawani said
am May 16 2009 @ 11:25 pm
Sacha,
Nice and simple approach.
A better way (and much easier/flexible) to display dynamic search results will be to bind the ListView to a DataMatrix. You can check out this approach from my article on the CodeProject.
http://www.codeproject.com/KB/WPF/WPF_DynamicListView.aspx
Stephan Keil said
am June 9 2009 @ 11:27 pm
Sorry, but I don’t get the trick of using a separate AppDomain.Can you please explain which kind of memory leaks it avoids?
IMO the AppDomain – as you use it – still leaks in dynamic assemblies and the type objects in these assemblies:
1. ‘domain.Load()’ loads the dyno.dll into the temporary AppDomain. But as a side effect of returning the loaded assembly, the AppDomain.Load() also serializes/deserializes dyno.dll into the current (”main”) AppDomain.
2. When calling ‘instanceHandle.Unwrap()’, the ‘EnumerableResultsCreator’ object is deserialized/serialized into the current (main) AppDomain (for this to succeed the download code correclty contains the necessary [Serializable] attribute which is omitted in the snipped of your post).
Thus the now unwrapped ‘EnumerableResultsCreator’ instance (along with its type object) lives in the main AppDomain and so do the classes dynamically compiled during ‘ObtainFlattenedResults’.
To see the effect you can set a breakpoint right after the ‘results = …’ in DataLoader.ObtainFlattenedResults (line 62 in your code snippet) and watch AppDomain.CurrentDomain.GetAssemblies() growing. For each call to DataLoader.ObtainFlattenedResults() a new assembly containing the dynamically compiled types is added to the domain.
As I understand it, you simply cannot unload an AppDomain and still use an object residing in that domain.
Either the object does not “really” reside in the AppDomain, but has been serialized/deserialized into the calling AppDomain (which needs the respective class to be [Serializable], as your EnumerableResultsCreator); or the object is accessed through a proxy (which needs the respective class to derive from MarshalByRefObject). In the latter case the proxy is “dead” after unloading the AppDomain.
A real nice feature to support dynamic applications would be that Type objects and the code associated with them would be (optionally?) subject to garbage collection. I.e. the Type and code is unloaded as soon as the last instance of that type is gone. But – AFAIK – this is not possible in .NET/CLR.
sacha said
am June 10 2009 @ 7:44 am
Stephan
These are some excellent comments.
And to be honest I was under the impression that you could unload and appDomain and it would also unload any items that were dynamically loaded into it.
I shall have to look into this a bit further. Ill try and do that soon and write my findings here.
Thanks for pointing this out.
Stephan Keil said
am June 10 2009 @ 7:50 am
Hm, my previous post doesn’t show up, so here is my comment again:
IMO your AppDomain does not avoid the leaks, at least if you mean the leaking of temporary assemblies and Type objects.
Please correct me if I am missing something:
1. While loading the Dyno.dll assembly (line 37: ‘domain.Load(…)’) into the temporary AppDomain, the assembly is also loaded into the “main” (current) AppDomain. I suppose that is because AppDomain.Load() returns the loaded assembly, which actually serializes/deserializes the assembly from the temporary to the main AppDomain.
2. When unwrapping the object handle (line 48: ‘instance = instanceHandle.Unwrap()’), the same occurs with the created EnumerableResultsCreator. I.e. it is copied by (de-)serialization from the temporary to the main AppDomain (for this to succeed, the downloaded code correctly has a [Serializable] attribute on EnumerableResultsCreator, which is missing in the code in your post).
All subsequent accesses to the instance then occur in the main AppDomain, thus the temporary AppDomain is actually useless.
For each call to DataLoader.ObtainFlattenedResults(), the invocation of EnumerableResultsCreator.ObtainFlattenedResults() (through reflection, see line 52: ‘results = instance.GetType()…’) finally uses CSharpCodeProvider.CompileAssemblyFromSource() to compile the dynamic code. The dynamic code actually contains two class types: ‘ResultsWriter’ and the anonymous type from the linq expression.
CompileAssemblyFromSource() creates a dynamic assembly for these types. And as the EnumerableResultsCreator ‘instance’ resides in the main domain, the assemblies are also loaded into that domain. So there is one dynamic assembly with two types for each DataLoader.ObtainFlattenedResults() call => leak.
You can nicely inspect the effect by setting a break point after ‘results = instance…’ (i.e. line 62 in your code snippet) and watch AppDomain.CurrentDomain.GetAssemblies() growing.
As I understand it there are two options for using an object accross AppDomains:
Either the object is serialized/deserialized from the source domain to the target domain (which needs the respective class to be [Serializable] as in your example), resulting in actually two instances, one per AppDomain. Here unloading the source AppDomain has no effect since the target AppDomain works on a copy of the object.
Or a proxy for the object is created in the target domain (which needs the respective class to derive from MarshalByRefObject). In this case, unloading the source AppDomain invalidates the proxy, so each subsequent call to the proxy will result in an exception.
To develop a dynamic data library for the company I work for (see http://www.quinlogic.com) I was also looking for a way to dynamically create types (classes) which only exist temporarily. For my understanding this can be achieved with a temporary AppDomain, if all accesses to the “dynamic” types are only made in that AppDomain.
This is not very convenient. My favourite approach would be if a dynamically created Type (and its associated code) is (optionally) subject to garbage collection. I.e. the type and its code is collected if the last instance of that type has gone. To my knowledge this is not possible with .NET/CLR.
Stephan Keil said
am June 10 2009 @ 7:53 am
Oops, now my original comment is there again. Sorry for posting the same ideas twice…