Bytecode analysis and enhancement becomes increasingly important as we strive to improve on persistence transparency. The expressiveness of mainstream languages of the day just won't cut it.
Db4oTool.exe is the swiss army knife of the transparency inclined.
Native Queries
The first big step towards transparency was the language integrated querying mechanism we first introduced with Native Queries - the ability to use native programming language constructs as query specifications that can run optimized through bytecode analysis. Consider the following code snippet:
1: DateTime start = DateTime.Now;
2: var randomItemName = "Item " + new Random().Next(NumberOfItems);
3: var query = container.Query<Item>(item => item.Name == randomItemName);
4: PrintAll(query);
5: Console.WriteLine("Time: " + (DateTime.Now - start));
The code prints how long it took to execute a Native Query and iterate through all the results. Line 3 contains the actual Native Query expressed as a lambda expression ("item => item.Name == randomItemName"). Running the code on my system I get:
Item 17103
Time: 00:00:00.4218750
But is it really being optimized?
It is possible to disable Native Query optimization for comparison:
1:
2: config.OptimizeNativeQueries(false);
3:
And when the code is executed with the configuration above:
Item 32613
Time: 00:00:02.9062500
It takes much more time. Without the optimization phase every Item object has to be brought into memory to be tested by the predicate.
What's the optimization overhead?
One can check it by using Db4oTool.exe to enhance the application with ahead of time Native Query optimization:
$ Db4oTool -nq NQ.exe
By using an assembly visualization tool such as Lutz Roeder's Reflector it is possible to see that the query executing code was transformed to:
1:
2: class3.randomItemName = "Item " + new Random().Next(NumberOfItems);
3: var query = NativeQueryHandler.ExecuteEnhancedFilter<Item>(container, new Db4o$Predicate$4(class3.randomItemName));
4: PrintAll(query);
5:
The IObjectContainer.Query<T> call was replaced by a NativeQueryHandler.ExecuteEnhancedFilter<T> call. Further inspection of the synthetic Db4o$Predicate$4 class reveals it implements IDb4oEnhancedFilter:
1:
2: public override void OptimizeQuery(IQuery query)
3: {
4: query.Descend("_name").Constrain(this.randomItemName);
5: }
6:
Db4oTool.exe translated the higher level description of the query into the equivalent low level SODA calls. Running the enhanced application gives:
Item 17497
Time: 00:00:00.0468750
A 10x improvement for this particular case.
Getting rid of just in time optimization overhead is not the only reason to use ahead of time optimization. Just in time Native Query optimization is seemingly impossible in the Compact Framework. There's no published way of finding out the actual method behind a delegate reference - which is all a Native Query really is as far as the interface is concerned.
Ahead of time optimization is also useful to spot unoptimizable queries:
1:
2: var query = container.Query<Item>(item => item.Name[0] == 'F');
3:
Db4oTool.exe would output something like:
WARNING: Predicate 'System.Boolean NQ.Program::<Unoptimizable>b__b(NQ.Item)' could
not be optimized. Unsupported expression: item.get_Name().get_Chars
The query could then be replaced by an optimizable version:
1:
2: var query = container.Query<Item>(item => item.Name.StartsWith("F"));
3:
Transparent Persistence
A previous post describes Transparent Persistence at length. It should be noted here however that ahead of time Native Query optimization is always implied when enhancing for Transparent Persistence (-tp command line option).
Custom Enhancements
Writing code to analyze and transform an assembly demands a deep knowlege of IL, Mono.Cecil (the assembly reading/writing library used by db4o) but it's a very powerful technique that when used creatively can give birth to things such as Native Queries.
Db4oTool.exe bytecode enhancement infrastructure can be reused by the way of instrumentation types - classes that implement the IAssemblyInstrumentation interface:
1:
2: /// <summary>
3: /// Prepends Console.WriteLine("TRACE: " + method.Name) to every method
4: /// in the assembly.
5: /// </summary>
6: public class Trace : AbstractAssemblyInstrumentation
7: {
8: override protected void ProcessMethod(MethodDefinition method)
9: {
10: if (!method.HasBody) return;
11:
12: MethodBody body = method.Body;
13: Instruction firstInstruction = body.Instructions[0];
14: CilWorker worker = body.CilWorker;
15:
16: // ldstr "TRACE: " + method
17: worker.InsertBefore(firstInstruction,
18: worker.Create(OpCodes.Ldstr, "TRACE: " + method));
19:
20: // call Console.WriteLine(string)
21: MethodReference Console_WriteLine =
22: Import(typeof(Console).GetMethod(
23: "WriteLine", new Type[] { typeof(string) })
24: );
25: worker.InsertBefore(firstInstruction,
26: worker.Create(OpCodes.Call, Console_WriteLine));
27: }
28: }
29:
The instrumentation above could be applied to this simple program:
1: namespace SimpleProgram
2: {
3: class Program
4: {
5: static void Main(string[] args)
6: {
7: Foo();
8: }
9:
10: public static void Foo()
11: {
12: System.Console.WriteLine("Inside Foo");
13: }
14: }
15: }
$ Db4oTool -instrumentation:CustomInstrumentations.Trace,CustomInstrumentations
SimpleProgram.exe
With the following result:
$ ./SimpleProgram.exe
TRACE: System.Void SimpleProgram.Program::Main(System.String[])
TRACE: System.Void SimpleProgram.Program::Foo()
Inside Foo
Conclusion
Creative use of bytecode analysis and enhancement can greatly improve the development experience by excusing developers from having to write boilerplate code or introducing alien base classes into their models and by allowing the use of familiar programming idioms without sacrificing performance. Db4oTool is our bytecode creativity sink.
For more information about Db4oTool - including how to integrate it with Visual Studio and MSBuild - don't forget to check out its reference page.
Complete code can be found here.