Interview with Carl Rosenberger, db4o Chief Software Architect, db4objects, Inc.
R: Carl, you are the Chief Software Architect of a database company. Not Microsoft, not Oracle, not IBM. How come that you have the courage to create your own product alongside these gorillas?
C: db4o was created from a desire to simply store objects. At the time we had a very complex application that worked with objects in memory. The effort to store all objects manually in SQL took up half our development time. We just dared to start developing db4o from scratch because we thought we weren’t alone with this problem. We were able to secure some customers who believed that with our open source approach we could occupy a whole product category. That explains why today we are the most well known brand in the niche category “object databases”.
R: So your courage was spurred by a niche category, a target that the gorillas have ignored. How is it that they don’t dominate this segment?
C: Object oriented programming has only really become mainstream since C# and Java compete with each other. Now the mass of developers is suffering under the challenge to have to store objects. The gorillas are reacting late because a paradigm shift would endanger their posistion. Why should they change anything if their SQL business is going well?
R: So the niche still exists because the gorillas are slow to respond to the current changes. However that doesn’t mean that they don’t want change. They are just happy with the status quo. Or is it that the relational paradigm - despite all the object oriented the code – is simply better for persistence?
C: When you say “better” you need to be specific about the use case. We never intended to replace large enterprise SQL servers. We see our strengths in the mobile device and embedded market where top performance is required in resource constrained environments and where only one application uses a database. A big RDBMS machine is oversized for a cell phone. In this case a tight integration between the application and the database is helpful to achieve top performance. db4o is also used for caching in near real-time environments when application models are complex, such as in the financial sector, for trading applications.
R: I heard you say that you have not declared war on the relational model but rather argue that each situation should be analyzed on a case-by-case basis. The relational model should not be the solution to every persistence problem. Object oriented code, new form factors (PDAs, phones) and new requirements (e.g. cache in grid applications) are good enough reasons for you to create a new database.
C: Relational databases will most certainly still be around in 20 years but perhaps with a smaller market share compared to today. Our objective is to simplify developer’s work. The way we do that is to add as little as possible to the programming language. With db4o one can store any object with just one line of code. In the past, adoption of object databases was slowed by the lack of a standard query language. LINQ has changed that. Microsoft realized that it makes sense to integrate the query into the programming language, because you get type-safety, IDE support, and automatic refactorings. That is why we support LINQ.
R: So there are requirements that are best fulfilled with an RDBMS. But there are more and more requirements and requests to simplify data persistence. Simpler with regards to the API, simpler with regards to price, deployment and also maintenance. One line of code to persist data sounds like a great simplification. But is that sufficient in combination with LINQ to overcome the hurdles that object databases faced in the past?
C: LINQ is abreakthrough. Thanks to LINQ the database behind the programming language can be replaced without changing a line of code. It is great for us that LINQ queries are object oriented – that way we can optimize our database and leverage performance advantages. You mentioned problems with object databases. I don’t know any. J Which problems?
R: I know FastObjects from Versant and ObjectStore from Progress. But I only know about them (from the past when I still programmed in C++ ;-) but don’t use them and don’t know of anyone who uses them. They have customers but not in the mainstream of software development. So the question remains: what does db4o do better than these old products that despite all efforts in the past ten years haven’t been able to excite the masses?
C: Both of these products were developed in a different millennium. The core of these products was written in C++ so it is more difficult for them to adapt to new languages. On the one hand we have the advantage that db4o is written natively in Java and C#. On the other hand we have a totally different business model: our product is freely available under the GPL. Our source code is freely accessible. That is how we got to a point where informed developers talk about db4o. Paradigm shifts are driven by users, by fans.. The db4o community has 30,000 registered users. db4o is used in lectures at universities. When you use object oriented programming languages it is just natural to store objects directly. Some day the majority will follow suit.
R: hm… is it important that existing ODBMs were written in C++ and are therefore more relevant for C++ projects and that despite the large existing code base compared to Java/C# or even dynamic languages the importance will diminish over time? Perhaps. But is was different in the past when ODBMS first appeared. And yet they weren’t successful. In my opinion your reference to the business model points to the more likely cause of the failure: in the past the objective of the business model was to replace the relational model and establish ODBMS as the enterprise database. That vision was too big; it failed to realize how established and sufficient RDBMS were and still are today. You are taking a different approach: a grass roots approach. Open source + “smaller” application scenarios instead of a huge ideology.
C: We see our largest commercial market in software development for cell phones where we expect a boom. In the near future there will be 5 billion cell phones and only 2 billion PCs. With GPS available on devices great new applications are possible: Where are my friends now? With which car can I drive home? Android, Google’s new development platform, will play a large role in this area. Windows Mobile will also be strong in that market. db4o is very well suited for both systems. This is where we hope for our largest commercial success. There will still be users that use db4o as a caching database for LINQ applications on large servers. We can handle this stretch because our open source model and the modular structure of the source code allows contributions from the community.
R: So your opinion is that software development is only at its beginning, RDBMS are not the end of a long development but only the dinosaurs of an IT era where servers and PCs were the platform. As this era is being replaced by the era of global networks (telephone, Internet) and the number of “software-driven” devices explodes that provide totally different software environments, developers should think more application centric. They shouldn’t just try to rescue outdated principles into the age of fast, dynamic software. The dinosaurs are still around but more and more small mammals are running circles around them. J
But now to LINQ – you fed me that little tidbit twice now: Despite being happy about LINQ I don’t quite understand your euphoria. If LINQ is a standard interface then why should I as an OO-programmer care whether I am working with a RDMS or an ODBMS? The desire for more simplicity should diminish thanks to this API with less impedance mismatch.
C: I think it works the other way around. The total market for databases is 5 billion USD. To date, object databases have only played a minor role in the range of 200 million USD. When databases are replaceable the equilibrium will tilt to our advantage. Performance, price, availability and support will play a large role. As a virtual company we can work cost efficiently with top developers no matter where they are located. Our core development team consists of two Germans, three South Americans and three Chinese that work from home. We offer a very interesting support model called XtremeConnect: our internal processes leverage VNC and Skype for remote pairing sessions. We replicated that process to support our customers: We connect directly with them and their development environment to help them tune their application and to improve it.
R: So it is less about competition but rather co-existence. But in the future it won’t be a niche but rather a new mainstream. If Microsoft’s ambition was to have one SQL server on each desktop then your goal could be one db4o on each cell phone J And with LINQ you feel supported because developers that until now were tied to RDBMS through APIs such as ADO.NET can now easily try something else that may even fit better.
The reduction of development cost is of course also a huge advantage. Dynamic markets require dynamic decisions and structures. Thanks to the international and virtual structure of your team you have local representation and understand local markets and developer mentalities. It sounds cool and cutting edge, including the pair-programming model that helps you maintains db4o’s quality high.
But I ask myself: it works for the current amount of customers. But does the model scale? You want to be successful. db4o is still in it’s infancy with mission critical applications. Two years in the market is not a very long time.
C: It isn’t hard for us to find good developers and add them to the team. Our large community helps us identify stars when they contribute articles and tips to our developerforums. Two of our best customer support engineers were hired that way. JBoss and MySQL have proven that the model scales beyond 100 employees. Linux (with how many contributors?) is also a great success story that shows that open source can steal away market share from established players such as Microsoft. However we don’t see ourselves as a low cost offering that copies existing technology. Some things we do are truly innovative.
R: So you are optimistic that you can grow with the market you defined. Small, agile, customer oriented. Sounds good. I’m curious to see how it goes. MySQL was just acquired by Sun for 1 billion USD. Perhaps db4o will be picked up for the same amount in a few years ;-)
That’s a long path until then and the question is: which features excite your users today. What is it that they like about db4o or miss with RDBMS. It can’t be just the size of the database as there are embeddable RDBMS such as TurboDb or VistaDb.
C: RDBMS and objects suffer from the famous “impedance mismatch”. Object hierarchies cannot be efficiently expressed in relational databases. We can do that a lot better. Our customers love db4o primarily because of how simple it is to store any object with one line of code. The class definition is the database schema, additional maintenance is not necessary. Even if class definitions change, the changes are recognized and stored objects adapt to the new structure.
With db4o, persistence is “transparent”: db4o recognizes when an object changes and only stores the changed object. In the same way, objects are only loaded from memory when they are really needed.
We manage objects in the database with a similar physical representation to how the programming language deals with objects in memory: References between objects are one-way, not tupels (joins) as relational databases use them. That’s why our performance is better than the performance relational databases. Continous performance optimisations are very important for us. We use the open source benchmark “Poleposition” (http:// polepos.org) that also compares db4o with other databases.
R: One important characteristic is the simplicity/ ease of use. That’s the bait with which you catch developers. The LINQ integration is part of that. Developers find it simpler than to write code for a RDBMS and db4o is fast, however in very specific applications. You don’t see your main market in the enterprise space. What are in your view the characteristics of an ideal db4o application?
C: db4o works particularly well for:
- Applications on devices that have constrained resources, where highest performance with minimal memory usage is required
- Applications that use large, complex classes with many attributes and/or deep object hierarchies
- Applications that require frequent refactoring of the class model and that don’t require a relational database that is maintained in parallel
- Applications that need to get to market quickly, where developer productivity is essential
- Rich client applications e.g. that were written in Silverlight
- … and of course all open source projects – they can use db4o free of charge
R: All your examples have a common denominator – they are dynamic. You see RDBMS as less dynamic: They are difficult to use for “large” object models and for smaller structures that are subject to continuous change. I would argue that prototyping is a good application for db4o – even if it has less revenue potential as one can live with the free open source version ;-)
A second common denominator is efficiency – of development (productivity) and the use of resources.
Highly dynamic, highly efficient – if an application has both requirements the developer would be well advised to take a look at db4o.
I can understand that. But in real life it is often the case that after initial success one gets burned. How well does db4o scale? Simple object persistence is very easy, perhaps easier than with ADO.NET/RDBMS. Are more complex applications also possible? How about C/S implementations? What about transactions? Can I control lazy loading to avoid roundtrips to the database? What about distributed applications?
C: Complex client/server applications are possible with db4o. We support an arbitrary number of transactions. Our type handlers und field handlers are modular and can be configured. You can also tune the transfer between client and server in detail if that is what you want. For distributed applications we recommend replication. We offer a product called dRS (db4o Replication System) to replicate objects between applications.
R: I heard you say: simple applications are simple and complex applications are possible but one has to be careful to ensure that one is still in the area where db4o is most effective. db4o isn’t trying to cover the areas where RDBMS are used today and are successful. I find that fully acceptable. No silver bullet for developers that just want to store objects. One has to think about the scenario and evaluate the options. db4o is a new, additional option to the existing predominant paradigm. Who thinks that persistence with ADO.NET is too complicated should look further than LINQ and SQL or O/R mapping in general.
C: O/R mappers are certainly a possible workaround if you already are using a relational database and can’t do without it. They have a series of disadvantages though, for example:
- Performance – the complete object graph needs to be loaded into memory to run business logic
- Complexity – O/R mappers have caching side effects due to the way they load objects. It takes a lot of knowledge to truly understand how these systems behave.
- Limitations when using language constructs: Because of the “impedance mismatch” and because O/R mappers work with relational databases they can’t store all constructs that OO languages offer.
R: So for you O/R mappers are workarounds. They force a fit between things that don’t match: object oriented data storage in memory and relational persistence. db4o doesn’t require you to force a fit as the persistence is object oriented. That promises efficiency as the translation, the mapping, is avoided.
To that extent in your opinion db4o is worth looking at whenever the application is dynamic, resources are scarce and isn’t required to work with an RDBMS. Because that can be the case when a large client database exists. Then there is no way around the O/R mapper workaround.
C: We don’t want to displace O/R mappers. As I said, I see the biggest opportunity for db4o in the rapidly growing cell phone applications market. It makes us happy to simplify developer’s work in all other areas as well – that’s why we started db4o.
R: If I relate to the beginning of our discussion then it sounds like a story that is common to many successful solutions: db4o was created out of a personal need, as you wanted to simplify your life. And then it developed its own momentum. That’s how db4o moved from a more traditional application area to dynamic, mobile applications.
The impression that sticks with me is that db4o simplifies, db4o supports dynamic applications, db4o stands for coexistence of different data models and persistence solutions. The future market of 2…5 billion devices is big enough for everyone.
C: Those are great closing words, thanks Ralf. I would be very happy if this interview would motivate to try out db4o. That’s easy to do via http://www.db4o.com/community/
Ralf Westphal (www.ralfw.de) is a freelance author with some 300 publications, coach/consultant, and speaker at developer events in Germany and abroad.
The focus of his work is on .NET software architecture and helping innovative software technologies into the market.
When educating developers he likes to walk off the beaten paths as his video serieses .NET TV and dotnetpro.tv as well as his training company, the Professional Developer College (www.prodevcollege.de) show.
Ralf Westphal is Microsoft „Visual Developer Solution Architect“ MVP, and from 1998 until 2005 has been one of the indepdendent Microsoft Regional Directors for Germany.