Back on August 24th of 2013 over at the Java, SQL, and jOOQ blog, the author refers to a presentation made by MIT professor Michael Stonebraker and his take on the future of DBMS. I am fascinated with the evolving state of hardware and software and how it is disrupting the old ways. In the tolling world, we thrive on the creation and processing of millions of daily transactions. I don’t claim to be a data wizard, so I pinged a colleague of ours over at Jacobs Engineering, Lutz Braeuer, for a more in depth look.
The following includes an interview with Lutz, a thought leader at Jacobs who brings more than 12 years of experience in data intensive markets, gives us his thoughts on the architecture that sits at the core of our livelihood.
With the increased performance of column based architecture, what new applications do you see in the horizon that otherwise would not be possible?
LB: Industry research analysts have predicted that column-based architectures will take over traditional data warehousing environments within the next two years. With ever-growing amounts of data, it is extremely challenging from both a performance and cost perspective for row-based architectures to keep up with the next generation analytics applications. Many companies are already struggling to support their business intelligence efforts with legacy row-based database management systems. The widespread adoption of columnar databases is only a function of data growth and performance to surpass legacy systems’ capabilities. Future applications that will see greater use of column-based architectures are exploratory systems in scientific database environments and highly parallel data processing systems in business environments where database systems will interpret queries by their intent, rather than as a set of predefined instructions.
Do you see traditional database companies, like Oracle, moving their architecture in this direction or will revenue from hardware like Exadata deter from widespread adoption?
LB: The big three players in the data warehouse DBMS market (Teradata, Oracle, and IBM) have made substantial acquisitions over the last five years to align themselves more closely with logical data warehouse concepts. All three vendors currently offer hybrid solutions that support both traditional row-based and column-based data stores. With greater adoption of column-based architectures, the big three are very well positioned to meet their customers’ demand to analyze ever-accelerating amounts of big data.
How much growth in the market place do see for companies like VoltDB?
LB: There is certainly tremendous growth potential for niche players such as VoltDB in the operational DBMS market. The company offers a very innovative in-memory operational DBMS solution that is geared towards customers performing transaction and analytical processing in the same database. As with many DBMS niche players, VoltDB’s revenue model is open-source which may prevent the company from growing at the speed of its competitors and may also restrict feature delivery in its DBMS solutions. However, there are still plenty of opportunities for niche players and much remains to be done in relation to customers’ demands for new data warehouse practices and capabilities. For example, vendors need to offer customers the ability to shift between relational, virtual and distributed data management and processing seamlessly, rather than via sometimes clumsy manual change-overs.