Cynical Data Guy Follow-up: Future of the CDO
Evolve or die
On the latest podcast, John, Eric, and I finished up by talking about the shrinking role of CDO in companies. Some companies are subordinating the CDO under the CIO, CTO, or CFO1, but others are eliminating the role completely and having the duties taken over by CIO and/or CTO.
If you are a data person, this is concerning, but not surprising, if you have been paying attention the last 5 years. Companies disillusionment with data teams is obvious from the layoffs that data teams experienced the last few years.
But for an industry that was supposed to be recession proof and had blank checks for a decade (actually this is part of the problem, blank checks are a red flag), what is causing companies to completely drop data?
There are two roots to our current situation, one from the company and one from the CDO.
Companies
“All we have to show for [our efforts] are prettier dashboards.”
The motivation for companies to first start investing in data they buy into the silver bullet theory of data: All of their data has value, all we need to do is spin data into gold.
It comes from consultants2, news reports, or looking at highly successful companies, and a general sense of FOMO. That fear of falling behind or gold fever for easy profits develops a “shut up and take my money” mentality. Companies commit large amounts of money without concrete use cases beyond “data transformation” or being “data driven.”
The lack of discipline and focus incentives spending money on technology, since it is a concrete focus. At first it is the cost of doing business, but as time goes on people start asking questions as results fail to come. Millions spent with no ROI causes a pain and makes executives leery of investing in data moving forward.
Chief Data Officers
“…few CDOs had considered profitability one of their KPIs”
My first impression reading the above quote was indignation. How could profitability not be KPI in the mind of a CDO? Maybe because I started working in consulting (where we badly under charged for our data services) or because the most successful CDO I worked for had a sales and product background, but I always think about and track my teams ROI.
Then when I thought back, outside of that one CDO, most data leadership I have encountered don’t think that way. They are much more internally focused. I think there are two different reasons for this:
They came from companies were data was the product or largely defined the product.
There was never a question of the value of their work. The better and more complete the data the more money it would generate3. But they didn’t monetize it, that was someone else’s job: product, sales, development, etc. When they moved to traditional corporate settings they never made the correction that data didn’t automatically equal revenue. Even when they saw opportunities to monetize, they had no experience how to make that happen.
Most CDO’s backgrounds is in infrastructure side of data.
This makes sense if you realize that for most companies view data management positions as super individual contributor. The position that needs the most experience early on is the data engineering and infrastructure roles. They rise up because they built the pipelines and they did it well4.
They were always several steps away from the conversion of data to revenue. Rewards and advancement came from keeping business units happy, regardless of the request. But data at a large scale has to pay for itself and that requires discipline and focus. In other words, the need to say “no,” a lot. The opposite of what they have been trained to do.
It also requires hunting for your dinner, something people on the infrastructure side rarely have to do. So data strategy becomes technical diagrams and trying to control the rate of increase on cloud storage and computing costs.
The future of the CDO
I see two extremes for the CDO role(with a bunch of more likely middle ground inbetween).
1. The elimination of CDO role
2. The evolution of the CDO role
Elimination of the CDO role
This is a continuation of the current trends. With the hype cycle crashing for data teams and the pendulum swinging in the decentralized direction, data analysts and scientists are hired directly by business teams. The compliance and data governance functions are absorbed into the CIO and data engineering for products is absorbed into the CTO.
This would have the advantage of bringing data professionals closers to the business units that they provide value to. But has significant downsides. Data returns to the isolated, siloed state we have seen in the past, that causes data knowledge and insights to also stay siloed.
Evolution of the CDO role
Here we see a mirroring of technology teams, where there is a split between internal and externally focused team. But as data has grown we have not seen this division of labor yet. Companies want the benefits of the externally focused elements of data (revenue generation) but hire and organize as an internally focused (cost center). In many ways companies are hiring for a VP of infrastructure, not a Chief Data Officer.
In this scenario companies start hiring CDO with product and sales experience, likely people with more data analyst and data scientist background. Underneath that person is two groups, internally and externally focused. This allows for the business to see an explicit split between the group focused on revenue generation and cost controls.
If I had my way, I would lean towards the latter model, but when data teams fall on their face (or spend millions with no hope of making anything back), companies get shy about investing again. I would expect many of the companies that feel burned from the last 10 years to eliminate formal teams or run them on shoe-string budgets. Which means tougher time for the current generation of CDO.
Which isn't new; most CDO roles are not a real "chief" position but closer to VP of Data or Data Infrastructure
The drivers and profiteers of almost every hype cycle
When your work is the revenue generating work, you can get away with a lot
or fast, tech debt be damned