Data Insurgents

Welcome to Real World Inc!

Marty has been at Real Word for 6 months, and everything still feels new and exciting as an early member of this new Data team. He’s been hunting data, creating dashboards, and doing some analysis. Now, something big has come up. Stacy, the new strategic product manager for a legacy product, needs help with pricing. The plan had been to raise pricing (again), but she’s seeing a surprising drop in orders and revenue compared to last year. She’s worried and asking for Marty’s help in determining the cause and analyze the impact of the planned price increase. The results will be presented at a high-level meeting with the VPs of Revenue, Product, Marketing, the CFO, and the CEO.

This is it! The big one! Marty’s chance to show the impact of data, and for the CEO! The meeting is in a week, so there isn’t much time. Marty meets with his boss, Angela, and comes up with a plan to completed in week, leaving time to polish the presentation (that he will be leading! For the CEO!).

It takes some long nights, but Marty crafts a solid narrative. He feels good about the work, the deck, and the recommendation: don’t raise prices. The cumulative effect of years of price increases has reached a tipping point, and the company can’t raise prices fast enough to make up for the drop in orders.

The day is here, Marty feels anxious, but a good anxious. Angela gives him a pep talk before the meeting, “You’re ready. It’s going to go great…But be prepared, they are probably still going to do the price increase.”

Wait…what? Why?

“Well, Dick, the VP of Revenue, is under a lot of pressure to hit quota and with soft demand he believes he needs the price increase…but don’t focus on that, you got this!”

OK, weird, but Marty is confident. Once they see the numbers, they will have to come over to his side. Even with the short turnaround, the case is rock solid.

The meeting started well with the CEO stating up front “we need to make decisions based on what the data says.” As Marty finishes the presentation, he knowns he nailed it. Everyone is polite and asks good questions. There is some good discussion, but then they decision to go through with the price increase.

Why? Because “we assumed the price increases in the budget.”

Marty finds the whole thing frustrating, but Angela congratulates him on great work and a great presentation. “It takes time to make an impact, but they were all very impressed with your work and how you handled yourself. Dick even wants to follow up about pricing analyses for other products.”

Well, at least there is that. It may not be the CEO, but Marty could make some real impact with the VP of Revenue. Dick had come in determined to keep the price increase, but now that he had seen what Marty can do, he will be more open.

As Marty gets his lunch from the break room, Stacy approaches him.

Stacy: “Thanks for all the work you did. It stinks the budget overrode all the great work you did. I think Dick is scared for his job and didn’t want to take any chances.”

Marty smiles through his teeth, not wanting to relive the meeting. “You know, it would be great to get your help when we start doing the budget for next year. That would give you more time to do analysis and they wouldn’t have the budget as an excuse. Let’s talk in 3 months when we start budget prep!”

Marty nods, hoping she will leave him alone. All he wants to think about now is the meeting with Dick.

Two weeks later, Marty is ready for the meeting with Dick. He has done some initial analysis and is eager to show how he can make a difference. Angela advises him to wait and see how it goes, but Marty has already penciled in projects for the next quarter with the Revenue team. This is only the beginning.

Angela opens the meeting:

Angela: “We are excited to follow up with you. Just a few questions before we get into details. What decision are you looking for us to help you make?”

Dick: “What? You might like pricing? Nothing you say is going to change our current plans - maybe if you say raise prices more, we might do it.”

The room goes white…

Angela: “So you’re looking for confirmation of what you already have planned?”

Dick: “Exactly, anything you can give me to support our plan would be great.”

Marty’s stomach drops, and the rest of the meeting is drowned out by the ringing in his ears. Angela limits the damage, of this farce, to one quick (and pointless) analysis. But at this point, Marty doesn’t care.

Where did it go wrong? This was supposed to be the start of the data transformation of Real World Inc and the beginning of Marty’s ascension. They came to us asking for help, VPs, executives…the CEO. Yet in the end they ignored all my work. How is this company going to survive, never mind grow, if they are going to ignore data as clear as this? Why am I even here?

As Marty wallows, his phone dings with an invite from Stacy for a kick off meeting in three months. Whatever, I won’t be here by then…

"You have power over your mind - not outside events. Realize this, and you will find strength." -Marcus Aurelius

Marty's story is all too familiar to many in the data world. It's a tale of high hopes dashed against the rocks of corporate inertia, where data-driven decision making is more buzzword than practice. But before you update your LinkedIn profile and start browsing job listings, let's take a step back. Marty's mistake wasn't in his analysis or his presentation skills – it was in his expectations and approach to organizational change.

Marty made a crucial mistake from the beginning, a mistake that has doomed teams and armies going back to the beginning of humanity: he assumed the battle would be quick and the war was already won. He looked past the current opponent and stated planning for his next round.

But in the world of data, there are no coronations and there are no easy battles. Many have gotten into the data world under the assumption that corporations have already decided to embrace data-centric culture, only to be confused and discouraged when they encounter reality.

Data teams are not a glorious army destined to rule the world. They are insurgents fighting a guerrilla war. It doesn’t matter that leadership invited you into the company or that they say they want to change, your existence is a threat to the ways thing are and have been. On top of that, there are a lot more of them then there are Marty’s. We can’t win with a full frontal attack.

So what can Marty do in this situation? While we can’t go over everything, here is a starting point.

Expand your time horizon: Strategy for an infinite war

Almost every war of the last 200 years years has started with a believe by one or both sides that, “one big battle and this will be over.” It is a statement of wishfulness more than reality. But the reality is most things take longer than we want to believe. There isn’t going to be a data revolution that overthrows the old guard, it is only going to happen through a slower evolution and that take time.

Therefore Marty has move his timeline from months to years. An established company does not change its processes overnight and most long term executives are set in their ways. Change is going to come on a smaller scale stacking small victories on top of each other over a period of years. That creates evidence that data is helpful and impactful and creates a team of believers and advocates below the VP level.

So if our horizon is now measured in years, there are two major focuses you need to have:

• Stay in the game

• Build cumulatively

Stay in the game:

In many ways this is an infinite game, and the first rule of an infinite game is to stay alive to keep playing. You can’t influence change 2 years from now if everyone gets fired in 6 months.

Ultimately that means you are going to need to stay leaner and grow slower than you want to. A team of three using open source tools can be successful because they can show strong ROI helping incrementally across the organization. A team of 30 with a Snowflake/Databricks/BigQuery bill increasing 20% a quarter needs to be foundational to the success of the company.

That is where so many make the mistake, there is so much work that could be done that we push for more resources, often on the feeling of “if we build it, the work will be there.” Only to discover that now all your time is being consumed in building and maintaining all the shiny new tools. And while people want you to do new and innovative things they are more concerned with making sure they get any value from your team. And once you start spending real money you have about 18-24 months to prove your worth.

So stay alive! Be prudent with the money you spend. Build a case and plan for how this money will used AND create value.

Building out databases, pipelines, and dashboards are not an ends. They are a means to something greater. That means you will create technical debt. You will make a simpler version of a process that will need to be replaced as it grows. It will feel frustrating and inefficient. At times you will see more work you could do but can’t focus on because you don’t have the bandwidth.

It will be slower, but it will be better in the long run. It will give you time to grow, to experiment, and build the foundation needed to succeed when it is time to make the big bet. In the end you will need to make a big bet, a project or initiative that data is going to be front and forward, where spending money will be required and the companies future will be on the line (either saving the company or expanding to new territory). That is what you need to be ready for and you want it to be the bet with the best chance of success.

Build cumulatively: Fortify your base

The danger of focusing only on surviving is you never build and improve. Bandaids and chewing gum patch everything together because we are in constant survival mode. This is puts you as much danger as expanding too soon.

Survival is not an end goal by itself. You want to survive for a purpose. That big bet is coming and Rube Goldberg development is going to get you killed.

That is why you balance staying in the game with building cumulatively. Everything you create should build off the previous. Your processes and tools should be continually improving. It won’t be prefect but it will help keep you off the technical debt treadmill.

What does that look like? Here are a few examples:

• Use version control before you think you need to: Don’t wait until you have 1,000 of lines of code over some kind of cloud storage (or even worse a shared network drive)

• Start every project with a common template: Everyone starts from the same place helps seeding standards early and setting consistent habits across the team.

• Build everything to be repeatable: They will ask you to redo that analysis in 3 months. That one pipeline you made will need to be replicated for other sources.

• Force the time to refactor and reorganize: Fixing and improving your system and codebase is not one else’s priority. Stop waiting for permission or approval.

• Consciously take on technical debt: Acknowledge that you are taking on this debt, understand the trade-offs you are making.

• Build out vertical slices of processes instead of horizontal slices: Focus on delivering end-to-end functionality for a small part of the system before going deep (usually on the middle), this will save you from trying to force manual processes at the beginning or end of the system at the last minute.

To reiterate, no one else is going to make this a priority. Most business users care about getting the data they need right now. It will not happen, unless you make make it happen.

That means you have to make the decision upfront to hold that line despite the short term consequences. And in the long term, when you start to see the fruit of your decisions, no one is going to celebrate or congratulate you for holding the line.

You are building for a future that you may not ever see. You may leave¹ before you ever see your work bear fruit and you need to be OK with that. Because even if you don’t see the fruit here, over time there is a compounding effect on your outcomes and career from focusing building cumulatively.

You are piling up knowledge on

• How to build and scale

• What is needed now and what can wait

• How to automate early and often

Learning how to organize and rally people:

• How to negotiate from a low power position

• How to influence and bring people to plan

• How to motivate people even when times are tough

And, maybe most importantly, building up resiliency and internal strength for yourself.

Marty may never turn Real World Inc. into a data powerhouse, he may leave for another company feeling like he never broke through. But over time, stacking up small wins, winning over one mind at a time, stacks up like territory won on a battle field. You win first slowly², and then all at once.

But I can’t guarantee ultimate victory, insurgence fighting rarely leads to a clear decisive victory. Luck is always needed, and sometimes we don’t get the luck we need in time. In the world of data, persistence and patience are key; the true measure of success is not in the immediate results, but in the lasting impact you build and create.

So don’t be afraid to plant trees you will never sit under, embrace it³.

1. Or be asked to leave

2. very slowly

3. “A society grows great when old men plant trees whose shade they know they shall never sit in.”