The Data Deluge and the Leadership Dilemma
The message of the past couple of decades has been clear and universal: collect data.
So we did — everywhere and on everything. Sensors multiplied across factory floors, customer touchpoints generated endless streams of behavioral data, and every interaction was logged, tracked, and stored. The promise was simple: more data equals better decisions.
Fast forward to today, and we find ourselves with warehouses full of disconnected, inconsistent data that nobody really understands or even trusts. Which begs the question: if we’re swimming in information, why are we still making critical decisions by instinct rather than insight?
I recently learned this lesson the hard way in Buffalo Gap National Grassland. I had taken my family SUV on ATV roads it had no business being on, and I found myself staring at a pretty sizable South Dakota mud pit. I convinced myself that if I just kept momentum and stayed on the ridge, my 4-wheel-drive would be able to power through. Instead, I ended up high-centered — all four wheels off the ground, spinning uselessly in the watery muck. More gas didn’t help. More effort didn’t help. The problem wasn’t lack of power — it was that I’d committed to the wrong path without thinking it through.
This is exactly where most organizations find themselves with data. We’re high-centered — spinning all four wheels, generating plenty of activity and noise, but going nowhere. We keep gunning it, collecting more data, adding more sensors, capturing more metrics, convinced that more effort will eventually get us unstuck.
It won’t.
The Real Problem: We Built it Backwards
The root of our current predicament is simple: companies collected everything “just in case,” hoping value would emerge later. Now, our faulty assumption has been laid bare.
Think about the typical enterprise landscape:
- Redundant sensors capturing the same metrics in different formats
- Survey data collected but never analyzed
- CRM systems that don’t talk to operations platforms
- Spreadsheets duplicating information across departments
We built data mud pits instead of living, breathing data ecosystems, because insight doesn’t come from volume — it comes from intention.
What we should have done was:
- Define what matters — Identify your core KPIs, customer experience priorities, safety metrics, and organizational values
- Decide how to measure it — Determine which specific data points actually reflect those priorities
- Build data flows around those priorities — Design your infrastructure to serve your strategy, not the other way around
Because so many of us started with a volume model, most of our data is descriptive of what goes on in the operational parts of our organizations. Automated processes were the easiest places to add in tracking data, so we did that. Customer feedback was logged and tagged. CRM data was archived. It was great at tracking “what happened.”
Your data shouldn’t just document your operations. It should tell the story of what matters to you.
The Framework: Designing Data That Flows
The good news: I’m not still in the South Dakota mud. After a couple of hours, a nice young man and his girlfriend came riding up on an ATV (a much more appropriate vehicle choice). He had the right vehicle (a big diesel dually truck), the right equipment, and the right plan. A couple of hours later, I was back on paved roads and headed to a car wash.
If you’re high-centered on data — spinning your wheels with dashboards that don’t drive decisions, warehouses full of information nobody trusts, and metrics that don’t connect to what actually matters — there’s a way out. You don’t need more horsepower. You need the right approach, the right tools, and a clear plan for getting traction again.
Here’s a simple framework you can use to pull yourselves out of your own data mud pit:
1. Clarify → What Matters Most?
Start by identifying what truly matters to your organization. This stage isn’t about only looking at what’s easy to measure; it’s about what drives your mission, values, and strategic outcomes. Are you focused on customer satisfaction? Operational safety? Production efficiency? Market responsiveness?
2. Curate → Find the Gold You Already Have
Before you rush to collect new data, look at what you’re already capturing. Chances are, buried in your existing systems, there’s valuable information that speaks directly to what matters most — you’ve just never connected the dots. That customer survey you’ve run for three years? It might contain the “delight” metric you need. Those maintenance logs gathering dust? They could be your early warning system for safety issues. Curation isn’t about collecting more — it’s about recognizing value in what you already have.
3. Collect → Gather with Purpose
Once you know what matters, collect only what’s necessary to illuminate those outcomes. Some of it you already have. Some will require new data pipelines. Some data you’re currently collecting is wasting time and energy and needs to stop.
If your core value is to delight your customers, how do you measure “delight?” If you want a fantastic company culture, what does that mean? It’s not enough to know what matters — you’ve got to know how to measure it.
4. Connect → Integrate across Systems
A maintenance log doesn’t mean much on its own, but when it’s connected to production schedules, quality metrics, and equipment sensors, it becomes part of a powerful predictive system. When customer support tickets are linked with product usage analytics, customer satisfaction surveys, and sales data, it can lead to targeted product features and improvements.
Data is rarely valuable in isolation. The true value comes when we learn to break down data silos and integrate insights across the organization. Everybody is hoping that AI can bridge this gap, but in my experience so far, the only reliable way of making this happen is with good old fashioned deterministic data pipeline coding.
5. Contextualize → Add Meaning with Analytics
Even when connected across systems, raw data rarely speaks for itself. A temperature reading of 87 degrees means nothing without context — is that normal or alarming? Trending up or down? This is where analytics, machine learning, and AI can transform numbers into narratives that drive action. Knowing that customer support tickets increased 15% last quarter is data; understanding that the increase correlates specifically with a recent product update, affects primarily enterprise customers, and predicts potential churn risk — that’s contextualized insight.
6. Communicate → Deliver Insights Where Decisions Happen
The most sophisticated analysis is worthless if it doesn’t reach decision-makers in an actionable format. This is where many data initiatives fail — not because the analysis is wrong, but because insights arrive too late, in the wrong format, or to the wrong people. Effective communication means designing dashboards, alerts, and reports that deliver insights at the exact point of decision — not buried in quarterly reports reviewed weeks after the opportunity has passed.
From Data to Insight: Real-World Integrations
Manufacturing
Problem: A mid-sized manufacturer is drowning in spreadsheets. Each shift tracks their own defects, and by the time quality managers compile the data for monthly meetings, the problematic production runs have already shipped. They know they have quality issues — they just can’t see them fast enough to act.
Solution: An automated pipeline can link defect data with material batches, environmental sensors, and equipment performance. Deterministic rules catch immediate problems. Machine learning predicts which combinations will likely fail. LLMs translate the patterns into plain language: “Humidity above 65% with Supplier B materials increases defects by 40%.”
Healthcare
Problem: A healthcare system is losing millions to insurance claim denials but can’t pinpoint why. Billing staff manually track rejections in scattered spreadsheets, and by the time patterns emerge, the appeal windows have closed.
Solution: An automated pipeline can link claim denials with procedure codes, payer requirements, and provider documentation patterns. Deterministic rules catch common coding errors immediately. Machine learning predicts which claim types are at highest risk before submission. LLMs translate the patterns into plain language: “Claims for Procedure X with Payer Y fail 60% more often when submitted without pre-authorization documentation.”
The Common Thread
In both cases, transformation doesn’t come from collecting more data — it comes from connecting the right data to what mattered most. These organizations should:
- Start with clear priorities (quality, safety)
- Design data flows to support those priorities
- Apply the appropriate level of technology (deterministic, ML, or AI)
- Deliver insights where decisions actually happen
Executive Playbook: How to Start Creating Clarity
Ready to get unstuck? Here are some next steps to get you moving:
1. Ask Your Team: “Is our data strategy driven by what matters most?”
Make an honest assessment of what’s driving your data strategy. If the answer is “what’s easy to collect” or “what we’ve always tracked,” you’ve identified your problem. Gather your leadership team and compare your stated priorities against what you’re actually measuring — the gaps will tell you exactly where you’re spinning your wheels.
2. Map Your Data Flows
Identify where data currently dies or duplicates. Where do handoffs fail? Which systems don’t communicate? Where does information get manually re-entered? Walk through an actual decision-making process in your organization and track where the data comes from, who touches it, and how long it takes to reach the decision-maker — you’ll likely be shocked by how many mud pits you find.
3. Assign Ownership
Someone must be responsible for data flow, not just storage. This isn’t about creating another IT role — it’s about ensuring a leader owns the question: “Does our data serve our strategy?” This person should have authority across departments and direct access to executive leadership. Without clear ownership, your data initiative will remain stuck between IT’s technical focus and operations’ practical needs.
4. Communicate Insights, Not Metrics
Dashboards don’t drive change — stories do. Transform your data communications from “here are the numbers” to “here’s what this means for our priorities.” Train your analysts and managers to translate data into narratives that connect directly to what your organization cares about most. A number without context is just noise — insight happens when someone understands what action to take.
Clarity as a Leadership Discipline
The journey from spinning wheels to solid ground isn’t primarily a technical challenge — it’s a leadership discipline. It requires the courage to say “no” to data that doesn’t serve your mission, the wisdom to connect information across traditional boundaries, and the vision to see data not as an IT asset but as a strategic narrative.
What if your data could tell the story of your values? What if every metric connected clearly to what your organization cares about most? What if insights flowed naturally to the people who need them, when they need them?
This isn’t fantasy — it’s the reality for organizations that have made the shift from spinning in place to gaining traction, from volume to value, from data collection to insight generation.