Listen now on YouTube | Spotify | Apple Podcasts | Amazon Music
▶ Watch the full episode and read the transcript: Data Quality for 140 Million Members | Intuit Credit Karma
What happens when a data quality alert fires at two in the morning and says that a few percent of last night’s data didn’t load properly? For many data teams, that’s a ticket to deal with later. Someone picks it up after their coffee or tea, reruns the pipeline, and all is copacetic. But what happens if you change your frame of reference and read the same alert as a count of people? At Intuit Credit Karma, Veenit Shah’s team treats that alert as a few percent of 140 million people who will open an app today and see a credit score that’s out of date. That habit of counting people rather than rows is the whole story of how the company got out of the firefighting years.
Part of the reason data quality programs fizzle is that nobody quite agrees on what quality data means anymore, and AI has made the definitions slipperier. Malcolm Hawker argued on this show that data is only AI-ready when it supports the use case in front of it, and Brendan Grady said that bad data didn’t matter much until AI started acting on it. Both of those conversations were about definitions. Veenit and Puneet Singh’s is about what a team does with an alert once the definitions run out.
I caught Veenit, Senior Manager of Data Engineering, and his colleague Puneet Singh, Senior Data Engineer, at the CDOIQ Symposium in Cambridge, Massachusetts, the day after they presented Credit Karma’s data quality journey to the conference.[1] The talk wasn’t a victory lap – it’s still a work in progress. Veenit described a program that started in a reactive state a couple of years ago and picked up what he calls “scars along the way,” and the two of them came to share their learnings rather than the trophy.
“Regardless of what level you’re at, whether you’re just starting as a junior engineer or whether you’re a senior manager as well, it is everybody’s job.”
— Veenit Shah, Senior Manager of Data Engineering, Intuit Credit Karma
So, who’s job is data quality? Everybody’s job is a line I’ve heard in quite a few governance decks, and when you hear this, it’s usually a signal that it’s nobody’s job. What makes it true at Credit Karma is a translation the team does on every alert, and I think that translation explains the pillars, the measurement discipline, and the AI agent that now does the investigating.
About Veenit Shah and Puneet Singh
Veenit Shah has spent close to five years at Credit Karma and leads the data engineering team that keeps credit scores current for the company’s members. Credit Karma became part of Intuit in December 2020, when it had more than 110 million members, and it now reports more than 140 million.[2],[3] Every one of those members expects an accurate score when they open the app, which is the number Veenit’s team measures itself against.
Puneet Singh is a Senior Data Engineer on Veenit’s team and has been at Credit Karma for four and a half years. His twelve years in data engineering started as a Hadoop developer, then moved to the cloud, and today he works mostly on Google Cloud, building microservices that ingest data and check its quality. He demonstrated the team’s agents writing production pipeline code on stage at the symposium.
In this episode, Veenit, Puneet, and I discuss:
- What startup-era data engineering looks like when there are no pillars to rely on
- Why five data quality pillars beat nineteen, and how a small proof of concept earned the whole program
- How every alert across 40,000 columns gets logged, measured, and reviewed each week
- Why a single ingestion alert has to be read as a count of members
- What an AI remediation agent does, what it is allowed to touch, and why it only works because the discipline came first
Watch the full conversation here:
Nightly calls and no pillars to stand on
When Puneet joined Credit Karma, the company still had what he calls the start-up feeling, even after Intuit acquired it. Move as fast as you can and put out as many fires as you can. There were no set data quality checks and no patterns to build on, so the job ran on long hours and nightly calls, and the same problems came back the following week because nothing had been fixed at the root.
“There were no set data quality checks, there were no patterns or pillars that we could rely on, and it was a mess, like long hours and nightly calls.”
— Puneet Singh, Senior Data Engineer, Intuit Credit Karma
It’s easy to stay in reactionary mode when there’s always another problem to fix, so how does a team ever win the mandate to stop and fix the root? Veenit says that a few incidents made it clear that the team’s data quality posture wasn’t where it needed to be, and the leaders above them let the engineers trust their instincts about what to double down on. Both things had to be true at once, because incidents without trust produce a blame cycle, and trust without incidents produces a roadmap that nobody funds.
What they had in front of them, in Veenit’s words, was a blank board, and they had to decide how to start drawing. The program is now at least three years old, and he was careful to say that it’s never complete. Generative AI and agentic applications are changing the end state faster than anyone can describe, so the team focuses on the next step rather than the destination.
Five pillars, not nineteen
A speaker in an earlier session at CDOIQ had offered nineteen different ways to do data quality. Puneet’s team went the other way and defined the pillars that fit its own data, including timeliness, completeness, accuracy, and governance. That sounds like a smaller ambition, doesn’t it? In practice, it’s the more demanding one, because five pillars that are measured on every table beat nineteen dimensions on a slide that nobody instruments.
Puneet was equally practical about how the program earned its funding. He pitched a data quality vendor for one narrow use case, showed a measurable benefit, and only then came back for the whole thing. Leadership trusting the team’s instincts didn’t mean signing a blank check. It was the the small proof of concept that translated it to a budget item.
“Each and every alert is logged, measured, and we do analysis on a weekly basis to make sure where the fires are, where we need to get better, and where we need brutal prioritization to remove them or fine-tune those.”
— Puneet Singh, Senior Data Engineer, Intuit Credit Karma
The team runs data quality checks on more than 100 tables and roughly 40,000 columns, and logs and measures every alert those checks produce. A weekly review then decides where the fires are and which alerts deserve brutal prioritization, either fixing the underlying issue or tuning the check so it stops crying wolf. Most data quality programs I’ve seen skip that review, and they end up with a dashboard full of red that everyone has learned to ignore. How many of those dashboards are in your own organization right now?
Translate the alert into people
I asked the question I ask every data leader: whose job is data quality? The business? The data engineering team? An agent? Veenit’s answer was the everybody’s-job line, and normally I would have moved on, because everybody’s job is what people say when they haven’t decided. Then he explained why it holds on his team, and the reason turned out to be a translation habit rather than a value statement.
“A simple alert could look like, ‘ Hey, we have not been able to ingest X percent of data. But what that actually means is X percent of members out of the 140 million are going to be impacted.”
— Veenit Shah, Senior Manager of Data Engineering, Intuit Credit Karma
Once an alert is read as a count of members rather than a percentage of rows, ownership stops being a governance question. A junior engineer who sees millions of people behind a failed ingestion doesn’t need a RACI chart to know it matters, and a senior manager can’t delegate it into a backlog. Veenit calls this a fundamental aspect of the team’s culture, and it’s also the pitch that I wish more data leaders would make to their own organizations. Asking people to care about the members, customers, or patients on the other side of the pipeline is a much easier sell than asking them to care about data quality.
Veenit also drew a useful line between the two sources of trouble. External issues arrive from partners and financial institutions whose data the team doesn’t control, and those are the cards you are dealt, so the only question is how to play the hand. Internal issues show up when the team onboards a new partner or builds a new feature, and better engineering practice can prevent those. Separating the two keeps the weekly review from treating an upstream mess and a homegrown bug as the same kind of fire.
The agent came last
Every conversation about data operations right now turns to agents within ten minutes, and this one was no different. Credit Karma built an AI remediation agent that picks up a data quality error, runs the diagnostic query, refines it, and keeps running queries and analysis until it can state the root cause and the blast radius. Puneet said that investigation used to take an engineer 30 to 40 minutes and now takes a few minutes. On the build side, the team has given agents the context to write production-ready pipeline code, including the data quality and observability pieces, in five to ten minutes.
Those results get headlines. So why does the agent work here when so many bolted-on agents don’t? The agent runs against alerts that are already logged and measured, on checks that three years of weekly reviews have tuned, with runbooks the team wrote for humans and then handed to the agent as explicit instructions. It operates in a protected environment, never touches personally identifiable data, and works only with anonymized data that was already approved for analytics. You can’t automate a mess, and Credit Karma’s agent finds the root cause in minutes because the discipline it runs on came first.
“The agent goes there, looks into the data quality error, runs the query, as well as refines those queries, and keeps running the queries and analysis till it comes to the conclusion with the blast radius, saving our time from 30 to 40 minutes to just a few minutes.”
— Puneet Singh, Senior Data Engineer, Intuit Credit Karma
I raised the obvious worry: agents sometimes go off the rails. The team places itself at level two on its own agentic maturity model, and everything planned for levels three and four is still under security review. Selective access, which tools an agent can use, and what its role should be will get decided slowly, and the team is comfortable saying it doesn’t yet know what levels four or five look like. Veenit put it plainly: the game itself is changing, and they have to keep playing it.
The benefit compounds
Veenit described the payoff as multifold, and the first part is the one everyone expects. Investigations that took 30 minutes to an hour are compressed into minutes, giving the team back time he described as insane. The second part is what I think matters more. With firefighting handled, the team can think about the next frontier, find gaps it didn’t know existed, and stop every data quality issue the program catches from compounding on the last one.
If your own data quality program is stuck in the firefighting years, the place to start isn’t a framework or an agent. Start by naming who is on the other side of the next alert.
Listen to the full conversation with Veenit Shah and Puneet Singh on their Data Faces Podcast episode page.
Based on insights from Veenit Shah, Senior Manager of Data Engineering, and Puneet Singh, Senior Data Engineer, at Intuit Credit Karma, featured on the Data Faces Podcast.
Podcast highlights
- [0:03] David introduces Veenit Shah and Puneet Singh on location at the CDOIQ Symposium in Cambridge, Massachusetts
- [0:27] Veenit on keeping credit scores current for about 140 million members
- [1:49] Veenit on the session, a data quality adoption journey from reactive to proactive, and the scars along the way
- [2:23] Puneet on the starter feeling, extinguishing fires, and nightly calls with no pillars to rely on
- [3:25] Veenit on the twofold mandate, incidents plus leaders who trusted the team’s instincts
- [5:08] Puneet on choosing five pillars over nineteen, and the small proof of concept that earned the program
- [5:46] Puneet on 100-plus tables, 40,000 columns, and the weekly review with brutal prioritization
- [7:07] Veenit on why data quality is everybody’s job when an alert means members
- [8:13] Puneet on the AI remediation agent and 30 to 40 minutes becoming a few
- [10:09] Veenit on level two of the agentic maturity model and the runbooks handed to the agent
- [12:21] Puneet on guardrails, anonymized data, and what stays under security review
- [13:22] Veenit on the compounding benefit and the gaps they did not know existed
About David Sweenor
David Sweenor is the founder of TinyTechGuides and host of the Data Faces Podcast. He is an international speaker, advisor, and the author of eleven books on artificial intelligence, analytics, and B2B marketing, including Generative AI Business Applications, The CIO’s Guide to Adopting Generative AI, and Modern B2B Marketing. With more than twenty-five years in analytics and AI at companies including Alteryx, Tableau, TIBCO, SAS, IBM, and Dell, David advises technology companies on product marketing, content strategy, and go-to-market execution. He holds several patents and has been named a top influencer in data and analytics by Onalytica, Thinkers360, and Analytics Insight.
Connect with David on LinkedIn and subscribe to the Data Faces Podcast for conversations with the people shaping enterprise data and AI.
Footnotes
[1]CDOIQ Symposium. “The 20th Annual CDOIQ Symposium.” July 21–23, 2026, Hyatt Regency Cambridge, Massachusetts.
https://2026cdoiq.org/
[2]Intuit Inc. “Intuit Completes Acquisition of Credit Karma.” Intuit Press Room, December 3, 2020. https://www.intuit.com/company/press-room/press-releases/2020/intuit-completes-acquisition-of-credit-karma/
[3]Intuit Credit Karma. “About Intuit Credit Karma.” https://www.creditkarma.com/about
