Tuning in with AI: How MLB My Daily Story creates truly personalized highlight videos

Even in a baseball stadium filled with 41,649 fans, no two are exactly alike. Most are there to root for the home team, but a few pockets are cheering on their opponents. Some are waiting for those thrilling double plays, others counting up strikeouts in their box scores, or holding their breath for a walk-off home run. Some just want to see the antics of the fourth-inning hot dog race .

Major League Baseball can tell you how wide and varied its fans’ interests are — it has the data and insights. Millions of users log onto MLB App each season, as well as watching on MLB.TV and at the ballparks, which are increasingly dotted with data-capturing technology. Over the years, the league has honed its understanding of these data streams, using them to deliver increasingly better and better fan experiences .

On Opening Day this year, MLB delivered its latest technological fastball, one that has a little extra mustard on it thanks to AI. It’s called My Daily Story.

Since its launch in 2008, MLB App has been a popular venue for game highlights; most were sorted by teams or clip types, though, and fans would have to scroll through to see what they liked. Over time, a recommendation engine helped suggest more relevant clips. Now, with My Daily Story, the league has taken this idea to the upper decks. Using both conventional machine learning and newer gen AI models, all built with Google Cloud, My Daily Story is able to cut together a personalized highlight reel of the previous day's games.

“There's a psychological aspect to personalization,” explains Josh Frost , senior vice president of product at MLB. “If I know something is built for me, it should be more valuable. And that connection is so important these days.”

Just as teams regularly make trades and evaluate their rosters, the MLB App rolls out new features throughout the season to excite fans and keep them tuning in. My Daily Story is one of the most ambitious projects yet, given gen AI’s ability to spot trends and fan preferences, whether they come out of left field or anywhere else in play.

We sat down with Frost to learn more about how My Daily Story was built, how gen AI helps tackle “thinking” problems and gives the reels it creates a sense of drama, plus where MLB sees AI playing a role next (you may just have to listen up for that one).

Over the past decade, the league has invested a lot in technology to enhance the fan experience. What does My Daily Story bring to the table?

My Daily Story allows fans to see highlights in a format they’re most familiar with, similar to Reels on Instagram, YouTube Shorts, or TikTok. We already had this at a broad level in MLB Game Stories — collections of game highlights tailored to specific interests — but we’ve always dreamed of a more personalized version.

Baseball fandom is deeper and more nuanced than a favorite team. If someone is a fan of the Cubs, but also a fan of MLB: The Show , our video game series, and still wants to stay informed of Cubs players that have gone somewhere else, or of promising young talent, their baseball fandom is already four-fold. My Daily Story draws on those interests, from using MLB apps and platforms and other fan data. With AI, we then package content together based on those interests in a really fun and concise format.

Someone can still watch the full game, and that’s just the beginning. My Daily Story is really an appetizer to show everything that happened yesterday for your favorite team, the teams and players you follow, and other national content, like a player beaming about his new baby . We wanted to create a product that could be part of someone’s daily routine, catching them up on everything that happened yesterday — personalized to who they are and what they want to see. It’s a new daily reason for fans to start their journey in the MLB app every morning during the season.

What factors are fueling the current journey for MLB with AI and why it’s now possible to bring projects like My Daily Story to life?

Personalization has been a major initiative for us for several years. In an attention economy, you need to compete even harder to earn someone's time. As a consumer, I only have a certain amount of time in the day. Why waste it on something I’m not interested in?

One of our core tenets has been to bring our products up to a modern consumer level, where almost every facet of your digital life is expected to be personalized. People control their own time, so you’ve really got to make it worth their while — just because you think it’s important may not be enough.

You’ve had some of these personalization features for a while…

It’s the step change from machine learning to AI to generative AI. We can deliver a much more enhanced experience, combining traditional programming with machine learning and gen AI.

Baseball is a sport of many disciplines — pitching, hitting, fielding, running, and more. Using gen AI, our systems have a better understanding of what fans are engaging with. We’re able to fine-tune our recommendations and algorithms based on what people watch and don’t watch to better deliver what people are interested in seeing.

We talked a lot about explicit signals like players and teams. But the implicit signals we’ve found a lot of engagement on have been really cool to see, too. For example, people that love pitching want to see a pitch that starts up here and curves down, and the batter has no clue on how to swing at it. Some people appreciate speed, some people just want to see a home run go 450 feet to dead center.

Can you cover some of the ways AI technologies play a role in how you tackled this project?

We have metadata that helps break down and categorize different types of video content. We do a lot of recording on every single pitch. We have our Hawk-Eye tracking system that helps us capture a lot of that data. We also use neural networks and computer vision to help distinguish between pitch types and give additional color to metadata. This then allows us to categorize between fastballs, home runs, etcetera.

In addition, we found that to deliver content at this scale, even if you can get the correct “playlist” of content, you want it to be a cohesive package. We’re not building one homepage or app — we’re building thousands, even millions. A human can’t really review all of those. This is an important place where generative AI comes in, it can help modify and augment the metadata descriptions of plays in English and Spanish. This gives us a lot more scale that we wouldn’t otherwise be able to do.

It sounds like the models powering My Daily Story are trained to understand what would create a dramatic and engaging video. How does that work?

My Daily Story is not a one-time achievement; it’s a daily product that fans with a wide variety of interests are expecting to have low latency, high uptime, available every morning in our mobile app. So, it was more operationalizing everything and getting it to deliver at the level of excellence we expect. Certainly, some of that was tuning and getting feedback on the play descriptions, but gen AI, through its predictive nature, can be trained on a bunch of old highlight reels to better distinguish what makes a good one.

We also recognize that personalization, as excellent as it’s become over time, is not a silver bullet. There’s still a really strong editorial hand in showing fans something they didn’t even know they wanted — from quirky content like a dad catching a foul ball in a beer that wouldn’t normally be in your feed to surfacing a significant play in last night’s Cardinals game because it affects your division race.

Where do you see the future of MLB headed with the ability to tap into new advancements like multimodal capabilities?

As far as next steps for gen AI and personalization, we are thinking about friction points. What is something that is difficult that would better serve fans if it was personalized? The other area is around opportunity — something that you don’t know is an issue but would improve your day-to-day life or experience.

We’re already actively exploring the audio space. That might be something like generating audio that can act as a companion to fan experience. Marrying the intimate nature of audio with our content in a new way could be really valuable for fans.

What are you doing to get in touch with fans that might be applicable for other organizations?

We love using a combination of both quantitative and qualitative data. Some of our best insights come from conversations with fans. We’ve done some research with key markets abroad and that has been really insightful to think about how people relate to baseball. It’s a truly global game.

We have customer panels, which are driven by key audience segments. What do they have to say? What problems are they trying to solve? Often, it doesn't need to be particularly expensive. It doesn’t need to be a six-figure research field study. You can have between five to eight conversations around a topic and get to the insight that you're looking for. Just go talk to your customers. You’d be surprised what they would tell you.

As much as we love product analytics, it needs to be rounded out by customer conversation. It takes asking questions and, frankly, listening, and not assuming you already know the path. You have to be willing to be wrong and change your mind.

Major League Baseball trademarks and copyrights are used with permission of Major League Baseball. Visit MLB.com.