2021 Knowledge ∞ Continuum

March 05, 2021

The Center for the Management of IT (CMIT) designed the 2021 Knowledge ∞ Continuum as a year-long virtual executive education engagement.  However, the final quarterly event in October 2021 will be held on UVA’s Grounds in conjunction with the Center’s 30th-anniversary celebration.

The 2021 cohort was limited to 50 participants (Patrons) to facilitate an interactive format and ample networking opportunities. Four quarterly events include a conversation with a featured expert or author moderated by McIntire IT faculty, followed by breakout group discussions to debrief and network.  Throughout the year, we will update this recap with details on each quarterly program, including a few key takeaways from the speaker and session.

If you are interested in joining the 2022 Knowledge Continuum, or just want to be added to our mailing list to be informed about future CMIT events and opportunities, please send an email to amandamorris@virginia.edu.


March 5, 2021: Digital Product Management with Marty Cagan, Founding Partner of Silicon Valley Product Group (SVPG) and author of EMPOWERED: Ordinary People, Extraordinary Products

Check out Marty’s blog post about “Product vs. Feature Teams,” in which he argues that empowered product teams are focused on and measured by outcomes and their purpose is to solve problems in ways the customer loves, yet work for the business. Feature teams are all about outputs and are there to serve the business.

  • Empowered product teams are a key element of many of the world’s most successful companies, including Netflix, Amazon, Facebook, Google, Spotify, CarMax, Guardian, and Atlassian.
  • Transformation be led from the top. What convinces CEOs to embrace empowered product teams? The profit motive is a primary driver, but fear can also be a powerful motivator. Scare companies by asking them to imagine that Amazon is coming to “clean your clock”; tech disruption is everywhere (don’t be Circuit City).
  • Transforming to the empowered product team model impacts far more than just the product team. Companies stop funding projects and start funding product teams in pursuit of outcomes. Empowered product teams don’t care about launching a new feature; they care about increasing profits, reducing retention issues, and other corporate milestones.
  • The role of IT needs to shift from cost center to revenue center, from serving the business to serving customers.
  • Three steps toward change:
    1. Hire skilled product leaders (or coach).
    2. Build real product teams (product manager, designer, engineer).
    3. PM leaders need to reintroduce product teams to the organization and move from subservient to a collaborative model.
  • Product vision is the “North Star” for product-driven organizations: It identifies the common goal, the future you’re trying to create, and how you will improve the lives of your customers. This vision should be focused on 3-10 years out.
  • Visiontypes of the future you’re trying to create need to be created: These are high-fidelity user prototypes (e.g., videos or storyboards) and can be used as a recruiting tool and an evangelism tool.
  • How to become a product manager?
    1. Go to work for a leading tech product company for 1-2 years.
    2. Associate PM programs are one- to two-year coaching programs.

Speed is essential to innovation. Innovation is a function of the number of “at-bats,” or attempts. “You want to fail in discovery, not fail in delivery.


May 7, 2021: Digital Transformation, Organizational Agility, and the Role of IT Leadership

Shamim Mohammad, Executive VP and CIO/CTO, CarMax Inc.

Overseeing digital innovations, technologies, and product engineering at CarMax to deliver revenue-generating products and solutions that drive the company’s competitive advantage, Mohammad has re-platformed major technologies, developed world-class supply chain capabilities, and elevated CarMax’s cybersecurity posture. He received the School’s CMIT IT Leadership award in 2019, is the recipient of the 2020 CIO Leadership Award from the MIT Sloan CIO Symposium, and was recently named to the Forbes’ 2021 “CIO Next List,” which spotlights 50 innovative technology leaders who are reshaping and redefining the role. Peter High recently cited Mohammad as part of the inspiration for the title of his latest book, Getting to Nimble: How to Transform Your Company into a Digital Leader.

In advance of the session, Patrons reviewed the case study “CarMax: Driving What’s Possible” by Professors Ryan Nelson and Ryan Wright. CarMax, based in Richmond, VA, is the largest retailer of used cars in the United States and has undergone a complete digital transformation over the past several years. The case describes the company’s technological evolution and product-driven response to new competition and changes in consumer behavior leading up to 2019. In the years since CarMax has steadily expanded its addressable market in the used car ecosystem while leveraging an omnichannel business model. On May 6, 2021, CarMax held an “Investor Analyst Day” to share with the investor community where the company will be in four years, announcing a bold projection to become a $33 billion company, growing market share from 3.5% to 5% by 2025.

Conversation Highlights

Corporate Culture and the C-Suite

  • Digital transformation requires that the entire C-suite be on board with the changes being proposed and introduced. Senior leadership—including the board of directors—must be engaged and aligned.
  • When Mohammad joined CarMax, he sought to create a recognition for a need for change by exposing the leadership team to technology-driven companies and conveying the need for a 360-degree view of their customers, including robust, real-time CRM capabilities.
  • As leadership aligned and early wins came, excitement grew, and product-driven thinking took hold. Growth from approximately five product teams several years ago to 50+ teams in 2021.
  • CarMax embraces teamwork and collaboration while recognizing the need to cultivate an exceptional talent base.

Empowered Product Teams

  • Start small. Create an operating model with a handful of teams that understand the mission—tell them what needs to be accomplished but not how to accomplish it.
  • Teams must feel empowered and have the latitude and flexibility to experiment.
  • Learn cheap and learn fast, without the fear of failure—empowerment is closely tied to how the leadership team behaves and reacts, especially to experiments that don’t work.
  • It is important to identify the business drivers and outcomes sought—it is very difficult to audit product teams based on traditional timelines, deliverables, or measurements.
  • Prioritization is up to the team—after they receive high-level business goals, product teams determine on their own how to achieve those goals.
  • Product teams are durable—teams are kept together, but their goals can shift as business objectives evolve and change.
  • Project teams (typically seven to nine people) present to the entire company every two weeks as part of CarMax’s “open houses” to share their progress toward goals and future plans.
  • The time commitment of four to five hours to attend open houses every two weeks enables the leadership team—and the entire company—to be engaged and remain nimble through a high level of transparency and collaboration.

Lessons Learned

  • Early on, product teams were split up based on brick and mortar versus website experiences. Over time, teams were reorganized to be based around customer journeys, with an overarching organization into retail, wholesale, and capabilities teams.
  • CarMax’s product team structure is constantly changing based on what works and what doesn’t, as well as changing market demands.
  • CarMax’s response to the pandemic was highly flexible and resilient, in large part due to their culture and empowered product team structure.
  • Technology and architecture strategy—rent, buy, build. Build what drives the company’s competitive advantage. Rent or buy what already exists and works well.
  • CarMax has embraced DevOps practices and continuous deployment to facilitate agility.
  • CarMax places a tremendous focus on data—hiring data scientists and engineers, building a data lake, focusing on data quality and governance, and employing machine learning.
  • Faced with tremendous uncertainty and competition, CarMax was forced to become a much nimbler company—transforming their culture, organizational structure, and work processes, with an emphasis on empowered product teams.

August 20, 2021: The Future of AI/ML in Business – A Conversation with McIntire faculty

Stefano Grazioli

Our third quarterly Knowledge Continuum program of 2021 brought together five of our IT/AI faculty for a non-technical conversation about each of their artificial intelligence (AI) or machine learning (ML) projects, with an emphasis on the managerial implications of their research.   Professor Stefano Grazioli, Director of the M.S. in the Management of IT Program at the McIntire School, moderated the discussion with Associate Professor Jingjing Li and Assistant Professors Brent Kitchens, Sarah Lebovitz, Reza Mousavi, and Lanfei Shi.  After a brief introduction from Professor Grazioli regarding the application of state-of-the-art AI and ML techniques and technologies in today’s businesses, each faculty presenter shared a brief TED-style talk on their particular area of study.

Reza MousaviReza Mousavi discussed his study into the technical and organizational challenges faced when deploying AI-based solutions within a large insurance company.  The premise was the organization’s transition from a traditional risk-assessment model to an AI-based algorithm to improve the recommender system, and ultimately the conversion rate, in their online customer experience.  Technical challenges included consistency of data, privacy issues, and equity of risk management and mitigation across departments.  Related work:  https://www.tandfonline.com/doi/full/10.1080/07421222.2020.1831775

Lanfei Shi took a deeper dive into recommender systems with a description of the design rationale behind the types currently encountered by consumers, namely accuracy-based systems driven by user profiles and/or past behavior (purchases, clicks, views), and diversity systems recently employed to broaden the offers and improve the personalization of results. Shi also described the ‘state-dependent recommendation framework’ that depends on the users’ current short-term consumption rate.  Related work:  https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3833940

Brent KitchensBrent Kitchens shared some key findings and implications of his research on social listening with context-aware AI to help address public health issues.  A wealth of content, including social media posts and google searches, exist online that can be explored to inform and improve health outcomes.  Professor Kitchens specifically described his efforts to monitor adverse events like drug reactions and product safety issues, and the importance of imbuing context and user intention in the analysis.  Related work:  https://ieeexplore.ieee.org/abstract/document/9171579

Jingjing Li presented on building machine learning resilience, a concept that has taken on new significance during the COVID-19 pandemic, which revealed that ML models were overly dependent on the assumption that historical patterns will repeat.  Professor Li focused on her research examining a recommender system aimed at promoting travel insurance during online air ticket purchases, and how changes in customer preferences and behaviors during the pandemic essentially “broke” their system by failing to respond to the unique impact of the pandemic on air travel including rapidly changing schedules, rising prices, health risks of travel, and location-specific issues (quarantines, testing, etc.)

Sarah LebovitzSarah Lebovitz is an organizational researcher and rounded out our session by discussing her approach to investigating the real-world implications of AI through field research, partnering with companies or institutions to observe and provide recommendations.  Specifically, Professor Lebovitz shared details about the ten months she spent inside a major US hospital system following managers, data scientists, and radiologists to learn how they develop, implement, and use their AI tools to inform – or conflict with – the specialists’ diagnostic recommendations and decisions for patients.  Related work:  https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3839601

Following these brief talks, Professor Grazioli moderated an interactive discussion between the faculty and our Knowledge Continuum patrons touching on regulatory compliance, the challenges of bias, and reiterating that existing AutoML tools can be of great value when imbued with the proper context.