CIO’s guide to creating a dashboard for tracking innovation projects

Dashboards are excellent tools for healthcare CIOs and other executives to keep track of projects, strategies and progress. But what about tracking an organization’s innovation? Can that be done? And can it be done on a dashboard?

The answer is yes. And forward-looking healthcare providers including Beth Israel Deaconess and NewYork-Presbyterian have innovation dashboards, or larger project dashboards on which innovation is also tracked. It’s important for executives to know if they’re attempting to stay out in front of the field, and if they’re succeeding in doing so.

What an innovation dashboard tracks

Beth Israel Deaconess Medical Center maintains a dashboard tracking all the elements of its information systems strategic plan, and part of that tracking is measuring how innovative the work being done truly is (see dashboard graphic).

“We develop a six-quarter plan continuously, rather than having fixed yearly plans, because the current pace of innovation requires agility,” said John Halamka, MD, CIO at Beth Israel Deaconess System. “Our approach is to determine business requirements from stakeholders – our 30-member governance committee – then survey emerging technologies, then suggest project tracks that explore novel technologies to support operational priorities.”

Current tracks on the dashboard include: cloud migration; mobile apps/EHR Plus, including the notion that real innovation occurs outside of the EHR via a suite of mobile apps that communicate with the EHR; machine learning/AI; Internet of Things; and enhancing productivity, for example, consider expanding the notion of Bring Your Own Device into Bring Your Own Computer, with Chromebooks.

“The major focus of our current work is embracing machine learning to predict workflow proactively instead of old school business intelligence that tells you about problems in the past long after they’ve happened,” Halamka noted.

Manu Tandon, CIO at Beth Israel Deaconess Medical Center, which is a part of the Beth Israel system, explained how the organization is tracking the innovation sections underneath the Beth Israel Deaconess Medical Center Priority Projects to Effectively Utilize Available Capacity in Service of Extraordinary Care section of the dashboard.

“We feel innovation is best when it’s embedded in regular operational practices and its development follows the same governance, triage, development and deployment path as the bulk of our regular EHR development does,” Tandon said. “The primary reason to track some items separately as innovation is to highlight the fact that technology advancements in non-healthcare sectors can be very useful for healthcare.”

The provider’s innovation center focuses on bringing in innovation from the consumer sector, working with leaders in that sector. It currently is focused on natural language processing, machine learning, AI, beacons, mobile apps, home monitoring, team collaboration and augmented reality, and their applicability for healthcare use-cases.

“We are especially focused on using these to maximize the operational capacity of our tertiary care hospital,” Tandon said. “How to identify and remove waste, how to discharge patients promptly, how to maximize the use of our operating rooms, how to reduce appointment no-shows, how to predict census spikes, how to reduce readmissions, how to enhance patient experience and access, how to enhance safety and quality, how to reduce wait times, how to dynamically schedule resources, how to make sure we provide the right bed at the right hospital to the right patient, how to enhance team collaboration, and how to synchronize services access in the entire hospital are some of the operationally driven questions we target innovative tools at.”

The clinically governed IT strategic plan dashboard intermingles innovative projects with regular development ones and highlights how they need to work together to provide maximum value.

Must-have KPIs

So which key performance indicators are necessary to be tracked on an innovation dashboard? Healthcare CIOs have clear ideas here.

“Innovation projects need standard tools and definitions – a well-organized team of analysts, subject matter experts and end users need to collaborate to define project goals, KPIs and measures of success,” said Daniel Barchi, senior vice president and CIO at NewYork-Presbyterian. “Our dashboard outlines our work in broad areas of telemedicine, artificial intelligence, robotic process automation and robotics, with sub-metrics on legal contracts, project management, staffing and testing.”

Gil Hoffman, CIO at the St. Louis-based Mercy health system, explained that while they don’t maintain an innovation dashboard per se, they have a “value” dashboard that qualifies the value of a project to the organization. One of the criteria for the value score is based on the innovativeness or ability to disrupt the market.

“Our key performance indicators make up the criteria for assessing the value,” he said. “New projects/ ideas are filtered and scored prior to the decision to invest. The criteria includes regulatory, safety, patient outcomes, patient experience, productivity, revenue impact, reach and innovativeness. We also consider the resources, time and level of difficulty to implement.”

Not all of the criteria are weighted the same; for example, regulatory and safety are weighted higher due to the nature of what Mercy does, and patient outcome, experience and productivity also are weighted higher as those tend to drive more innovative solutions or transformational benefits leading to disruptive changes in healthcare, he added.

Tracking innovation processes

In addition to key performance indicators, there are important processes that healthcare CIOs track on their dashboards that are part of the overall innovation work. It’s crucial to monitor these processes as they relate heavily with the key performance indicators and help tell the tale of innovation within an organization.

“As a CIO, it is important that there is oversight to the governance process for scoring and approving as well as tracking the technology market as the project progresses through development and implementation,” said Hoffman of Mercy. “With the rate of change of technology, it is critical to understand if the market is changing, does the scope of the project need to be adjusted, and does that change the expected value or return on the project?”

Barchi of NewYork-Presbyterian added that it is important for a healthcare CIO to think about processes and innovation from two directions.

“First, what problems exist in the health system and how can we apply innovative technology to address them,” Barchi said. “And second, what innovative technology is out there and how can I make my colleagues aware of it as a potential tool to improve efficiency and patient outcomes?”

On innovation dashboards or dashboards that include innovation as a major factor, healthcare provider organizations should be tracking the implementation of innovative projects, proofs of concept and the like.

“Innovation projects are just like any others – project managers and leaders need to track milestones, budgets and metrics,” said Barchi of NewYork-Presbyterian. “One element to add on innovation projects is return on investment. Tracking the ROI of innovation projects is important because this work is often an investment in the future – recognizing that the early time and dollars spent on innovation are down-payments on future patient benefits is important.”

The key to being able to track the value and implementation of an innovative project is to have a base of the existing status quo or current state benchmark that is/will be impacted by the implementation, said Hoffman of Mercy.

“For example, while we focus on patient areas the most, it is important that we understand did we improve patient outcomes, experience, provider productivity and how that impacts the patient?” Hoffman said. “Did we provide new services or deliver them in a revolutionary new way that we can demonstrate compared to the current state benchmark or against the original value score? This is where we still need to improve as we measure value improvements, we do not always measure against what we used as our justification of the project.”

How to tell if your dashboard is working

In the end, how does a provider organization know that an innovation dashboard is working. Is it providing the kind of tracking and insights that are valuable to CIOs and valuable to helping push the progress of innovative projects?

“An innovation dashboard is primarily a tool of the IT innovation team to advance their work,” contended Barchi of NewYork-Presbyterian. “The tool is an effector arm of the larger innovation infrastructure of the organization and it works when it is helping focus use of innovation to improve operations.”

Healthcare leaders need to be communicators – the dashboards are tools to keep colleagues and customers informed, though it still is the job of the CIO to tell the story of how innovation is beginning to move the needle on efficient operations, quality metrics and patient outcomes, he said.

“Our value scoring is relatively new and began as a means to allocate resources and investment into technology projects and opportunities that would have the greatest impact on or return on investment for our organization and the people whose lives we are entrusted in caring for,” said Hoffman of Mercy. “A multi-functional team of clinical, operational and managerial coworkers developed the value score matrix and applied it to the existing pipeline of projects.”

As a team, the coworkers determined those projects with high value and those that would be put on hold. The team meets monthly to review progress and new opportunities.

“So far it is working well,” Hoffman said. “As a technology organization we are delivering value we can prove.”