General Chat

Hello Lisa, 

I see no one else has answered so far. So let me put in my two cents here. This is going to be a long answer, so grab yourself a cup of coffee, tea, or whatever you like. 

To start, I am the Manager in the training department for the Bozeman Yellowstone International Airport. We are an airport authority, reporting the Gallatin Airport Authority, this will come in to context soon. We have about 85 employees currently, and with our current expansion rate, we will be well over 100 in just a couple of years to stay up with the growth of the airport. Mind you, these are airport employees, not airline or the tenant like the shops or car rental employees. 

We track a lot of data for the board. They control our budget, including the Training Dept.'s budget. We need to justify what we are spending. 

We track the basics:

• How many training classes were offered to the employees internally and externally both required and optional vs how many were completed
• How many hours of training we completed
• Compliance of required trainings, since we are governed by the FAA we have to maintain 100% compliance and we do. 
• Cost of the training (this is the total cost of off the shelf training and outside vendors)
• Cost of development and implementation of internal training 
• Location (in-house, online, Local (Bozeman), In-State (Montana), out-of-state)
• Travel costs
• Expiration date of any certifications
• Employees trained (a count of heads by the organization level, department level, job title level)
• Date of completion of training (required and optional for employee records and for recertifications if needed)
• Development process

Further information. We also do some training for the tenants and operators at the airport (the restaurants and airlines) for badging, security, and safety purposes. So we track:

• Total number of people completing these classes (used for billing)
• Completed hours of training
• Completion date (for recertifications) 

We are building and implementing a full LMS to capture this data (it is a Moodle system) and report using SQL queries to report the data to the Gallatin Airport Authority board, and to the executive team. 

Thanks Scott - I was wondering why no one answered this question which is so pertinent to the work we do. I appreciate your comprehensive reply giving context is also very helpful. Congratulations on doing such a great job tracking your outcomes supporting your organizations growth.  

Sadly, I think the reason you have not received answers is that there are too many organizations that do not track training results. I had a former team member who hated doing any evaluations of any kind. We track these too, but for internal use, not for the board. But for effectiveness of the training themselves. Therefore they do not know when or how to update their training courses. Hopefully you get more answers to this though. 

Scott - I had a feeling most organizations are doing at the minimum Level 1 evaluations and a basic level- in all transparency I am not analytics so metrics/data is not my strength but we recognize it is necessary for best business practices and in L&D if you can't measure it it doesn't count.  lol

Hi Lisa,

At my last organization, we used Metrics that Matter, supported by Explorance. In addition of keeping track of the quantitative data (courses, number completion, etc),we had a learning analytics team that assisted learning project teams on what level of measurement was appropriate for the specific project. Level 1 was consistently completed but based on how the learning project aligned with strategic initiatives, level 2- 3 was measured.  We measured level 3 primarily by 60-90 post training surveys.  In addition, we studied key elements from the survey to determine effectiveness and retention of learning.  From there, we tried to establish value.  

I agree with you that most establishments complete level 1 surveys, but learning effectiveness needs to be a foundational tenet of any learning organization. The data collected, if evaluated in a way to show trends of effectiveness will lead you to return of value.  

This is a big question, but I'd like to provide some perspective on your request for showing value in the special requests you get. To show the value of those requests, the measurement begins with you asking the requestor -- are they looking for content or are they driving change? When they say driving change (which is the best answer!) then you can ask how people are acting differently after the request. Those answers become your first metric. You can then go one step further and ask how what people are doing impacts the business. Those answers lead you to your next metric.

Setting yourself up for success begins with the initial conversation so you are identifying what matters to the requestor and tying that to your organization's strategic priorities. Other metrics like attendance, hours, etc, are nice to have but don't show the real value that your L&D is providing. 

Hello Lisa, 

Tracking, tracing, and communicating training effectiveness and efficiency is such a big opportunity for talent development in my opinion. I recently spent some time researching this topic, and this white paper outlines practices that I have long wanted to implement where I work. I believe talent development needs to speak the language of business, and having our own version of financial statements as noted in the white paper is an important step in that direction.

I am also including a link to the ROI Institute. While I don’t have direct experience with them, they share a lot of great insights. Sorry, this is a bit general and brief, but I wanted to respond and hopefully provide some useful information.

Here is the link to the white paper: https://centerfortalentreporting.org/wp-content/uploads/2023/03/whitepaper-full-version.pdf

Here is a link to ROI Institiute: Center for Talent Measurement and Reporting – ROI Institute

Here is a link to one of my #chorechats where I talk about an exercise I learned from the ROI Institute that helps you calculate training ROI with the C-suite. 

White Paper Brief summary:
The white paper discusses the Talent Development Reporting Principles (TDRp), which provide a framework for measuring, reporting, and managing talent development in ways that align with business practices. It introduces the idea of creating standard statements—similar to financial reports—that show training outcomes, efficiency, and effectiveness. This approach helps talent development professionals demonstrate value, improve decision-making, and better connect their work to organizational results.

If you have any questions, please let me know. 

David

Hi Lisa,

In addition to what Scott outlined (which we do as well), we also track the L&D team's time. For example, how much time do they spend facilitating versus creating training materials. We use this information to better plan projects, deadlines, and training.

We use Kirkpatrick's four levels of training evaluation to measure training impact and effectiveness. Level 4 is where we can truly measure how our training is supporting our organization. Nearly every training we do is aligned with measurable organizational goals, and we do this to ensure that we aren't doing training just to do training.

We measure how much time employees spend on training outside of their mandatory training requirements. Our LMS, Axonify, gives us these metrics and has helped us see what employees want to learn and how much time they are allocating to it. 

Another thing we measure is knowledge growth. Again, using our LMS, we are able to see what the baseline is for knowledge on a subject and see how it grows (or doesn't) over time. This helps us identify any areas and/or employees that need additional training.

I have a monthly meeting with the President, and one with the Chief Operations Officer. I am also on the board of directors, so I have the opportunity to share these metrics frequently. We also have an L&D Committee that meets bi-monthly, and several of our SVP-level managers participate in this meeting. We share metrics, plan training, identify needs, etc. during these meetings. We also have a couple of dashboards through SmartSheet. One shows how time is spent in L&D, and one tracks the training evaluations we have employees complete after each training.