INTERNAL AUDIT REPORT Operational Audit Equity Policy Directive Compliance April 2023 – October 2024 Issue Date: November 13, 2024 Report No. 2024-16 TABLE OF CONTENTS Executive Summary 3 Background 4 Audit Scope and Methodology 5 Schedule of Observations and Recommendations 6 Appendix A: Risk Ratings 10 Executive Summary Internal Audit (IA) completed an audit of the Equity Policy Directive (Directive) for the period April 2023 through October 2024. The audit was performed to comply with Section 6, Item 3 of the directive, which states that “Internal Audit shall conduct an audit of the outcomes of the policy requirements outlined in this directive on a periodic basis.” In its efforts to become a national and regional leader in achieving equity and social justice, the Port of Seattle (Port) created the Office of Equity, Diversity, and Inclusion (OEDI) in 2019. From this initiative, certain milestones were developed throughout the years, with one of them being the Directive. The Directive was formally adopted in April 2023 by the Port Commission to serve as an overall guide for the Port’s direction. Internal Audit selected significant clauses within the Directive, performed testing, and used the results to assess compliance. In general, Port management’s compliance aligned with most policies and procedures included in the Directive, such as establishing a permanent internal Change Team at the Port, setting annual departmental Equity, Diversity, and Inclusion (EDI) goals, utilizing an equity in budgeting toolkit, working with external stakeholders to further advance equity initiatives, etc. However, our audit identified opportunities where internal controls could be enhanced or developed. These opportunities are listed below and discussed in more detail beginning on page six of this report. 1. (High) Port-wide compliance with the annual mandatory racial equity training stated within the Equity Policy Directive is not being enforced. In 2023, only roughly 453 employees, representing 19.6% of the Port, completed the mandatory Equity, Diversity, and Inclusion (EDI) training requirement for the year. 2. (High) The Human Resources (HR) report used by OEDI in their analysis of employees’ training progress for the year is not complete and accurate. The data generated from the Port’s Learning Management System (LMS) cannot be fully relied upon due to discrepancies noted during testing. We extend our appreciation to Port management and staff for their assistance and cooperation during this audit. Glenn Fernandes, CPA Director, Internal Audit Responsible Management Team Bookda Gheisar, Sr. Director Equity, Diversity, & Inclusion Katie Gerard, Sr. Director HR Background In 2019, the Port committed to being a leader in regional and national efforts to achieve equity and social justice. As part of the Port’s comprehensive Century Agenda Strategic Plan, the Port created and established the first-ever OEDI for a U.S. port authority. From the 2023 OEDI Annual Report Presentation, the main goal of OEDI is to transform the Port into an organization that “embeds equity and justice into all operations, and to create an organization where all people have opportunities for success.” Since 2018, the Port has made numerous advances towards equity and social justice, with OEDI playing a pivotal role and taking the lead on these initiatives and programs as seen in the graphic below. Source: OEDI Annual Report Presentation In April of 2023, the Port Commission formally adopted the Equity Policy Directive. Its main purpose is to “guide the integration of equity, diversity, inclusion, and belonging into the Port’s practices and policies, and to move our work beyond compliance and mandates towards long-term commitment and sustainable systems change.” The Directive serves as a broad guidebook for the Port’s overall strategy and vision related to Equity, Diversity, and Inclusion. It encompasses various policies including structural, operational, external-facing, and environmental justice. To evaluate the effectiveness of the Directive, a report to the Commission is presented annually, detailing the following: each department’s equity goals and progress made towards achieving these goals, the most significant barriers that employees face in fairly accessing Port resources and opportunities, and continuous process improvement opportunities. Results from annual surveys and assessments are shared and made public. Additionally, IA is tasked to conduct an audit of the outcomes of the policy requirements outlined in the Directive on a periodic basis. Audit Scope and Methodology We conducted the engagement in accordance with Generally Accepted Government Auditing Standards and the International Standards for the Professional Practice of Internal Auditing. These standards require us to plan and execute the engagement to obtain sufficient, appropriate evidence to support our findings and conclusions based on the engagement objectives. We believe the evidence obtained provides a reasonable basis for our findings and conclusions. In some instances, we used judgmental sampling methods to determine the samples selected for our audit test work. In those cases, the results of the work cannot be projected to the entire population. The period audited was April 2023 through October 2024 and included the following procedures: Document Review In addition to the Equity Policy Directive itself, IA reviewed various key documents and reports, such as: * 2023 OEDI Annual Report Presentation * 2023 Belonging and Inclusion Survey Results * Equity in Budgeting Playbook * EDI Assessment Report * Port of Seattle Equity Index Memo * Port of Seattle Women of Color (WOC) Assessment Results * Language Access Assessment Presentation, including Language Access Flyers Inquiries Interviewed employees in key roles within OEDI to gain a better understanding of: * The overall directive * Related and existing detective controls in place * Opportunities for improvement * Mandatory training requirements, including overall compliance and challenges experienced Testing Compliance with Annual Equity Training: * Obtained and reviewed reports from the Port’s Learning Portal, also commonly known as the Port’s Learning Management System (LMS) * Performed Port-wide testing to assess whether the mandatory annual equity requirement is being met by all Port employees. The requirements are: five (5) hours for all employees with no direct reports and six (6) hours for employees in a supervisory/managerial role. o Randomly selected 10 employees from different departments across the Port o Obtained each employee’s LMS Transcript, to review training courses attended for the year 2023 and 2024 o Cross referenced information from the LMS transcripts against the LMS reports provided by OEDI and HR, to determine accuracy and completeness o Investigated any variances and differences Schedule of Observations and Recommendations Port-wide compliance with the annual mandatory racial equity training stated within the Equity Policy Directive is not being enforced. In 2023, only roughly 453 employees, representing 19.6% of the Port, completed the mandatory Equity, Diversity, and Inclusion (EDI) training requirement for the year. The Port of Seattle’s Century Agenda includes the goal of becoming a model for equity, diversity, inclusion, and belonging. According to the Equity Policy Directive, the training is designed to support the growth of equity culture and practice among Port employees, to deepen employee understanding and awareness of systemic, institutional, and anti-Black racism. The Directive lists two training requirements: 1. These trainings shall consist, at a minimum, of a mandatory annual racial equity training for all Port employees. 2. Individuals in leadership or supervisory roles shall undertake at least one training, orientation, or other learning opportunity to advance a culture of belonging and inclusion per year in addition to the mandatory annual equity training requirement. OEDI determined that mandatory training would be five hours annually, with individuals in leadership or supervisory roles completing six hours. In the testing performed, IA noted that compliance with this mandatory requirement is lacking. As of December 31, 2023, only 453 out of a total of 2,314 employees, or 19.6%, successfully completed the training requirement. See table below for a detailed breakdown by department and position: 2023 Completion Status of EDI Training Requirement Department Manager Non-Manager Total Aviation 53 195 248 Central Services 63 82 145 Economic Development 3 6 9 Maritime 24 27 51 Grand Total 143 310 453 Additionally, from the detailed testing we performed, IA noted that for 2023, zero out of the 10 sampled employees were compliant. Meanwhile, as of October 2024, only two out of the 10 sampled employees were compliant. Based on discussions with OEDI, they are aware that overall participation has been low. However, mandatory training is challenging to enforce since there are no repercussions if an employee does not meet the minimum requirement. Furthermore, the burden is primarily placed on each employee and the employee’s manager to ensure that the training requirements are being met annually. Recommendations: 1. Continue building and developing the “EDI Training Requirement Dashboard” that the department is currently in process of developing to support tracking. This would ensure that training requirements are being tracked and monitored annually by OEDI to track compliance. Once it has been tested and validated, consider sharing with employees Port-wide so they are also able to track and monitor their overall progress throughout the year. 2. Collaborate with HR to explore the possibility of correlating the successful completion of EDI training with employees’ annual performance ratings. While EDI goals can be pre-populated into employees’ annual goals in Performance Link, the current system relies heavily on each individual to self-report accurately and for managers to validate the accuracy of the data. 3. Due to the low completion rate Port-wide, OEDI should determine the reasons for low participation. Whilst there is a mandatory training requirement within the Directive, the five- and six-hour requirement for employees and supervisors/managers, respectively, was dictated and decided upon by OEDI. By reducing mandatory hours, the Port may see an increase in overall compliance. Management Response/Action Plan: Accurate tracking and reporting of the annual EDI training requirement has been a challenge since the requirement was implemented. Most of this challenge derives from complications with LMS. However, OEDI and HR have been working closely together to generate completion reports and a corresponding dashboard, as referenced in the first recommendation. We will continue to refine this tool, and once we can ensure its accuracy, we will make this tool accessible with both supervisors and employees. We believe it will be an effective way for individual employees to track their hours and for supervisors to hold their teams accountable. Our goal is to have this completed by the end of Q1 2025. We also agree with the observations and issues identified related to the second recommendation. While this is a requirement, as per the Equity Policy Directive, there is no consequence, per se, for failing to meet the requirement. This does not seem to be specific to the EDI training requirement but seems to be an issue related to all required training. Doing more to incentivize the completion of this requirement could create a positive impact. OEDI will partner with HR to explore what is possible, including the suggestion of correlating this requirement to an employee’s annual performance rating. We will also connect with our regional and national partners to see if any best practices and/or policies may be helpful with this issue of enforcement and accountability. Our goal is to have a new policy in place in time for 2026. Finally, regarding the third recommendation, OEDI is in the process of evaluating the most effective ways to engage employees while also assuring that our organization is deepening its understanding of and ability to practice racial equity. We will examine the hours requirement and experiment with some models in the coming year. Our goal is to roll out any adjustments by Q1 2025. The HR report used by OEDI in their analysis of employees’ training progress for the year is not complete and accurate. The data generated from the Port’s Learning Management System (LMS) cannot be fully relied upon due to discrepancies noted during testing. Throughout the year, OEDI offers various opportunities, such as In-Person Workshops, Online Classes, Lunch and Learns, and Virtual Learning Courses, for all employees to be able to meet the mandatory annual EDI training requirement. Employees register for these opportunities online through LMS for tracking and recordkeeping purposes. As part of our testing, IA randomly sampled 10 employees from various departments within the Port and reviewed their 2023 and 2024 LMS transcripts to determine what EDI courses they registered for and completed each year. For 2023, we noted that three of our 10 samples had discrepancies between their transcripts and the HR report. More courses were listed on the employees’ transcripts versus the HR report. For 2024, we noted that one of our 10 samples had discrepancies between the individual’s transcript and the HR report. Like 2023, more courses were listed on the employee’s transcript than what was listed on the HR report. As a result, the HR report undercounted and underreported the hours completed by the affected employees. Consequently, the total hours reflected in the report were incorrect and misleading. According to OEDI, the report from LMS is run by HR on OEDI’s behalf. Essentially, the report is a data download of individual records showing all the different courses and workshops completed by Port employees, the respective completion date, number of credit hours awarded, and other employee specific information (such as division, job title, employee ID, etc.). All changes made to this report are done manually and are a collaborative effort between both HR and OEDI. From this report, OEDI then analyzes the information and creates a dashboard with filters to track all employees’ progress. Below is a redacted screenshot of the dashboard (employees’ names have been redacted for privacy): This reporting tool is extremely new and still a work-in-progress. As such, OEDI has not had a chance to fully validate and test the accuracy of the information. Based on our testing and conversations we have had with OEDI, IA determined that the information from LMS is inaccurate and cannot be relied upon. Overall, this is problematic as employees, as well as OEDI, do not have a reliable tool on hand to ensure that employees are on track with training requirements. Recommendations: 1. The discrepancies in LMS need to be identified and resolved. OEDI should consider working with various departments like ICT and HR to pinpoint the root cause(s) of these differences. Once remediated, this will result in accurate analysis and dashboard reports. 2. OEDI should collaborate with HR to learn how to fully utilize LMS so that OEDI can generate their own reports, eliminating the need to rely on HR. This will also eventually reduce HR’s workload related to OEDI-specific requests. Management Response/Action Plan: As discussed in our response to the first issue, we are in the process of working closely with HR to identify the discrepancies so that we can generate accurate reports and therefore accurate dashboards. Again, we hope to share this dashboard widely and believe it will be a useful tool for both individual employees and supervisors in tracking and accounting for their hours accurately. It will also be a useful tool for OEDI and leadership, displaying the aggregate/big-picture view of this training requirement across the organization, every division, and every department. HR is currently working with and training two members of the OEDI team to utilize LMS more effectively, which includes being able to generate these reports. HR is doing this with other departments as well, and they are creating tools and resources to assist in this process. By the end of Q1 2025, we hope to be generating these reports on our own without the assistance of HR. Appendix A: Risk Ratings Observations identified during the audit are assigned a risk rating, as outlined in the table below. Only one of the criteria needs to be met for an observation to be rated High, Medium, or Low. Low rated observations will be evaluated and may or may not be reflected in the final report. Rating Financial/ Operational Impact Internal Controls Compliance Public Commission/ Management High Significant Missing or partial controls Non-compliance with Laws, Port Policies, Contracts High probability for external audit issues and / or negative public perception Requires immediate attention Medium Moderate Partial controls Not functioning effectively Partial compliance with Laws, Port Policies Contracts Moderate probability for external audit issues and / or negative public perception Requires attention Low Minimal Functioning as intended but could be enhanced Mostly complies with Laws, Port Policies, Contracts Low probability for external audit issues and/or negative public perception Does not require immediate attention Equity Policy Directive Compliance