Promoting a Culture of Change through Data
Dr. Elizabeth (Elise) Van Winkle, Executive Director, Office of Force Resiliency
Dr. Elizabeth (Elise) Van Winkle will discuss the role of data and how an in-depth understanding can promote cultural change. She will address the importance of transparency of information; instances where the knowledge gained from surveys, focus groups, and reports have shaped the military’s understanding of these problems; and ways that climate assessments can help decision-makers better prevent sexual assault and sexual harassment.
Transcript
I have the pleasure of introducing Dr. Elise Van Winkle, who currently serves as the Executive Director of the Office of Force Resiliency for the Undersecretary of Defense for Personnel and Readiness. In this role, Dr. Van Winkle serves as the Advisor to the Undersecretary of Defense, as well as to the Secretary of Defense. She has a huge portfolio, and this is a subject matter expert that is highly esteemed that you’ll want to prepare your questions for. She works on integrating activities in the areas of sexual assault prevention and response, suicide prevention, harassment, including hazing and bullying, diversity management, equal opportunity, drug demand reduction, total force fitness, and the department’s collaborative efforts with Department of Veteran’s Affairs. Will you help me welcome Dr. Elise Van Winkle. (audience applause)
Good morning, and it’s truly a pleasure to be asked to speak today by the secretary and the chancellor. Today, we’re going to talk a little bit about data. We’ve started the conversation. I’m going to talk a little bit more about how the department uses data. I want to start by acknowledging that this has been a pretty tough year for the department. It’s been a tough year for me getting the data back on our sexual assault surveys, certainly not going in the direction that we had hoped. I do want to call out the superintendents, as well as General Lecce, who’s here. He, can you stand up, General Lecce? I know, I know. He is one of the co-leads with me on the Sexual Assault Accountability and Investigation Taskforce, which Senator McSally mentioned, we started up. The reason I’m calling them out, the superintendents, as well as Secretary Spencer, is that with such a hard year that we’ve had it’s difficult to find strength to continue doing what you’re doing. And I will say that these gentlemen have been so resolved in this issue that it has given voice to people who don’t have voice, and it has given strength to people who feel defeated, and I would put myself in the category of that. Admiral Buck is new to Naval Academy, but I’ve been working with him since 2012. It is just an honor to work with them. Their dedication is amazing. So if we could have a round of applause for them, it’s warranted. (audience applause) Before I get into my slides I want to talk a little bit off script and level set. I started my work in the field as a clinician for about 10 years before moving into the Department of Defense where I’ve been for about 10 years. This is really difficult work we’re doing. It’s really difficult. We all say the right things. I say the department is committed to combating sexual assault and harassment and I mean it. Everybody here today has shown that level of commitment. I’ve said this here many times, but I’m disheartened by the data that we’ve seen. And I am. I’m devastated by the data that we’ve seen. Between 2006 when we started measuring sexual assault and harassment we went from 34, we estimated 34,000 active duty service members experienced a sexual assault. In 2016, when I rolled out the report we were under 15,000. We cut that number in half. The number of military members coming forward to report went from 7% to 32% because of our efforts of the response system side. In 2018, we saw an uptick, an increase. It’s devastating. Those numbers are devastating. Whether it’s the 15,000, whether it’s the 34,000, whether it’s the 737 academy students. These are individuals. And most of you in the room know how devastating it is for these individuals. So this is hard work. And I really appreciate everybody taking the time to come here today to start to get at this, because that weight of disappointment as you start to collect data is real. But it is necessary. The data that we have is invaluable to understanding the scope of the issue. I’m going to talk a little bit about how all of the policies that fall under me, under all of my portfolios are held to a standard on data. When we collect data it is critical, but it is very intimidating. When we collect data on the number of sexual assaults that are happening on our campuses, at our academies, in our active duty, in our reserve force, in our National Guard force, it is a liability. We are trying to protect our institutions. Each of you in colleges and universities rely on your reputation. So what happens when you put that number out there? And I’m going to be very honest. No one wants to be known as the rape campus. No service wants to be known as the rape service. No academy wants to be known as the rape academy. And often we lose control over the message once it gets out into the public. It’s very difficult. At the same time, we are all sitting here today, because we are dedicated to this issue. And we are dedicated to eliminating these crimes on our campuses and our force. We can’t do that if you don’t know what’s happening. So what I’m going to talk to you about today is how the department has started to use data over time and how we continue to use data to inform our process. It is hard work. But when we go after the data, that’s how we make a difference. Clicker is not working, Joe. There we go. I’m terrible with clickers. I’m very good at data and policy, clickers is not my strong suit. I want to start with a quote. This quote is a good summary of what I want to talk about today. You’ll see in the footnote the citation, but I have to honest. I had not seen this quote until I was reading Jackson Katz’s book, “The Macho Paradox”, which is a fantastic book about the role that men play in this conversation. The quote says, “Not everything that is faced “can be changed, but nothing can be changed “until it is faced.” You’ll hear me say today, and I often say we cannot fix what we don’t understand. James Baldwin said it a different way, but similar. “If we expect to make a difference “we must be willing to face the harsh reality “of what is happening and how bad the situation is.” This is true of all the challenging issues that fall to me. And before I talk about how I approach these policies, I’ll first discuss what issues do fall to me. I’m going to try to. There we go. So I oversee the secretary’s programs and policies on sexual assault, harassment, equal opportunity, diversity and inclusion, drug misuse, and suicide prevention. Many of these, as you know, overlap and intersect. And there’s many policies that fall under each of these portfolios. Every part of my portfolio can only be understood through the data that we collect. That’s because each of these issues have a very unique data challenge. And we must overcome that data challenge if we’re going to make a difference in the space. That challenge is that people don’t want to talk about it. Nobody wants to tell you that they were raped by their best friend when they went out drinking one night. They don’t want people to know that they’re being sexually harassed in front of their friends who are laughing. They don’t want to talk about how maybe, just maybe, they didn’t get that job or opportunity because they’re disabled, or because of the color of their skin. They don’t want us to know that they’ve been getting really good grades with the help of amphetamines, or maybe cocaine. And they don’t want to tell us that at some point they, alone, outside of the buzz of the world, thought that maybe they’d be better of dead. People don’t want to talk about these things. And yet, if we’re going to make a difference, we need to know what’s happening. But I don’t want to force anyone to talk about these things. So this is a unique challenge that we all share. I need to know what’s happening if I’m going to make a difference. You need to know what’s happening if you are going to make a difference. Because of the criticality of good data in my portfolio, the policies that fall under me are held to a very high standard of data and analysis. I refer to this sometimes as my policy loop. It starts with an assessment. What is the scope of the issue? For sexual assault we look at two primary metrics. How many people are experiencing sexual assault? We also look at how many people come forward to report sexual assault to us, which is a very good indication of how our response system is working. Your assessment process allows you to understand if you have a problem and how bad that problem is. Next, there’s the analysis. Where are targeted areas of concern? Does this issue affect your whole population, or does it affect sub-populations? Your analysis lets you understand where your problem is most prevalent so you can adjust your policies and efforts accordingly. Then there’s collaboration. This is often overlooked. If you’re trying to solve a global problem, like violence and prevention of violence, you can’t do it alone. For the department, our collaboration is with the services, other DOD components. We rely on subject matter experts, academia, research, many of whom are here today. The collaborations are really important so that we can make sure our policies and our guidelines are the most up-to-date with the information that we have. And then it goes into the policy. These things will then help you develop your policy and modify your policies. I’m going to talk a little bit about each of these elements. So again, I’ll start of by saying what I said before. You can’t fix what you don’t understand. The assessment element is, again, how we understand the scope of the issue. There’s a number of obstacles, many of which that I’ve already mentioned about getting good data in this space. There’s a gap between what we know and what we don’t know. The experiences that we’re talking about bring out amazingly powerful emotions of embarrassment, shame, vulnerability. These are huge emotions that are very difficult for us to get through and it keeps people from coming forward to talk about it to get help. But when people don’t come forward it’s harder for us to hold offenders appropriately accountable and to try to change things. We use surveys. You’ve heard us talk about this today. The surveys that we do are large scale scientific surveys. The active force, for example, is 1.3 active duty military members. We go out to over 700,000 military members. At the academies, we do an in-person survey. We go out, we don’t force them to take the survey as we discussed, but we do pitch why it’s important to take the survey. One of the things I often say to them is your policies are going to change based on what we learn on these surveys. You can stand up and you can walk out. You can lie on the survey. The policies will still change. We need to know what’s happening so we can adjust them to help serve you better. We get very high response rates in doing that at the academies, between 70 and 80%. And then in the active force we do these large scale, mostly web-based, some paper-based, surveys, and you may hear about them a little bit later in the day, where we do use sampling techniques and weighting techniques. The weighting allows us to take the proportion of people who took the survey and generalize it to the full force. It is the industry standard for understanding what’s happening in your population. So we rely a lot on our surveys. But as I mentioned, these are difficult issues to talk about. So we need to be cautious about how often we ask people to take these surveys. We can’t ask them over and over. We’re asking a great deal of them to go through these very personal accounts of what happened to them. We do it every two years in order for us to get a good snapshot of what’s happening, but we wouldn’t want to do it any more frequently than that. Without this data, and without this assessment, we’re limited by what we see versus what is happening. We not only use surveys, but we also use the number of reports we get. We do focus groups. We talked about every other year we do a survey. In the off-year we go out and do focus groups with the active force, National Guard, reserve. We also do it at the academies to hear firsthand what’s going on. What are we missing? What aren’t we asking about? We also have command climate surveys, which are more at the unit level. Those are not weighted up to the full population, but give commanders an understanding of what’s happening in their ranks. So I want to play this out a little bit to illustrate the importance. Using sexual assault and sexual harassment as an example, I’m going to start with the easiest data source that you have available to you, which is the number of reports that you get. The data I’m using here is from 2018. On the left, or your right, these are the number of active duty members who reported a sexual assault in 2018. So we have about 6000 reported to us a sexual assault. A little less than 1500 filed a sexual harassment complaint. Now, I should pause for a minute to say not only does data influence your policies, but your policies influence how you look at your data. For sexual harassment, our policy encourages folks to handle it at the lowest level if they’re experiencing sexual harassment or some of those behaviors. And in our surveys we know, and in our focus groups we know that many folks say I experienced this. I talked to the person directly and I resolved it that way. Per policy, that is what we do encourage, those types of conversations. So it is important as you start to look at the number of formal complaints. But if we go to the next slide we’re going to see how often these behaviors actually occur. So we estimate by our survey that while 6000 active duty members reported a sexual assault over 20,000 active duty members experienced a sexual assault in 2018. Sexual harassment, while about 1500 filed a complaint, we estimate over 116,000 experienced sexual harassment in 2018 in our active force. If you are not surveying your population you are not getting the full picture of what is happening and you can’t fix it. One of the things I will note here that’s an added challenge is we do a lot in our response system to encourage people to come forward and report. So another problem is if I’m only looking at reports as my metric for progress, and that goes up, what is that telling me? Did the crime go up? Or are more people coming forward because they trust our system? We went from 7% of our force coming forward to report in 2006 to 32% in 2016. That remained relatively stable in 2018. Nearly one in three people came forward to report. In our minds that’s good. But it’s only when we look at the prevalence that we can understand that. So I now want to talk a little bit about the analysis. Oops, I went too far. There we go. Once you have your primary metrics, so again, we were talking about sexual assault, and we’re looking at who reported, and how many experienced it, you want to dive into that data to understand where the areas of greatest concern are. We talked a little bit about this with the superintendents as they talked about what they learned from their data. What are your areas of greatest risk? Is your top line number masking certain impacted sub-populations? We also want to know if our efforts have somehow shifted the problem inadvertently in ways that we didn’t anticipate. We look for where our efforts may have had the greatest impact and we look for where our efforts have had little impact. Analysis can take a number of forms. We use simple regressions. We also use OSS ratios to get a sense of risk. We also use more robust predictive analysis. We do qualitative analysis of our focus groups and structured interviews, and various methods to visualize the data, including most recently we started using heat mapping to get a sense of where the problem is. So I want to take an example so that we can talk a little bit more about why this analysis part is important. So this is just an example. This is hypothetical data. Let’s say we have a commander who has a unit with a hundred service members in it. 80% male, 20% female. Using one of our command climate surveys, let’s say the commander finds out that she has about 6% of her unit experience sexual harassment. She sees this as about six people out of a hundred. So she may be continuing doing what she’s doing. Six out of a hundred certainly, we want it to be zero. The question is whether she has all the information to inform her decision. Does she know her targeted areas of concern? I should, again, say this is a hypothetical breakdown of the hypothetical situation. But the data you see here is not unlike what we actually see. When we look at the data just broken out by gender, a different picture emerges. Instead of six of her people experiencing sexual harassment, or six out of a hundred, she’s actually got one out of four of her women experiencing sexual harassment. So this is a much bigger issue, particularly as we know from our data that sexual harassment is a leading indicator for sexual assault. Without the targeted data she may not have all the information to understand what’s happening. You need to know how are your women experiencing things? How are your men experiencing things? How are your minority students experiencing things? How are your students who are disabled experiencing things? You need to know how all of your folks are experiencing things and understanding how those sub-populations are working within your policies and your rules and your guidelines. We need to get this information so we understand how to shift our efforts. One of the things we learned in the department when we first started a lot of our problems were at the leadership level. Leadership didn’t want to talk about it. We did not have these kinds of conversations. And so we really worked with leaders to get the message out, and they did. What we learned from our most recent data, as was mentioned by the superintendents, is that we saw the problem now really concentrated within the 17 to 24-year olds. We heard from them that they hear from their leaders. But the peers are not picking up that same message. We also hear things like, “Well, I care more about what my peers think about me “than what my leaders think about me.” This is new. This is new information for us. What worked to drive our sexual assault rates down by half is no longer working for this group. If we didn’t have our survey, if we didn’t look at the deep dive into the analysis, we wouldn’t know how we need to shift things. So once you have the data you need to talk to people. Start by thinking about who influences your policies and who is impacted by your policies. For sexual assault you need to be talking to survivors, those people who have forward to report. Talk to them. What were some of the things that drove them to report? What were some of the things they were concerned about? You also need to be talking to your population as a whole. They’re going to be held to the standard that you set in your policies and your rules. So you need to talk to them. What you think may be the right thing may not be the right thing for your population. And I would say one of the examples that I use sometimes, and we talked a little bit about it, is when we first started this a long time ago we felt like when you report a sexual assault it should automatically go to an investigation. This is our way, as a military, to hold people appropriately accountable for these crimes. That was what we thought. What we heard is that people wanted the opportunity to get help without initiating an investigation. So we developed the restricted report option, which you heard a little bit about. This allows people to come in and get medical care, mental health care, get the restorative care they need. This was not something we thought was important when we first started. It was only after talking with people did we know that this was an option that was important. What we found, it was extremely beneficial. We know from our survey data when we ask those people who made a restricted report, what would you have done if you didn’t have this option, the majority say I wouldn’t have reported at all. Now, combine that with another data point we have. 25% of our folks who made a restricted report later convert that to an unrestricted report and go through with the investigation. Once they receive the care that they need, they’re strengthened and they may be willing to come forward and give us the name so we can hold people appropriately accountable. So that’s just an example of what we can glean and how we need to talk to people so we can understand what types approaches you need. When we’re dealing with an issue like this, no one has solved it. So we’re all in this together. Talking with subject matter experts, talking with those folks who work with your populations. Military looks different from colleges and universities, that look different from the general American population. Have opportunities for discussions. For anybody who’s worked in the DOD, who currently works in the DOD, we love our working groups and task forces. But they’re great opportunities for us to get together and have these conversations with our stakeholders. It’s also important to have large scale conferences and discussions just like this one. This is how we get our ideas out. This is how we can be a little vulnerable and say we tried this, I don’t think it’s working. Has anybody else tried this? We’re dealing with something no one has solved. So these conversations become extremely valuable and extremely important. All of this leads to the development of your policies. So when we talk about policies, policies, rules, guidelines. Without a policy that is specifically addressing your issue, you’re sending the wrong message. If you say you don’t tolerate something, but you have no policy or directive that governs this, you’re sending two different messages. In the military, we saw this. We had policies on harassment. And then we learned that there was a huge problem on social media. A lot of policies didn’t cover social media. We changed our policies. Specific to my portfolio, and any portfolio aimed at violence prevention, make sure that your policies have aspects of prevention, response, support care, and also data and metrics. Make sure that you build that into your policies in terms of trying to address these problems. And then get your message out there. You want to be having these conversations. One of the things that I think is wonderful about these conversations is we’re saying it out loud. We’re talking about sexual assault. That hasn’t always been the case. So it just adds to the charge of the issue. So we need to have those conversations. Town halls, small groups, put it on social media. Send out memos and directives talking about why you want this to change and what is important to you. Talk to your force, whatever that may look like. Once your policies are developed, you’re not done. It’s a constant cycle. Develop your policy and assess yourself again. Analyze the data that you get. Collaborate again. Here’s what we’ve learned. The department has these surveys every two years. We can come out here every two years, and we’ll give you a new snapshot of what we’re dealing with and it helps the conversation. We are all in this together. I truly appreciate everybody coming out today to talk about this. As I said in the beginning, this is really difficult work. When I’m at forums like this I truly have hope that we are going to make a difference in this space. Just seeing the dedication and hearing from people, you really are making a difference just by being here today. I want to thank you again for the opportunity to talk, and I do want to provide some time for any questions and answers. I also may rely a little bit, as we get into the policies, on Dr. Nate Galbreath, who’s sitting over there, who is currently the Director of our Sexual Assault Prevention Response Office and oversees our sexual assault policies. So I would open it up for questions to either myself or Nate. Hello.
Thank you, again, for a wonderful presentation. So I’m sure you would have imagined, based on our brief conversation, I might have questions. So you talked a bit about how when developing your policy and procedures and analyzing your data you look at and question whether or not the data are masking certain sub-populations. And you also mentioned that you facilitate focus groups. So I’m curious to know if when you facilitate those focus groups do you create a space that targets some of those masked populations? For example, that would allow people with disabilities to talk about their experiences, or folks who would be identified as underrepresented minorities to talk about their experiences so you get some of that qualitative data?
Absolutely. So our analysis really does drive our research agenda. So as an example, we went out, or we’re going to be going out to do focus groups in the active force based on the data we learned. We found out a great deal about the 17 to 24-year old group that was really driving the rates of sexual assault, perpetration and victimization. So we’ve set up the focus groups so we are targeting 17 to 24-year olds to hear directly from them, instead of having just a random assortment of people from any certain installation. So it does absolutely drive that as best as we can while maintaining the privacy, understanding some sub-populations depending on where we’re at get very small numbers. We do need to report out our focus group findings, so we do want to protect the privacy and anonymity of folks. But absolutely.
[Questioner] Okay, thank you.
[Susan] Hi, how are you?
Hi.
Thank you so much for your presentation. My name is Susan Perkins, and I’m with SUNY’s University Center for Academic and Workforce Development. When you were talking about focus groups, and earlier we heard from the panel this theme of creating a culture of intervention where people are encouraged to talk about difficult issues, have those challenging conversations, I was wondering if you can talk a bit more about some of the best practices that you would encourage around creating what some of us at SUNY would refer to as courageous spaces, having a climate where people can share and feel comfortable sharing, so they can really get at some deep-seeded issues that they can work together and focus on discovery and healing and collegiality and respect. Is that?
Yeah, I think so. I can talk to you about the military space mostly. That’s where my focus is. Within the military space I would say it absolutely starts with leadership. As we started to have folks at the highest levels talk about it, as you hear leaders that you respect talk about it, talk about suicide, talk about sexual assault, Admiral Buck up here was talking about male on male sexual assault. That’s not something that everybody has always wanted to talk about. The minute you see a leader that you respect talking about these things automatically that door is opened just a little bit of maybe this is a safe space I can start talking. But what we’re learning now is it’s not enough for just leaders. You need to have the peers to provide that. That’s part of having those conversations, whether it’s town halls about this. The focus groups are great opportunities for us to get folks kind of in small groups, separated we do men and we do women, we do combined groups to try to give them that space to talk a little bit. When we go to the academies we talk to the freshmen and sophomores separate from the juniors and seniors so that we can give them a little bit more of an ability to speak openly. Some of that is based on feedback from focus groups about what would work best to get them to feel most comfortable talking to us. We also have things like survivor summits where we bring folks together to talk about it. We talk more about the recovery side of it. So again, sometimes just having these conversations is the best starting point for this, and having folks who wouldn’t normally talk about it talk about it.
[Susan] Okay, great. Thank you.
Hi, I’m from Baruch College. I think we are all very intrigued by these focus group and I’m just wondering do you have any sort of care for the people who are in the focus groups? So they’re revealing, disclosing, getting traumatized and what sort of steps do you have in place?
That’s a fantastic question. A couple things. Our focus groups are scientifically done. We go through the IRB process. We go through all those processes in order to try to protect human subjects. Most of you in the college and universities are used to that. We do go through that process. We go through a number of coordination steps to make sure that we’re not inadvertently asking the wrong question. Of course, we do all of that, and we may still be inadvertently traumatizing somebody in the room and we didn’t anticipate it. What we do, is while only the researchers are in the room conducting a focus groups, and I’ll use the academy as an example, no one from the academy, no academy staff is in the focus groups, no academy staff is in the room when we’re surveying folks either. It’s only the researchers that are conducting these focus groups and the surveys. But we do have right outside, we have some of the sexual assault response coordinators, or victim advocates, who can be there in case there is a situation where somebody is traumatized or is impacted, so they have somebody to talk to. So we have them available to them. Any other questions? I really, again, want to thank you all for taking the time to come here today. It is energizing to be in this room to say the least. And I really appreciate all the amazing work that each of you are doing. And I hope that you have a very fruitful day in terms of your conversations on this. Thank you. (audience applause)
Can we have one more round of applause, Dr. Elise Van Winkle. (audience applause)