from Frontlines, the SLDN Blog
A new poll of subscribers to the various Military Times publications (e.g., Army Times, Navy Times, etc.) may have enough selection bias and negatively biased questions toward gays serving in the military that it will likely provide support for those who do not want “Don’t Ask, Don’t Tell” (DADT) repealed.
The new poll, currently being conducted, follows on the heels of last year’s poll of Military Times subscribers which purportedly showed that a majority of active duty members didn’t want DADT repealed, and showed a small percentage of active duty members who said they would not reenlist if DADT was repealed. That previous poll suffered from extreme selection bias in that it did not survey a random sample of active duty military, but instead, it apparently sampled only subscribers to the Military Times publications.
Furthermore, the responses represented only those who chose to participate, creating yet another source of selection bias. Finally, there was no attempt to adjust the responses so that they more closely resembled the age/rank/service affiliation of active duty troops. Because Military Times subscribers tend to be more senior in age and rank, the responses reflected primarily the opinions of senior military members, and then only of those who chose to reply. As a result, only about 5 percent of the responses were from junior active duty troops (E4 and below), despite the fact that these members comprise a large percentage of the active duty workforce.
Nonetheless, the negative findings from that previous poll got great media attention, despite the Military Times disclaimer at the end of the poll admitting that it did not represent an accurate view of the active duty workforce.
The current poll is even worse, because it not only may suffer from the same selection biases contained in the previous poll, but worse, it contains questions that put a negative slant on gays serving in the military.
Furthermore, the demographic information contained in the poll asks not only the usual identifiers of age, rank, gender, service affiliation, etc., but shockingly, it asks people to identify their sexual orientation! To be sure, respondents have the option to “decline to answer,” but realistically, how many gay, lesbian or bisexual active duty troops will be unthinking enough to self-identity in a poll sent specifically to them by name, and which includes a request for their name, address and telephone number (so that they can be contacted if they win a sweepstakes award – an inducement to encourage participation in the poll)?!
I would be most surprised if the poll finds any active duty gay, lesbian or bisexual respondents, leading to the problem that some unscrupulous opponents of gays serving honorably in the military might argue that because there are so few GLB members on active duty, as reflected by this poll, why bother worrying about DADT at all?
Regarding the potential negative bias, the first question following the demographics asks respondents if they’ve ever been "hit on" by someone of the same gender, and follows that by asking what their reaction was. Notably, though, the poll does not ask respondents about being hit on by a member of the opposite sex. I imagine many female active duty personnel would be interested in that data, because that situation is all too common in the military today, yet it is ignored by the Military Times poll. We will thus have data only on presumably unwelcome gay/lesbian sexual advances, but there will be no comparative data on unwelcome straight sexual advances. This constitutes an obvious negative spin to gay military service, focusing only on apparent gay misconduct.
The poll then proceeds to ask respondents if their superiors ever knew about a gay service member but refused to do anything about that situation. You can easily see the negative bias in the way this question is worded as well. It and the previous question inferring gay/lesbian misconduct play to the fear that gays serving openly will misbehave and create disruption in the unit; and further, commands may not be doing anything about gays serving (presumably openly and thus illegally). And the questions appear first in the order of opinion responses sought from subscribers, thus potentially generating a negative connotation about gays in the military even before asking opinions about gays serving honestly and honorably.
Two academic institutions which have expertise in designing and interpreting polls agree that biased wording of some of the new poll’s questions is problematic. In an editorial titled “Scholars Question Methodology of New Poll on Gay Troops,” the Palm Center of the University of California, Santa Barbara, stated the following:
Researchers are particularly concerned about the potential for response bias, a phenomenon that can occur when questions are not worded or arranged neutrally. After a number of questions about the respondent’s age and rank, the Military Times survey poses questions about sexual advances by gay troops, and follows those by asking about attitudes toward “don’t ask, don’t tell.”
Professor Aaron Belkin, director of the Palm Center at the University of California, Santa Barbara, said that by prompting respondents to link sexual predation with the gays-in-the-military issue, the survey may generate misleading results. “If you set up a scenario about sexual predation and then ask someone how they feel about gays in the military, you can predict the response is not going to be positive.”
In addition, the Williams Institute of the University of California, Los Angeles, stated the following:
Dr. Gary Gates said that on hot-button emotional issues like gay rights, 'survey researchers must pay particular attention to framing their questions in neutral ways.' Gates is a co-author of a report just released in conjunction with the Williams Institute which addresses how the “potentially stigmatizing nature of some of the questions surrounding sexual orientation” can bias responses. Gates is Williams distinguished scholar at the Williams Institute at the UCLA School of Law, and the report is titled, 'Best Practices for Asking Questions about Sexual Orientation on Surveys.'
Finally, Dr. Nathaniel Frank, also of the Palm Center and author of the most comprehensive text ever published on gays in the military, Unfriendly Fire: How the Gay Ban Undermines the Military and Weakens America, stated:
"The relevant question is not what the troops want, but whether they are capable of serving with gays without becoming undisciplined. From a poll that does ask the right questions, we’ve learned that three quarters of troops are comfortable around gay people and that two thirds already know or suspect gays in their units," he said, referring to a 2006 Zogby poll of troops returning from Iraq and Afghanistan.
The new Military Times poll fails to ask the most relevant question of all, and that is: Are there known gays serving alongside their straight peers?
Because operational readiness is known to be excellent, and because our military is currently considered the finest, the most battle-ready and most combat experienced in the world, if known gays are serving now, in both theaters of war, where are all the assumed problems that formed the foundation for DADT?
Where are all the problems with unit morale, unit cohesion and combat readiness? The primary assumption underlying the DADT law was that any known gay in a unit would seriously discomfort heterosexuals in the unit and thus degrade operational readiness. One would think that a fair poll of active duty troop opinion would attempt to measure this critical factor, as other polls of military opinion have successfully done.
The 2006 Zogby poll specifically asked about known gays serving in a unit. The results showed that 23 percent of U.S. veterans of the Iraq/Afghanistan wars knew for certain there were gays in their own unit, and that the majority of them stated that this fact was widely known by others in their unit. Furthermore, another 45 percent of these Iraq/Afghanistan war veterans stated that they suspected there were gays in their own unit, and 73 percent of the troops said they were comfortable working with gays and lesbians.
Again, with tens of thousands of straight troops indicating they either know for certain or suspect there are gays in their own unit, where are all the problems? Because evidently the problems are few, if any, so how can anyone justify keeping DADT and discharging competent, trained and experienced gay and lesbian troops if they are not negatively impacting operational effectiveness?
The problem with the current Military Times poll, if its results indeed prove to suffer from selection bias and purport to demonstrate a negative view of gays serving in the military (based on questions that support negative views of gay service members), is its potential to influence decision makers in the Pentagon and in Congress. Hopefully these leaders will be aware of the problems with any poll conducted in an unscientific manner and the results of which therefore cannot be said to accurately reflect the opinion of the active duty workforce.
Ideally, the Pentagon should conduct its own research on this issue through statistically valid polls which contain only unbiased questions. I would love to see an updated version of the Zogby poll conducted in which the troops are again queried on their knowledge of serving with gay, lesbian or bisexual peers. If, as I suspect, there is an even higher percentage than in the previous poll of troops serving with known gay, lesbian or bisexual peers, and if indeed the Pentagon can verify that there are no demonstrable problems with unit morale, unit cohesion and combat readiness, despite the known presence of gay service members, then that should herald the end of DADT.
If the Pentagon won’t or can’t conduct such a poll, perhaps a group of interested independent agencies could jointly sponsor and fund a scientifically valid poll of the active duty troops. At the very least, it might provide a more realistic view of gays serving in the military than is likely to result from the current Military Times poll.