Wednesday, February 9, 2011

A "Brauny" Analysis of Survey Ethics

I normally don't do phone surveys. They're tedious and tend to have pretty lame questions. Especially frustrating is when none of the answers really fit or the question doesn't provide adequate information to give a meaningful answer. In short, they're annoying.

At 5:55pm last night, my phone rang, displaying the number 609-279-0220. I didn't recognize it, but, for whatever reason, decided to answered anyway. A woman's voice greeted me, introducing herself and her organization: Braun Research, a company that conducts surveys on behalf of third parties. I found myself the lucky participant of a phone survey.

Splendid.

The woman from Braun, we'll call her "Brianne", gave a quick introduction of the survey. She was going to ask me some questions about recent events that have been in the news. "Okay," I thought. "This shouldn't be too bad." Without much of a pause, she launched into the barrage of questions.

It didn't really surprise me, overall. Most of the questions were simple opinion, agree/disagree types, but there were definitely some that were too vague to properly answer. Brianne was friendly and nice to talk with, but she couldn't provide anything beyond what the questions stated. To be expected, but I asked for clarification every now and then on the off chance that she might know a bit more. All in all, the questions focused on my opinion of President Barack Obama and whether I thought a list of Republican/Tea Party presidential hopefuls would do a good job. The revolution in Egypt also came up with such questions along the lines of "Do you think the violent protests in Egypt could happen here?" There was definitely a conservative slant to the questions.

I was also asked about my religious beliefs, how often I attended religious services and whether I thought Islam was a mostly peaceful religion or a violent religion bent on taking over the world. No loaded question there. The odd questions out were a handful about Valentine's Day.

Other than the noticeable bias in the questions and the weird V-day questions, not a whole lot surprising. And yet, the experience inspired me to write a post about the ethics involved in such a survey. At the end of the call, as Brianne was about to hang up, I decided to ask a couple questions. First, who was the sponsor of the study? Second, would the results be published anywhere? The answers were, respectively, "We can't tell you the sponsor because it might bias the results" and, from her manager, "I can't tell you that" (though I did hear something in the background about appearing in a/the "Gazette" when Brianne first asked about publication). These answers struck me as a bit, well, fishy.

The thing that struck me about this is that Braun Research was, potentially, violating human research protections. I'm not all that surprised that Brianne and her supervisor were unable to answer my questions, since most people probably don't think about them. But surveys such as this are considered research involving human subjects. The same rights that apply to people volunteering for, say, a drug study, also apply to surveys. There is a right to know certain information, the right to informed consent.

Now, before I delve into this, since I don't know who the sponsor is, I can't state for certain that this survey was subject to human subjects research regulations (45 CFR 46). The question revolves around whether or not a government agency has oversight of such a study, either due to regulatory responsibility (e.g., FDA's oversight of investigational drug studies) or provides funding for it. As stated at 45 CFR 46.102(e):

(e) Research subject to regulation, and similar terms are intended to encompass those research activities for which a federal department or agency has specific responsibility for regulating as a research activity, (for example, Investigational New Drug requirements administered by the Food and Drug Administration). It does not include research activities which are incidentally regulated by a federal department or agency solely as part of the department's or agency's broader responsibility to regulate certain types of activities whether research or non-research in nature (for example, Wage and Hour requirements administered by the Department of Labor).

Assuming that it is covered by these regulations, then Braun Research had a responsibility to provide me with certain information. For example, they did not disclose to me "(2) [a] description of any reasonably foreseeable risks or discomforts to the subject". This would include letting me know how my answers and demographic data were going to be used, who would have access to it and what steps would be taken to minimize the risk of disclosing information that could identify me. Although my name was never provided to them, I do not know if they were keeping my phone number. Furthermore, the demographic data they collected (ZIP Code, birth year, ethnicity and income range) could be used to identify me, were someone determined enough to do so. Closely related to those issues, I was never given "(5) [a] statement describing the extent, if any, to which confidentiality of records identifying the subject will be maintained" beyond assurances that my information would be confidential.

"(7) An explanation of whom to contact for answers to pertinent questions about the research and research subjects' rights, and whom to contact in the event of a research-related injury to the subject" was also lacking. This would have consisted of the name of the principal investigator and/or their study coordinator, as well as contact information for the Institutional Review Board (IRB) overseeing the research. Without this information, I have no means of ending my participation or requesting that data which has not already been analyzed or shared with others be removed. Nor can I contact the IRB/ethics committee if I have any complaints or feel pressured/coerced into participating.

Although not explicitly required by regulation, the informed consent process should provide me with any and all reasonable information that would affect my decision to participate. In addition to the above information, that would also include information on the funding source for the research. Granted, knowing the funding source/sponsor before starting the survey could bias my answers, when I asked, I had already completed the survey. At that point, the information should have been provided so I could decide whether or not I still wanted to let them use my responses.

It may seem like I'm being too nitpicky about this, but research ethics is a subject about which I care quite strongly. And I am not the only one. Braun Research is a member of several public opinion research organizations, such as the American Association for Public Opinion Research. They have a FAQ focusing specifically on IRB issues and human subjects protections, reiterating many of the points I highlighted above. Likewise, the Council on Marketing and Opinion Research (now part of Marketing Research Association) has a compliance guide covering applicable laws and regulations, as well as a "Respondent Bill of Rights". And the Council on American Survey Research Organizations has a "Code of Standards and Ethics" that mainly discusses privacy issues and states in the preamble (emphasis added):

Respondents should be:

a. willing participants in survey research;
b. appropriately informed about the survey's intentions and how their personal information and survey responses will be used and protected;
c. sufficiently satisfied with their survey experience;
d. willing to participate again in survey research.

It should be clear that professional research organizations take these issues seriously. As a member of these organizations, Braun Research should be complying with appropriate ethical standards and applicable regulations. As I said toward the beginning, it's unclear, based on the information I was given, whether the Federal regulations at 45 CFR 46 apply to the survey in which I participated. Yet even if they did not apply, Braun should have followed industry standards of proper ethical behavior in their research by informing me (and all other participants) of the risks, confidentiality measures, contact info and how the data were going to be used (e.g., where it would be published).

If you get one of these calls, know your rights. If the researchers don't tell you what you are entitled to know, then let them know which regulations they're violating and let them know that you are going to report them to the appropriate authorities.

3 comments:

  1. I noticed a prior poisoning of the well in several surveys in recent years(usually prior to political campaigns), so I instituted a policy that the moment I got a biased question, or a stupid one I would end the conversation. I do warn the person of my policy.

    This started with a survey where the first question was: Do you feel the youth of today are in trouble?

    BZZZZt... leading question without context!

    I don't seem to be getting through many surveys lately. Well, except for the ones run by the city. They declare upfront that the survey company was hired by the city to figure out future policies (like would I be willing to have garbage pickup every two weeks alternating with recycling, with yard/food waste every week).

    Funny story about surveys: I live across a highway from an insular upper class neighborhood. They are uptight about everything, and seem to like to have things done just their way. A company that had a research campus within their boundaries was trying to expand, but hit resistance from the neighborhood association. They decided that if they could not modify their campus, they would just sell it to a developer. Uh, oh... NIMBY alert!. So either the neighborhood association or the company commissioned a phone survey, and not realizing that they are only about a tenth of their zip code I got to answer the questions. I often responded with "Sure, let them build, it is not my neighborhood!". I had fun with that survey, which was weird in that it asked if I liked/disliked certain famous people living there, including a former governor/US senator. I read in the paper his wife got called for that survey too.

    ReplyDelete
  2. It's kind of tricky when it comes to surveys. (By the way, "student," we'll cover this in a future night school epidemiology "class." Basically, if there is a reasonable expectation that you will be in any way affected by the research, they have to disclose everything to you, even the kitchen sink. An example of this was AoA telephone survey because it can be reasonably assumed that leading questions about vaccine "injuries" may make someone not get a vaccine. You can see where that could cause trouble.
    On the other hand, asking for your preference in yogurt should not reasonably affect you in any way. Know what I mean?
    Now, if they are receiving any kind of federal money (and money from states in many states), they also need to disclose it all to you, within reason. If they're an academic institution, they also need to disclose it all to you, unless an IRB agrees that disclosing stuff to you will have a negative effect on you or the general welfare that could come from the results.
    I'll cover this more in detail in the "night school," but I can see where not wanting to disclose something after you've given your answers is "fishy."

    ReplyDelete
  3. @Ren

    Yep, as I said, not knowing where the money's coming from and whether or not any Federal agency covered by the Common Rule has jurisdiction, hard to say if the regs apply. But, assuming that they are covered, because the questions involved opinions on political matters which could reasonably affect one's standing within their community and/or employment, and because they collected demographical information that could identify the individual respondent, they should have provided information, especially after all of the answers were given and there was no chance of biasing the results. I reviewed the waiver regulations, and it didn't look like they would have been able to get one from their IRB (assuming an IRB even looked at the thing).

    ReplyDelete

Spam comments will be deleted.

Due to spammers and my lack of time, comments will be closed until further notice.

Note: Only a member of this blog may post a comment.