What’s the issue with online data?

Hi All,

Having just read the Orgad piece on online/offline research, I understand and agree with most of what she says about deconstructing the distinction between online and offline research.  What still confuses me a bit is what she’s arguing against.  What reasoning supports maintaining an online/offline distinction in research?

Early in the article she points out that looking at the Internet as either a cultural artifact or a culture informs our choices about whether to collect information in an online or offline setting, and seems to be saying that this is the reason people put online data in a different category.  But rather than supporting a distinction, this seems to me to just confirm her point, that everything depends on what kind of information your research is trying to get at.

Towards the end, Orgad provides the quote by Pitts, who talks about the lack of offline data in her research as a limitation.  Pitts writes that she can’t confirm the identities of her subjects, and that got me thinking that maybe that’s the real reason researchers might feel that online research isn’t as reliable as offline, whether consciously or intuitively.  Is identity the whole issue?  Is there more to it than that?

Again, I definitely agree with Orgad, but I’m trying to think about this from different angles.


Reflections on Peer Review

Hi Guys,

How’s the peer review going? Any interesting thoughts about the review process? Has it taught you anything that may be useful for assignment 4 or other research?

I just finished my review. I did one of the survey articles:  Privacy and Modern Advertising: Most US Internet Users Want “Do Not Track” to Stop Collection of Data About Their Online Activities. I don’t plan on using surveys for assignment 4, but I thought this sounded like the most interesting article.

One of the things I learned while doing the review was how important explaining your assumptions and technique is in writing the paper. Many of my critiques with the paper was stuff that I think was based on my confusion because I didn’t understand why they decided to use the methods they used. It harks back to stuff that both Luker and Knight talked about in the first or second weeks readings about the importance of writing throughout the whole research process. Otherwise, it`s easy to forget to include those things in your final write-up. Reviewing this paper helped drive home that point for me so it`s definitely something I need to remember while writing up assignment 4.


Ethnography: A Checklist

Ethnography, according to Shaffir, is the a descriptive science that emphasizes the observation and description of a group’s behaviour in order to understand their culture. But Luker argues that it should be the practices of the group being studied rather than the culture, which makes for the salsa-dancing methods. Based on Luker observation is done in order to build theory. According to Luker ethnography research should be about generating theory rather than testing them, as most researchers do.

Ethnography is good for observing the subjects in his or her natural environment, which leads to data being collected under different conditions than if the subject was taken out of his or her home environment. It perceives the normal habits in a familiar environment to the subject. The researcher is able to observe not just the subject’s behaviour but also the context or surrounding of his or her environment. For example if a researcher was interested in cookbooks they would go to the subjects home and observe how the cookbooks are placed and what kinds the subject owns.

The problems with ethnography pertain to how difficult it can be to gain the data and as Shaffir states researchers normally omit some of his or her methods used for gathering data when writing reports on how he or she conducted the research. I found this to be interesting because of how it was discussed in class and through the readings of the importance of writing how you conducted the research project so that other researchers may observe what was done and apply the methods to future research. There is no simple way of conducting ethnography because it depends on the environment and the research has to adapt his or her research model based on the problem and setting at hand.

I found Stebbins article be helpful in what one should and should not do in regards to ethnography. I think these are really good tips for anyone who plans on conducting ethnography as part of his or her research methods. According to Stebbins one must first gain the trust of the group being studied and this must be done early on in order to prevent the subjects from feeling threatened by the researcher’s presence. Researchers often experience problems in gaining entry to the group but are able to maintain his or her scientifically distance. A researcher should not act like an expert but should have some knowledge that is sufficient. The researcher should be genuinely interested in the group’s way of life so it should not be faked. Stebbins stated that the subjects are willing to voluntary teach the researcher about his or her way of life as long as it does not reach unpleasantries. A research can ask about information already known to him or he in order to maintain an impression among the subjects. A research should participate in the group’s affairs but should refrain from going native- joining the group. This will allow the research to learn something valuable by participating but it should be limited due to dangers from getting overly involved and abandoning the research project. A research can fulfill the role of a helper but the problem is it being time consuming as the research project is limited. A researcher will learn more about the settings being observed by being socialized into it, which allows for them to provide different kinds of help based on the knowledge gained but as Stebbins notes this can sometimes be less technical and more social in nature.

Thinking about ethnography and ethics

I had one more thought on ethnography, before we move on…
When reading the Shaffir and Stebbins articles I kept thinking back to this mini-saga I had with the Lyndon LaRouche Youth Movement back when I was an undergraduate.  I was taking a feature writing class, and I thought it would be a great idea to write a profile of a member of the LYM, as it’s called.  I’m assuming students in Toronto will be familiar with them — they seem to be widespread, and spend a lot of time on campuses, ostensibly to recruit new members.  To be clear, I went into it knowing that the LYM is a badly misguided and even potentially dangerous political cult, but all the weirdness made me all the more curious about what it was like on the inside.

I found them on campus, started chatting casually, and before too long told them that I wanted to write something about them, and could we arrange to talk some other time.  There was one woman who seemed most willing to talk, she gave me her email address and I followed up.  Long story short, there were several phone conversations and a couple in person meetings with campaigners, and then the local leadership got wind of what I was doing and clamped down hard.  Very soon everyone knew my name and knew that I was trying to do, and they were forbidden to cooperate in any way.  The reason given was that it was a waste of time to talk about or do anything that didn’t further the cause of fighting fascism in the U.S. (their understanding of politics, history, and ideology is completely addled, but I don’t have time to go into it).

I had my ‘aha’ moment during one of the last face-to-face encounters I had with them.  I had asked if they saw any value in simplifying the content of the literature they distribute, to make it more accessible — even to someone like me who loves reading about economics and politics, it’s very hard to follow.  There was a particular look of glee in the face of one woman as she explained “we don’t want it to be easy to understand!”  At that point I realized that the cohesion of this group depends on feeling that they are outcasts, that the rest of the world cannot understand them.  I never liked it when someone referred to the LYM people as crazy or stupid, because from talking to them I could see that they were neither.  What I think is that they’re very lonely, and when you’re intelligent and lonely and have a leftist outlook and George W. Bush is the president, a tight-knit group of similar individuals who say they’re working to change the world can seem very attractive.  But once you start spending every waking moment with them (they all live together), it can seem like the only place you belong.

So from that standpoint, it seems to me that being genuinely sympathetic, respectful, and interested in what they had to say was just about the most threatening thing I could have done.  Thinking back, there were a few moments when I started arguing with them about something, and that seemed to be when they opened up the most.

I still think about writing that story someday, but after my experiences I seriously doubt anyone could do it without concealing their intentions.  You could talk to people who had left the group, and you can find those interviews online, but obviously that’s not a representative sample and it doesn’t get at the meat of what it’s really like to be on the inside.

As an undergraduate, they taught me journalism ethics, which basically said deception is almost always inappropriate, but that there are situations where the story is important and there’s no other way to get it, and those have to be considered on a case-by-case basis.  What interests me about ethnography is it seems like a very similar process in some ways to journalism, just on a much larger scale.  I didn’t notice the authors we read last week taking a stand one way or the other on outright, conscious deception in ethnography, and I wonder if that’s because it’s taken as given that it’s unethical and should never, ever happen?  It’s clear that there is a large grey area, but I wonder where the ethical boundary lines are.

Participant Observation and Researcher Identity

Something that stood out for me when reading Robert Stebbins article about participant observation was his brief discussion of the impact of ethnographic research on the identity of the researcher, specifically as a participant observer. He discusses participant observation as a humbling experience in which the researcher must admit a degree of ignorance and give up their role as a scholar to essentially become the student again. He says that in order to be successful in this research method the researcher needs to step out of their role as a professional and adopt a new identity.

William Shaffir also says that ethnography requires some degree of role playing or acting, that an important aspect of conducting this research is learning how to present a particular image of yourself in order to better fit in and achieve greater acceptance and therefore better results. In relation to participant observation Luker also mentions the effects of immersion in a different culture in terms of being off-kilter and living in an altered state of consciousness. She talks about the researcher having a new awareness of themselves as an observer.

I find it interesting to consider the effects of conducting ethnographic research on the researchers themselves. When conducting participant observation the researcher is at once a member and a non-member of the group they are studying. This weeks readings also talked about the risks of being too involved with the community they are studying. For instance overlooking important dynamics and taking things for granted. It seems like the researcher has to always be very aware of themselves and constantly framing and reframing their identity. They have to balance belonging to the community with remaining external enough to maintain the required objectivity. It makes sense that if the identity of the researcher has an influence on the research process, which it inevitably will, that the research itself will also affect the sense of identity of the researcher. It is interesting to consider how complex and probably daunting it is for researchers to establish themselves in the field.

Thoughts Concerning Calories and Peer Review

Since this week’s lecture I have been thinking a lot about the calorie study done that compared Starbucks stores in New York with Boston and Philadelphia. I found it interesting how by posting the calories in the products the companies revenues increased but it did not change what a person ordered for drinks. I think this study is interesting based on its relevancy to natural experiments because it shows the before and it will be tide into the the impact of future studies that focus on health concerns in the U.S. like obesity. Health conscious people will most likely change their eating habits based on the calories that are posted but not everyone is able to understand the calories in the way the medical professionals or nutritionists do.

In regards to peer review I think that it is an important part of the scholarly world that helps to make for a better and high quality works. I feel that anonymous peer reviewing is a more efficient way of conducting it as it allows for greater autonomy, which enables the reviewer to be critical in analysing the paper without being greatly biased by knowing the person’s identity. I have done peer review in my fourth year of undergrad but it was not anonymous as everyone in my fourth year history seminar class had to review everyone’s papers and discuss them in class. We all had to be critical in our analysis by stating what could be improved and what was good. I found this process to be intimidating because it was the first time I had to do this and after discussing peer review in this class I think my experience would have been better if it was anonymous.

Research and Legitimacy

I was thinking in class on Wednesday about something Knight mentioned near the beginning of Small-Scale Research, about the novelist and the researcher being more similar than we might think.

When I was an undergrad, I read a novel for a twentieth century britlit class called Regeneration, for which the author had obviously done a lot of research, and which took real people, in a real place, and imagined their interactions. One of the characters, a psychiatrist named William Rivers, used psychiatric jargon, worked at a hospital that treated people in ways historically employed during the First World War, and made reference to ethnographic studies that he, William Rivers, had actually carried out. Seigfreed Sassoon also popped up with drafts of his poems, which, as the author, Pat Barker, mentions in the acknowledgements, are taken unedited from Sassoon’s manuscripts.

The thing is, the author’s research methods are opaque. We can imagine that she looked at historic documents, read biographical materials, consulted experts (Pat Barker’s husband was a neurologist who was, according to wikipedia, familiar with William Rivers) and historians, and structured the novel from that research. From the thank yous in the acknowledgement section, that’s probably exactly what she did. But as a novel it’s also imagined and improvised. We’d treat a reading of Regeneration differently from an academic journal article on Sassoon or on Rivers. It doesn’t have the legitimizing strategies that scholarly articles employ: methods aren’t explained, there’s no esoteric jargon, no citations, no double-blind peer review. Possible biases or sources of error aren’t announced.

Reading Regeneration, I have no doubt the material events didn’t happen as the novel describes them. But reading most journal articles, I have the same impression. It’s not that the researchers’ subjectivity contaminates the results; that’s not what I’m getting at. Academic journal articles seem more “true” than fiction. They seem more “objective”, to give more pertinent, real information, than fiction might. And it has nothing to do with the validity of the information itself (Professor Cherry talked a little bit about the crazy stuff that does get published in academic journals), but the legitimizing conventions journals employ. They’re taken as more “true” because of the way they are structured and reviewed, the listing of methods, previous studies, circumvention or acknowledgement of sources of error or uncertainty etc, more so than the actual research.

What I think is making me cranky about journals is that I’m reading The Four-Gated City and in the back of my head, a voice is saying, “If you ever wanted to do research about about the surveillance of communist sympathizers, or about the unification and dissolution of communist-sympathizer groups, or something like that, you could never talk about The Four-Gated City, because it’s not ‘true’ in the sense a journal article about those things would be.” I mean, a person can only take so much empiricism.