is362 : RSS Feed

Comparative Research: Don't we all do this to a certain extent?

Evren Eryilmaz — Apr 23, 2007 - 5:57pm

Among the research methods we have seen so far, I would say that historical comparative research is the most difficult one. From my point of view, moving from field research (hanging out with some exotic group of people) to historical comparative research (“achieves located in a dusty, out of the way room of a specialized library”) seems like a big change.

In this chapter, I liked comparative research and secondary sources (similar to Chapter 11). However, I am not sure whether I understood Neuman when he said “most positivist research is not comparative”. When we conduct a positivist study, to a certain extent, don’t we compare our study to other current formal studies in order to show that what we are doing is in tune with other researchers’ attempts.

This chapter also made me think about our previous review. I think it is possible to consider the “Complementary Use of Modeling Techniques: Insights from Representation Theory and Practice” as a comparative research to some extent. There are conflicting ideas on ontological foundations of conceptual modeling between Wand\Weber and Wyssusek. In this context, the authors of that paper compare these conflicting approaches and try to justify their study based on this comparison. Furthermore, the authors also compare the results of this study with a similar, recent study to provide further confidence for the conclusion.

Comment on "Neuman Chapter 13"

Jill — Apr 16, 2007 - 3:29pm

Neuman includes a quote in this chapter that "The price of fieldwork is very high, not in dollars, but in physical and mental effort" An interesting statement that clearly may separate the researchers willing to invest a very specific (and rigorous) amount of time and resources.

Comment on "Neuman Chapter 13"

Nimer — Apr 15, 2007 - 10:52am

Good point Evren. Although rare but not impossible. A book named "The Soul Of A New Machine" by Tracy Kidder may be interpreted as ethnography or a case study or both. The author virtually lived there with these guys who were developing a computer. Kidder reported on every small aspect and wrote the book as a narrative, which made for an interesting read. I also agree that this type of research is not for everybody. It requires more resources and skills to do.

Neuman Chapter 13

Evren Eryilmaz — Apr 13, 2007 - 5:52pm

Based on the papers we have read, I think conducting any study (case study, field study…) in a systematic, skeptical, and ethical study is both a form of science and art. Although there is no complicated statistics in the field study, a researcher who intends to use this method should have “a strong sense of self, an incredible ability to listen and absorb details, tremendous patience, sensitivity and empathy for others, superb social skills…” Therefore, I do not think that field study is proper for every researcher.

According to Neuman, “field research is valuable for micro-level or small-group face-to-face interaction and it is less effective when the concern is macro-level processes and social structures”. In this context, I wonder to what extent field study is applied in our discipline. For instance, is it possible to link ethnographic study to case study or perhaps to grounded theory when conducting IS research?

Comment on "Significant Digits and the amount of information in the data"

Kevin Williams — Apr 9, 2007 - 6:26pm

In comparing the difference between statistics and calculus, it seems that statistics starts with preciseness (significant digits) and calculus ends with it.

Comment on "Significant Digits and the amount of information in the data"

Lorie Obal — Apr 8, 2007 - 11:47pm

Yes, let's hear it for levels of significance. It doesn't look like engineering calculators get used much in social science research. I guess it's a good thing we're not designing and testing flight-critical systems here - ouch!

I'm glad you gave the example of carrying out the decimal point. The other issue is when people make the mistake of trying to claim that p <.000000001 is a more significant result!

Significant Digits and the amount of information in the data

Mitch Cochran — Apr 8, 2007 - 8:37pm

One of the points that appears in the chapter is that data contains a limited amount of information. You can summarize it but it is difficult to make it more accurate after the fact.

The concept of significant digits or figures states that you cannot add digits after the initial values. For example if you have a value of 3.5 you cannot arbitrarily make it 3.500. The value that you took is actually be between 3 and 4. It is closest to the middle but how close? The researcher may not know if it is 3.6 or 3.4. So looking at the raw data it would be unreasonable to assume that if you saw a value of 3.5 to assume that it is 3.500000000. You need to need the precision of the experiment and how many significant digits there are.

We can tie this concept back to the chapter by looking at the amount of information in the different scales. An ordinal scale will not have as much information in it as an interval scale. There is a hierarchy in that you can go downward from an interval scale to an ordinal scale but not vice versa.

COMMON STATISTICAL ERRORS

Evren Eryilmaz — Apr 7, 2007 - 4:22pm

The article below points out common forms of statistical errors in the biomedical literature. I wanted to talked about this discipline rather than IS because everything I read about dietary or healthy food does not sound credible. For multivariate analyses, these are:

1. Not confirming that the data met the assumptions of ANOVA.

ANOVA assumes that the response variable is approximately normally distributed within each level of the explanatory variable and that the variability of these distributions is approximately the same. If data are not data are not normally distributed, than the data need to be mathematically transformed into distributions that are more normally distributed.In a way, this error also sound similar to a researcher using secondary data or existing statistics that are inappropriate for his or her research question, which was described in the previous chapter.

2. Not Identifying the Procedure Used to Adjust for Multiple Comparisons in ANOVA.

ANOVA is a group comparison that determines whether a statistically significant difference occurs somewhere among the groups studied. If a significant difference occurs, ANOVA is followed by a multiple comparison procedure that compares combinations of groups to determine which groups differ statistically.

3. Not Testing the Explanatory Variables for Interaction or Colinearity

Two explanatory variables are said to interact if the effect of one of the response variables depends on the level of the other. Interaction implies that the factors should be considered together, not separately. Two variables are said to be colinear if they are highly associated and therefore provide the same information in the model.

4. Not Indicating the Goodness-of-Fit of the Model to the Data

Goodness-of-fit indicates how well the model expresses the relationships observed in the data. In multiple regression analysis (not ANOVA), the value of R² should be reported. This value indicates how much of the variation in the response variable is explained by the factors included in the model. Thus, the higher, the better.

Errors in interpreting differences between groups include:

1.Not Reporting Confidence Intervals with Estimates

When interpreting any difference, whether it is statistically significant or not, the direction and magnitude of the difference should be evaluated. However, because a study is based on a sample of the population of interest, rather than on a census of the population, its results are actually estimates of the differences expected if the study were to be repeated on the entire population. Thus, another factor that should be considered when evaluating differences is the precision of the estimate.

2. Reporting Only Relative Differences and Not Absolute Ones

The absolute difference between groups is simply the mathematical difference between their values, whereas the relative difference is the absolute difference expressed as apercentage. By themselves, relative differences can mislead because they can make differences appear to be larger or smaller than they really are.

3. Not Differentiating Between Unit of Observation and the Number of Patients Improved

The unit of observation or the unit of analysis is what is being studied. In clinical research, the unit of observation is usually the patient. However, sometimes the unit issomething other than the patient. The problem comes when, say, differences are reported for the unit of observation but not for the number of patients in whom differences occurred. For example, if a drug markedly improves mean glomerular filtration rate in patients with renal disease, it may also be helpful to know how many patients actually improved.

Reference

Lang Tom. Common statistical errors you can find. Errors in multivariable analyses and in interpreting differences between groups. AMWA Journal, Vol 18, No 3. 2003

http://www.aliquote.org/cours/2006_cogmaster_A4/ressources/Statistical_Errors_Part2.pdf

Neuman Chapter 12: Correlation versus Causation and the Midterm Paper

Evren Eryilmaz — Apr 7, 2007 - 3:32pm

This chapter talks about the analysis of quantitative data, which we have already discussed in the notes Professor Ryan gave us. Although Neuman described the three elements of causality in the third chapter, he elaborates the difference between causality and correlation in this chapter. Based on the third chapter, three things are necessary to establish causality: temporal order, association, and the elimination of plausible alternatives.

In this context, I want to go back to the midterm paper we evaluated one more time. As you all remember, the midterm paper was an experimental study that looked at the relationship between “behavioral intention” to use and “positive mood” while considering uncertainty. The author established temporal order by measuring the mood of the subjects with a survey prior to the judgment task. The tables that are provided (for instance, uncertainty manipulation check with p value less than 0.001) showed that there was an association among the variables. However, I am not sure whether the author did a good job on eliminating alternative explanations for results. As Rosemary indicated, we do not know whether the students consumed the candy or perhaps some students were still a little bit drunk because they went to a club on Thursday night (I used to do this on Thursday nights often during my bachelors degree). Therefore, I believe that in the current format the author shows a correlation between positive mood and behavioral intention to use. However, the elimination of plausible alternatives needs to considered in order to show causality.

Comment on "Chapter 11 – Neuman"

Mitch Cochran — Apr 2, 2007 - 6:48pm

I like the chapter discussion of similar points. It shows that the same terms can mean a great deal of different things to different people. What is population? What is a person? When is a person a person in terms of the diamond carpool lane?