In this chapter, I liked comparative research and secondary sources (similar to Chapter 11). However, I am not sure whether I understood Neuman when he said “most positivist research is not comparative”. When we conduct a positivist study, to a certain extent, don’t we compare our study to other current formal studies in order to show that what we are doing is in tune with other researchers’ attempts.

This chapter also made me think about our previous review. I think it is possible to consider the “Complementary Use of Modeling Techniques: Insights from Representation Theory and Practice” as a comparative research to some extent. There are conflicting ideas on ontological foundations of conceptual modeling between WandWeber and Wyssusek. In this context, the authors of that paper compare these conflicting approaches and try to justify their study based on this comparison. Furthermore, the authors also compare the results of this study with a similar, recent study to provide further confidence for the conclusion.  

According to Neuman, “field research is valuable for micro-level or small-group face-to-face interaction and it is less effective when the concern is macro-level processes and social structures”. In this context, I wonder to what extent field study is applied in our discipline. For instance, is it possible to link ethnographic study to case study or perhaps to grounded theory when conducting IS research?

The concept of significant digits or figures states that you cannot add digits after the initial values.  For example if you have a value of 3.5 you cannot arbitrarily make it 3.500.  The value that you took is actually be between 3 and 4.  It is closest to the middle but how close?  The researcher may not know if it is 3.6 or 3.4.  So looking at the raw data it would be unreasonable to assume that if you saw a value of 3.5 to assume that it is 3.500000000.  You need to need the precision of the experiment and how many significant digits there are.

We can tie this concept back to the chapter by looking at the amount of information in the different scales.  An ordinal scale will not have as much information in it as an interval scale.  There is a hierarchy in that you can go downward from an interval scale to an ordinal scale but not vice versa.   

1. Not confirming that the data met the assumptions of ANOVA.

 ANOVA assumes that the response variable is approximately normally distributed within each level of the explanatory variable and that the variability of these distributions is approximately the same. If data are not data are not normally distributed, than the data need to be mathematically transformed into distributions that are more normally distributed.In a way, this error also sound similar to a researcher using secondary data or existing statistics that are inappropriate for his or her research question, which was described in the previous chapter. 

2. Not Identifying the Procedure Used to Adjust for Multiple Comparisons in ANOVA.

 ANOVA is a group comparison that determines whether a statistically significant difference occurs somewhere among the groups studied. If a significant difference occurs, ANOVA is followed by a multiple comparison procedure that compares combinations of groups to determine which groups differ statistically. 

3. Not Testing the Explanatory Variables for Interaction or Colinearity

 Two explanatory variables are said to interact if the effect of one of the response variables depends on the level of the other. Interaction implies that the factors should be considered together, not separately. Two variables are said to be colinear if they are highly associated and therefore provide the same information in the model. 

4. Not Indicating the Goodness-of-Fit of the Model to the Data

 Goodness-of-fit indicates how well the model expresses the relationships observed in the data. In multiple regression analysis (not ANOVA), the value of R² should be reported. This value indicates how much of the variation in the response variable is explained by the factors included in the model. Thus, the higher, the better. 

Errors in interpreting differences between groups include: 

1.Not Reporting Confidence Intervals with Estimates

When interpreting any difference, whether it is statistically significant or not, the direction and magnitude of the difference should be evaluated. However, because a study is based on a sample of the population of interest, rather than on a census of the population, its results are actually estimates of the differences expected if the study were to be repeated on the entire population. Thus, another factor that should be considered when evaluating differences is the precision of the estimate.  

2. Reporting Only Relative Differences and Not Absolute Ones

The absolute difference between groups is simply the mathematical difference between their values, whereas the relative difference is the absolute difference expressed as apercentage. By themselves, relative differences can mislead because they can make differences appear to be larger or smaller than they really are.  

3. Not Differentiating Between Unit of Observation and the Number of Patients Improved

The unit of observation or the unit of analysis is what is being studied. In clinical research, the unit of observation is usually the patient. However, sometimes the unit issomething other than the patient. The problem comes when, say, differences are reported for the unit of observation but not for the number of patients in whom differences occurred. For example, if a drug markedly improves mean glomerular filtration rate in patients with renal disease, it may also be helpful to know how many patients actually improved. 

Reference

Lang Tom. Common statistical errors you can find. Errors in multivariable analyses and in interpreting differences between groups. AMWA Journal, Vol 18, No 3. 2003

http://www.aliquote.org/cours/2006_cogmaster_A4/ressources/Statistical_Errors_Part2.pdf 

In this context, I want to go back to the midterm paper we evaluated one more time. As you all remember, the midterm paper was an experimental study that looked at the relationship between “behavioral intention” to use and “positive mood” while considering uncertainty. The author established temporal order by measuring the mood of the subjects with a survey prior to the judgment task. The tables that are provided (for instance, uncertainty manipulation check with p value less than 0.001) showed that there was an association among the variables. However, I am not sure whether the author did a good job on eliminating alternative explanations for results. As Rosemary indicated, we do not know whether the students consumed the candy or perhaps some students were still a little bit drunk because they went to a club on Thursday night (I used to do this on Thursday nights often during my bachelors degree). Therefore, I believe that in the current format the author shows a correlation between positive mood and behavioral intention to use. However, the elimination of plausible alternatives needs to considered in order to show causality.

http://academic.csuohio.edu/kneuendorf/content/resources/flowc.htm

• Content analysis describes what is in the text.
• The data are in permanent form and hence can be subject to re-analysis, allowing reliability checks and replication studies.
• It is based on existing documents. Hence, a researcher can observe without being observed.
• It may provide a low cost form of longitudinal analysis when a run or series of documents of a particular type is available.

• Content analysis can not reveal the intentions of those who created the text or the effects that messages in the text have on those who receive them.
•  The documents available may be limited or partial.
• The documents have been written for some purpose other than for the research.
• It is difficult to assess causal relationships. Are the documents causes of the social phenomena the researcher interested in, or reflection of them? (Robson, 358)

Opportunities:

• A researcher can measure large amounts of text with sampling and multiple coders.
• It is helpful when a topic must be studied at a distance.
• Content analysis can reveal messages in a text that are difficult to see with causal observation.

Threats:

• Intercoder reliability. The researcher must check the degree of consistency among coders.
• Visual text communicates messages or emotional content indirectly through images, symbols, and metaphors. Hence, it is difficult to measure.  

In this context, as Nicole indicated, I wonder how the PhD students in our school approach to content analysis other than literature reviews. I have not heard a student conducting a meta-analysis in our school. It seems to that most of them prefer conducting surveys. As for me, the findings from a meta-analysis paper can be as interesting as a survey research.

For instance: Dubé and Peré's Rigor in Information Systems Positivist Case Research: Current Practices, Trends, and Recommendations paper or the working paper I cited last week. These papers showed me how other researcher conduct research and the mistakes that they made.

The data is already available .Why not use it? Isn't anyone interested in examining existing data?

1) Single method designs where multiple methods are needed

2) Unsystematic and often inadequate procedures

3) Low response rates

4) Weak linkages between units of analysis and respondents

5) Over reliance on cross-sectional surveys where longitudinal surveys are really needed 

Based on the findings, the authors say that “the quality of surveys varies significantly among studies of different purposes: exploratory and descriptive studies are of moderate to poor quality overall, and explanatory studies are of good quality. The lack of rigor in descriptive and exploratory surveys is unfortunate”. In summary, the authors in this article show disappointment after considering the extent to which survey research is used and the proportion of survey-based studies in MIS that fail to measure up. 

Pinsonneault A., Kraemer L. K: SURVEY RESEARCH METHODOLOGY IN MANAGEMENT INFORMATION SYSTEMS: AN ASSESMENT (Working Paper )