This section covers the analysis of quantitative and qualitative data, and reporting the enquiry. Data analysis can broadly b divided into two main categories: exploratory and confirmatory. Exploratory analysis attempts to find out what can be induced from the data while confirmatory analysis attempts to establish and provide support to is expected. The author presents a clear statement about some key terms including statistical significant, measure of central tendency and variability, analysis between two variables, and relationship between variables. Statistical significant, p value, is the value that tells whether the result is due to chance. If p<0.01 or 0.05, the result is less likely due to change. Meaning, p effect is >0.99 or >0.95, showing there exist an effect.Measure of central tendency and variability includes mean, mode, median, range, variance, standard deviation, standard error. Tools for analyzing relationship between two variables includes cross tabulation using Chi-square tests for distribution.

For hypothesis testing, varieties of tools are employed depending on whether the distribution describes attributes or variables. Analytic tools for parametric normal distributions includes: independent sample T-test, One-way ANOVA, Factorial ANOVA, and GLM MANOVA. Non-parametric (non-normal distribution) analytic tools includes: Pearson Chi-square (test of association), Mann-Whitney U test for two independent samples, Kruskal-Wallis Test for several independent samples, McNemar significance of change  for related sample, Wilcoxon Signed Ranks Test (ordinal) for two related sample, Friedman Test for several related samples.
For relationship between factors, structural equation modelling is used to assess the relationship and to determine goodness of fit for the model.
Analysis of Qualitative Data. I looked at the coding scheme using NUDIST aspect of analysing qualitative data. When creating codes, good place to start is creating a ‘working list’ of codes prior to fieldwork. The list should contain thematic items from the conceptual framework, research questions, hypotheses and problem area. Coding can be done using open coding, axial coding and selective coding.

Under reporting the enquiry, I found the items the author talked about very useful. They reinforced Booth’s et al. guidelines on crafting research. In particular I found the following guidelines helpful: now what is expected, provide an ‘executive summary’ – an abstract, put as much material as possible into appendices, and adhere to professional standard.

Overall, I am grateful to Robson and every author referenced in this book for providing the discipline with this real world research manual. Within this very short time of reading, I was able to learn new ways of discovery and navigate through difficult research formation terrains using some of the guides in this book. I also enjoyed the stimulating discussions held in class. Thanks everyone.

This part describes how we should deal with the data, which is quantitative or qualitative, collected from previous step. SPSS (The Statistical Package for the Social Sciences) is introduced to show how to analyze quantitative data. Before analyzing data each response for question in the questionnaire should be translated into appropriate code then enter these codes in the SPSS. If there is missing data ninety nine or -1 are frequently used instead 0. Data analysis is commonly divided into two broad types such as exploratory and confirmatory. While exploratory analysis explores the data, trying to find out what they tell you confirmatory analysis seeks to establish whether you have actually got what you expected to find. Particularly confirmatory data analysis (CDA) is the mainstream approach in statistical analysis. The main aspects of SPSS are follows.

 -       Significance test : Quantitative analysis is virtually synonymous with significance testing.   What a p value actually tells you is how likely it would be that you would get the difference you did. If the p value is small rather than large, this makes it less likely that the result is due to chance variation rather than to a real difference. In addition the chance of obtaining a statistically significant result increases as the sample size increases.

-       Measures of central tendency : The most commonly used are Mean, Median, and Mode.

-       Measures of variability : Some commonly used measures are Range, Inter-quartile range, Variance, Standard deviation, and Standard error.

-       Deviation : deviation = (individual score) – (mean score)

-       Standardized Score : standardized score = (individual score – mean score)/(standard deviation)

-       The normal distribution: The shape of normal distribution is completely determined once the mean and standard deviation are known. Here median and mode coincide with the mean.

-       Correlation coefficients : This gives an indication of both the strength and the direction of the relationship between the variables.

-       Multiple regression: This is multiple in the sense that it involves a single dependent variable and two or more independent variables.  The regression equation is Y= a + b1x1 + b2x2 where y is the dependent variable, x1 and x2 are the two independent variables, a is the intercept, and b1 and b2 the regression coefficients for the two independent variables.

-       R-squared : This is the multiple coefficient of determination, measure of the proportion of the variance in the dependent variable which is explained by the independent variables in the equation.

-       Factor analysis : This is an approach to making sense of a large number of correlations between variables.  

 

The author also explains how to analyze qualitative data in chapter 14. Tesch (1990) lists forty-six labels that qualitative researchers have used to describe their approach and reduces these to four basic groupings. These are the characteristics of language, the discovery of regularities, the comprehension of the meaning of text or action, and reflection. Crabtree and Miller (1992) produce a different typology, more closely linked to the method of data analysis used. These are quasi-statistical methods, template approaches, editing approaches, and immersion approaches. Author empathize that the central requirement in qualitative analysis is clear thinking on the part of the analyst. For the qualitative analysis NUD*IST is introduced and author describes how document system and index system can be used, how to make link the document and index system, and how to display data.  

 Reporting on the Enquiry part, especially, was useful for me because it describes the appropriate format for the report depends on the nature and purpose of the enquiry. The checklist of report for scientific journal which is introduced in Box 15.1 was very clear to understand and I need to follow this instruction for my final research paper.

Robson uses the term “attractive nuisance” to advise against our falling into the trap of being so taken in by the advantages of qualitative data that we fail to consider its limitations and incidental problems.  Some of the caveats to be borne in mind are:

·         The researcher may only know roughly in advance what he/she is looking for. As such, it is difficult, especially for beginning researchers, to determine when enough data has been collected.

·         Because it usually reflects individuals’ interpretation of events, the qualitative research of which it is a part is less able to be generalized than quantitative research.

·         The research problem, not our aversion or liking for a particular research method, should be the overriding determinant for the type of data, collection methods, and research design we adopt.

·         The fact that qualitative designs are not fixed but emerge as the study unfolds is a good reason to be well prepared when undertaking such research.

·         Being well prepared includes the mental readiness to rigorously analyze decisions points reached and succinctly explain conclusions made.

·         Because qualitative data comes in the form of words, pictures, or objects, qualitative studies can generate huge amounts of data, often requiring a lot of care to manage.

Robson also gives an in-depth discussion on the use of the NUD*IST software for analyzing the large amounts of qualitative data necessarily generated in the course large scale studies such as a doctoral dissertation might entail. But one requires having the software installed in order to fully understand this discussion. I know we have SPSS installed in the computers at the lab. Does anyone know what the situation is regarding NUD*IST, NVivo, or any qualitative research software at that? 

According to Robson's, there are three traditions in qualitative research: case studies, ethnographic studies, and grounded theory studies. He mentions the importance of the quality of the analyst in qualitative research. As Fettrman (1989) puts it, in the context of an ethnographic stance, the analysis is as much a test of the enquirer as it is a test of the data. Qualitative analysis remains much closer to codified common sense than the complexities of statistical analysis of quantitative data. However, humans as “natural analysis” have deficiencies and biases corresponding to the problems that they have as observers.

 

He also mentions some common features of qualitative data analysis:

1. Giving codes to the initial set of materials obtained from observations, interviews,
  etc.;

2. Adding comments, reflections, etc.;

3. Going through the materials trying to identify similar patterns;

4. Taking these patterns out to the field to help focus the next wave of data collection;

5. Gradually elaborate a small set of generalizations;

6. Linking these generalizations to a formalized body of knowledge in the form of
  constructs or theories.

  

One of the more famous qualitative research studies is the work of Margaret Mead, who studied the Samoan culture. Such studies not only rely on personal observation but also often require the recruitment of informants. Studies such as those conducted to see what life was like for first-generation immigrants who came to the United States in the early part of the twentieth century can be a qualitative study. Interviewers can interview a number of first-generation immigrants and develop life histories. With enough life histories showing similar patterns of behavior a picture can be developed as to what life was like for those who lived in that era. The interviews might be tape recorded. The interview process is conducted in such a way and in such a length to enable the informant to adjust to the interviewer and the recording device. It is part of the plan of qualitative research to carefully choose the interviewer in order to have the best match to the informant.

 

Quantitative research seldom deviates from the research plan. Qualitative research, on the other hand, is more flexible. However, qualitative research has several advantages over quantitative research. Qualitative research uses direct observation and semi-structured interviewing in real-world settings. The researcher looks for social transactions and interactions between people and events. The data collection process is less structured than quantitative research. The researchers may make a number of adjustments during the observations. The researcher may even develop new hypotheses during the research process. Qualitative research is more naturalistic, participatory, and interpretive.

   

Because data quality, reliability, and validity is a huge issue in research, a researcher must carefully plan his or her survey instrument. Survey is defined broadly as a research strategy that describes the practical and tactical matters surrounding survey instruments. These instrument are mostly questionnaires and interviews.
Questions in Questionnaire exhibit the following properties:

• Simple language avoiding jargons and ambiguities
• Question frame of reference is clear
• Closed-ended option laden question
• Partially closed-ended questions option laden question with ’others’ as option
• Questions means the same thing to all respondents
• Rating numeric scale indicating the direction and strength of the response
• Likert rating scale to judge level of agreement to a statement

When designing questions, the following should be avoided:

• Loaded questions including nonneutral or emotional laden terms
• Leading questions swaying the responded to answer in a desired manner
• Double-barreled questions asking for more than one thing
• Questions in negative making understanding difficult
• Prestige bias
• Creating opinions
• Direct questions on sensitive topics (de Vaus, 2001; Jackson, 2008)

Participant observation is perhaps the most personally demanding and analytically difficult method of data collection to undertake. It requires us as researchers to spend a great deal of our time in surroundings that we may not be familiar with; to develop and sustain relationships with people with whom we have little personal affinity; to take a lot of notes on apparently mundane activities; to expose ourselves to incidental risks of the environment of interest; and to spend months of analysis after the fieldwork, analyzing field-notes and diaries. Yet, for those of us hoping to do qualitative research, it is known to be a most rewarding method, yielding interesting insights into participants’ social lives and relationships that far outweigh any benefits derivable from fixed design methods.

Robson distinguishes four different participant observer roles:

• Complete participant: the researcher employing this role hides their identity as an observer and attempts to engage fully in the activities of the group or organization under investigation. This method could be used to collect more accurate data by a researcher investigating, say, a racist or fascist organization. It advantages, notwithstanding, its clandestine approach has strongly been objected to in terms of being ethically indefensible.
• Participant as observer: the researcher adopts an overt role, making their presence and intentions known to the group. Some have raised doubts as to whether the observer will be able to establish the necessary level of rapport with the participants after revealing their identity. As Robson observed, it is important for the observer to get the trust of key member of the group.
• The marginal participant: the researcher is uninvolved and detached, and merely, passively records behavior at a distance (e.g., a researcher sitting in a classroom, making observations of pupils and their teacher).
• Observer as participant: here the researcher makes known their status as researcher but moves away from the idea of participation. This would usually call for relatively more formal observation (e.g., ownership and structure of a firm, rather than its internal practices and norms) than either informal observation or participation. Here, there is a higher possibility of misunderstanding as as the researcher and participants are not as bonded as is the case with other observational methods. Robson argues that it is still questionable whether the researcher can be said to be a complete non-participant since he/she becomes a meaningful member of the group throughout the duration of the research.

The part III describes various methods of data collection such as observation and interviewing using questionnaires, as well as administering tests.  Surveys are more like a research strategy than a tactic or specific method and it gives reassuring scientific ring of confidence. Also reliability and validity of survey data depend to a considerable extent on the technical proficiency of those running the survey. If the questions are incomprehensible or ambiguous, the exercise is obviously a waste of time and this is a problem of internal validity. Another problem, generalizability or external validity can be produced by the sampling fault.

Survey are almost carried out as part of a non-experimental fixed design and while this can be for any of the research purposes like exploratory , descriptive, explanatory or emancipatory, surveys are not well suited to carrying out exploratory work. Most of surveys include the use of questionnaire and there are three main ways in which this questionnaire is administered. These are self-completion, face-to-face interview, and telephone interview. In addition questionnaire needs pre-testing and the larger the sample, the lover the likely error in generalizing.

Interview is a one of research methods and it includes structured, semi-structured and unstructured interviews. While fully structured interview has predetermined questions with fixed wording, in a pre-set order semi-structured interview has also predetermined questions, but the order can be modified based upon the interviewer’s perception. Unstructured interviews are used for obtain interviewer’s general idea of interest and concern and it can be completely informal. Both of semi-structured and unstructured interviews are widely used in flexible, quantitative designs.

This part also explains various tests and scales to measure attitudes such as Likert scale, Thurstone scale, Guttman scale, and semantic differential scales. Critics of both Thurstone and Likert scale have pointed out that they may contain statements which concern a variety of dimensions relating to the attitude of concern. However the Guttman approach overcomes this complexity by seeking to develop a unidimensional scale. Last chapter of this part describes observational methods. The two polar extreme types are participant observation and structured observation. While participant observation is an essentially qualitative style and originally rooted in the work of anthropologists, structured observation is a quantitative style which has been used in a variety of disciplines. Also participant observation is a widely used method in flexible designs and structured observation is almost exclusively linked to fixed designs, of both experimental and non-experimental types.

I learned there are so many research methods for each appropriate research question to collect and analyze data. This part is very helpful to consider research method which fit into my research question.    

 

I found the third part of Robson textbook of a great reference to researchers once they reach to the stage of designing a questionnaire or surveys. Robson emphasizes in chapter 8 the various types of surveys and, advantages and disadvantages for each kind. Basically, survey can be defined as a questionnaire that consists of questions that need to be answered by sampling responders. Researchers need to drive Survey’s questions from the research questions or hypothesis they seeking to test. Questionnaire is like a piece of art that researchers draw. It has certain principles that need to be followed to get the best out of it. Interviews can be categorized into three categories; self-completion where responders fill the answers by themselves and the questionnaire is posted by mail, face-to-face where researchers ask questions to responders and fill the answers by themselves, and telephone interviews where researchers dial the responders, ask the questions, and record them. Robson gives a clear comparison among those different approaches in terms of costs, time, biases and other different critical criteria in page 237.

 

How can researcher carry out a survey? This question stops many beginner researchers including me. However, Robson answeres this question clearly in the third part of his textbook. There are 5 steps to carry out a survey; those steps are:

 

1. Initial design and planning: during this stage, researcher determines the purpose of his/her study, the questions he/she seeks to get answers for, the study population, and the sampling frame.
2. Designing the questionnaire: In this stage, researcher design the questions in a way that helps in achieving the goal of research and answering research questions
3. Pre-testing: In this stage, researcher takes constructive comments and thoughts from friends, colleagues, responders from the sample group after they read the questions to avoid unambiguous questions and finally implement those comments and suggestions.
4. Final design and planning data collection: This stage involves editing the final questionnaire after modifications.
5. Data analysis and reporting. Researcher gathers the returned responses and may re contact responders in case there is no response, if any, and finally the coding stage starts to help researcher organize, quantify, and analyze the collected data.

 

Robson also in chapter 9 gives explanation for the different types of interviews and he categorized them into three types, structured, semi- structured, and unstructed interviews.

Structured interviews have predetermined questions with fixed wording. As for the semi structured interviews have predetermined questions but the order of questions and its wording can be modified. For the unstructured interviews, the conversation between the researcher and responders about certain topic raises issues within the context of this particular topic. In chapter 10, Robson sheds the light on the different scales and tests (such as Likert, Guttman, and Semantic Differential approach) Moreover, Robson explains the steps in developing each kind of tests and he also provides different examples for those tests. I found this chapter of great value because it helps the reader to differentiate among those tests and to explore the advantages and disadvantages of each. For chapter 11 and 12, I would rather prefer commenting on them when I finish reading them.

 

After reading some few pages of (Kohli & Kettinger, 2004), I delved into the observational methods in Robson part III to improve my understanding of the article.
Observational Methods. Observational methods enable researchers to observe human or animal behaviour. Most studies starts using these method either to gain an initial understanding of the phenomenon in question or as a main research method. Observational methods could be classified broadly as formal and informal observation based on the degree of pre-structure in the observation exercise. Formal observation imposes a lot of structure and direction regarding what can be observed while informal observation does not. Researches that employ observational methods could be further be classified as following:
Nonparticipant observation research – where the researcher does not participant in the situation involving the participants.
Participant observation research – where the researcher is actively participate in the situation involving the participants. Here a researcher might play the role of an observer at the same time as a participant, though this dual role might be difficult.
Disguised observation research – where the participants are unaware that their behaviour are being observed. This method might involve ethical issues if the participants are human subjects. Here are some key points to note when using observational method in action research.
Collecting data. A participant observer observes the people in the research focused group while being involved. Data are recorded on the spot during the event using methods similar to that of interview like voice recording. The observer might include the following data: running descriptions, notes of recalls of forgotten materials, notes offerings on the interpretation of the situation, personal impressions and feelings, and reminders to look for additional information.
Before the data can be analyzed in a meaningful way, it must be coded using schemes. Common coding schemes include checklist and category systems. Checklist might be static – a tally sheet used to record attributes that does not change, or action – a tally sheet used to record the presence or absence of behaviours. Category systems utilised coding scheme to record what is being observed. It is advised that researchers should use an existing coding scheme when available. But if a new coding scheme is required, it should have the following properties: (1) focus on selected aspect of the inquiry, (2) be objective, (3) non context-dependent, (4) exhaustive, (5) mutually exclusive, and (6) easy to record.

In part 3, Robson discusses the reliability and validity of survey data. He mentions that they both depend to a considerable extent on the technical proficiency of those running the survey. Poorly designed questions may lead to the problem of internal validity, where the survey may not obtain valid information about the respondents and what they are thinking.

 

On the other hand, if the sampling is faulty, this produces a generalizability or external validity problem. Another type of external validity problem occurs if we seek to generalize from what people say in a survey to what they actually do.

 

Robson mentions that reliability is more straightforward. By presenting all respondents with the same standardized questions, carefully worded after piloting, it is possible to obtain high reliability of response.

 

Survey research has a unique advantage among social scientific methods: it is often possible to check the validity of survey data. Some of the respondents can be interviewed again, and the results of both interviews checked against each other. It has been found that the reliability of personal factual items, like age and income, is high. The reliability of attitude responses is harder to determine because a changed response can mean a changed attitude. The reliability of average responses is higher than the reliability of individual responses. Fortunately, the researcher is usually more interested in average, or group measures, than in individual responses.

 

One way of checking the validity of a measuring instrument is to use an outside criterion. One compares the results to some outside, presumably valid, criterion. Ordinarily, individual behavior is not checked because information about individuals is difficult to obtain, but group information is often available. This information can be used to test to some extent the validity of the survey sample and the responses.