CONSTRUCTING QUESTIONNAIRES ASKING QUESTIONS CONSTRUCTING QUESTIONNAIRES

Survey Research Designs: Questionnaires Administer questionnaires to large samples to explain larger populations. Can guarantee anonymity Used extensively today by researchers, politicians, and news organizations in: Marketing Social research Public opinion Attitudes A widely used research technique Goal is to obtain an accurate picture of the individuals being studied. Snapshot of the group at a particular time. Surveys can be used to obtain scores for a variety of different research designs, A survey research design is conducted simply to obtain a description of a particular group of individuals.

Surveys: WHY? To obtain information from a large number of individuals To provide anonymity to respondents To provide confidentiality to respondents When observation is impossible – impractical When information is obtained for descriptive purposes When information is obtained for broad explanatory purposes May be used for exploratory, desrciptive and explanatory purposes. MOstly used in studies where individuals are the units of analysis.

Survey Information is collected by means of standardized procedures so that every individual is asked the same questions in more or less the same way. The survey's intent is not to describe the particular individuals who, by chance, are part of the sample but to obtain a composite profile of the population.

Steps in conducting a survey Follows a deductive approach: Develop hypotheses. Develop an instrument (questionnaire). (remember operationalization) Pilot test the questionnaire. Decide on population, get a representative sample. Administer the survey Analyze, interpret, and communicate the results Begin with a theoretical or applied research problems and ends with empirical measurement and data analysis. Questionnaire – to measure variables. Conceptualizes/operationalizes the variables as questions. Write and re-write questions. Plan how to analyze. Need to clearly define the population and then develop a strategy for adequately sampling from the population The more representative the sample, the more valid our conclusions from the survey 1. What should be asked? 2. How should each question be phrased? 3. In what sequence should the questions be arranged? 4. What questionnaire layout will best serve the research objectives? 5. How should the questionnaire be pretested? Does the questionnaire need to be revised?

Remember, the questionnaire is a total package and needs to be considered as such. It needs a strong introduction conveying to the respondent what the survey is about. It should indicate why the questions are being asked. It needs interesting and readily answerable questions at the beginning to gain respondent attention and build rapport. The conclusion should be gentle and friendly, expressing gratitude for the respondent's time and effort.

As respondents hear or read a survey question, they construct a “pragmatic meaning” that incorporates their interpretation of the question, why it is being asked, and what constitutes an acceptable answer. Respondents may also expand or restrict the meaning of concepts because the wording of a question evokes meanings that then dominate the definition of the concept. The respondent’s interpretation of the question can also be influenced by information in response categories

different groups may use different vocabularies or organize their behavior in different ways. to provide the analytic definition and ask the respondent to answer in its terms. to allow respondents to answer using their native concepts and structure the sequence of questions so that the analyst can map native concepts onto the analytic concept.

two general approaches to defining a target event or behavior. to provide the analytic definition and ask the respondent to answer in its terms. Next I’d like to know about vehicles, including automobiles, motorcycles, motor scooters, and trucks. How many vehicles do you own? to allow respondents to answer using their native concepts I’d like to know about the vehicles you own. How many vehicles do you own?  

Surveys: Types of Questions Open Ended What is your income? ________ What situations are particularly stressful for you? _________________________ A questionnaire is only as good as the questions it asks 1. What should be asked? 2. How should each question be phrased? 3. In what sequence should the questions be arranged? 4. What questionnaire layout will best serve the research objectives? 5. How should the questionnaire be pretested? Does the questionnaire need to be revised? Types of questions important: Different types of questions encourage different types of answers. Answers can be restricted or participants can be given more freedom in their responses. Wording of a question is very important. Type of question to be asked depends on the issue, the audience + the statistical analyses. If nominal questions-- cannot compute averages etc, only count the numbers. +ve: greatest flexibility in choosing the answer. Likely to reveal individuals’ real thoughts or opinions -ve: making sense of the answers, grouping the answers is a challenge.

Surveys: Types of Questions closed ended Do you agree that Turkey should join the European Union and have free labor exchange with other European countries? Yes/No Eğitiminde hedefin nedir? Bu okuldan sonra okumayacağım Lise’ye gideceğim Meslek okuluna gideceğim Üniversiteye gideceğim Provide a limited number of response alternatives and ask to select the best/most appropriate answer. Easy to analyze, and summarize. Can compute some statistical analyses. Choices should be exhaustive and mutually exclusive.

Developing survey items Statement Rating - Likert Scales Asks the participant to rate items using a predetermined scale. Rate the below statements on the scale shown to indicate your level of agreement: I think that everyone should vote. strongly agree agree uncertain disagree strongly disagree Most politicians cannot be trusted. SCALE of measurement? Interval -- produce numerical values allowing for statistical analyses. We make sense of these scales- by adding responses given to a set of items. Then compare individuals’ scores. Distances between the responses are accepted to be equivalent. Should have response sets for both alternatives, ranging equally between agree- disagree, negative - positive… +ve: statistical analyses. Easy to understand. -ve: possibility of a RESPONSE SET. P marks the same answer for all questions. Solution: present a mixture of positive and negative statements. ask the same question using some alternate wordings.

[Not: Anketörlerin doğru meslekleri öğrenip hatırlaması gerekiyor. ] İSİM AŞİNALIĞI Şimdi size bazı isimler sayacağım. Bana bu isimleri tanıyıp tanımadığınızı ve tanıyorsanız ne iş yaptıklarını söyleyebilir misiniz? [Not: Anketörlerin doğru meslekleri öğrenip hatırlaması gerekiyor. ] Tanıyor ve mesleğini doğru biliyor Tanıyor ama mesleğini bilmiyor ya da yanlış biliyor Hiç Tanımıyor Nuri Bilge Ceylan 1 2 3 K1. Dan Brown [Den Brawn] K2. Orhan Pamuk K3. James Cameron [Jeyms Kemırın] K4. Fazıl Say K5. Johannes Brahms [Yohannes Brams] K6. Vincent Van Gogh [Vinsınt Van Goh] K7. Tony Blair [Toni Bleyır] K8. Nimet Çubukçu K9.

Kesinlikle katılmıyorum Katılmıyorum Katılıyorum  [d1]Din dersleri Şimdi size bazı cümleler okuyacağım. Bu cümlelerin belirttiği görüşlere ne derece katıldığınızı veya katılmadığınızı söyleyebilir misiniz? NOT: KARTI GÖSTERİNİZ. Kesinlikle katılmıyorum Katılmıyorum Katılıyorum Kesinlikle katılıyorum Bilmiyorum/ Fikrim yok Ülkemiz evlilik ve aile konusunda geleneksel değerlerine geri dönmelidir. 1 2 3 4 99 K1. Dini değerlere daha çok önem verilmesi ülkemiz için daha iyi olur. K2. Üniversitelerdeki başörtüsü yasağı kaldırılmalıdır K3. Ramazan ayı içinde alkol satışları yasaklanmalıdır. K4. İfade özgürlüğü her koşul altında korunmalıdır K5. Devlet memuru kadınların başörtüsü takmasına izin verilmemelidir. K6. Okullarda din dersleri  zorunlu olmalıdır. K7. Kadın ve erkeğin birlikte yaşaması için imam nikâhı gereklidir. K8. Yabancılar ülkemizde toprak ve gayrı menkul satın alabilmelidir. K9. Ülke ekonomisi dışarıya daha fazla açılmamalıdır. K10. Ahlaklı olmak dindar olmayı gerektirmez. K11. Gerektiğinde asker yönetimi ele almalıdır. K12. Gerektiğinde siyasi partiler kapatılmalıdır. K13.

Yabancılar ülkemizde toprak ve gayrı menkul satın alabilmelidir. 1 2 3 4 99 K9. Ülke ekonomisi dışarıya daha fazla açılmamalıdır. K10. Ahlaklı olmak dindar olmayı gerektirmez. K11. Gerektiğinde asker yönetimi ele almalıdır. K12. Gerektiğinde siyasi partiler kapatılmalıdır. K13.

Developing survey items: Guidelines Make items clear Short Avoid negatives Avoid double barreled questions Answering the question should be possible Questions should be relevant Respondent cooperation should be likely Questions should have face validity Avoid biased or loaded terms Avoid Complexity: use simple, conversational language Avoid leading and loaded questions Avoid ambiguity: be as specific as possible Avoid double-barreled items (omit: and, or, etc.) Avoid making assumptions Avoid burdensome questions

Make items clear Short ??? Avoid ambiguity and vagueness? What is your income? Do you exercise regularly? What do you think about the proposed peace plan? How many times did you take the subway last week?

Avoid negatives Appearence of negation leads to misinterpretation Examples The United States should not recognize Cuba Students should not be required to take a comprehensive exam. (A respondent who disagree with this statement…) Does it seem possible or does it seem impossible to you that the Nazi extermination of the Jews never happened?

Avoid biased items and terms More support less support Assistance to the poor welfare Halting rising crime rate law enforcement Dealing with drug addiction drug rehabilitation Solving problems of big cities urban assistance program

Avoid double barreled questions Examples: United States should abandon its space program and spend the money on domestic programs Do you listen to music and have the TV on while you are studying?

Answering the question should be possible Questions should be relevant Example Biliyorsunuz devlet verilen vergilerle bazı hizmetleri yerine getiriyor. Aşağıda size bazı hizmet alanları sayacağız. Bunların her biri için devletin aldığı vergilerden daha az mı, daha çok mu, yoksa şimdi harcadığı kadar mı harcama yapması gerektiğini düşünüyorsunuz? Yani mesela, devlet toplam 100 lira vergi topluyorsa bunu aşağıdaki hizmetler arasında nasıl dağıtmalı, paylaştırmalı? Eğitim Savunma Sağlık Sanayi-kalkınma SanatSu-kanalizasyon-yol Bilimsel araştırma-geliştirme Yoksulluk Göç

Avoid biased or loaded terms Bias refers to any property of questionnaire that encourages respondents to answer in a particular way Examples: Do you agree or disagree with the recent Supreme Court decision….. Should the government spend even more money on military?

Avoid asking questions beyond the capabilities of respondents: How many hours of TV did you watch last month? Evinizde yaklaşık olarak kaç kitap var? 1> 1-25 arası 2> 26-50 arası 3> 51-200 arası 4> 201- 400 arası 5> 401 + 99> Fikri Yok / Cevap Yok Geçtiğimiz sene içinde kaç tane kitap okudunuz?  Cevap:................................................................

Theratening questions: Topic % very uneasy Masturbation 56 Sexual intercourse 56 Use of marijuhana 42 Drugs 31 Getting drunk 29 Kissing 20 İncome 19 Gambling 12 Happiness 4 Education 3

OTHER ISSUES Close vs. open-ended questions Contingency questions Question order Order effects Context effects non-attidues and middle positions Probing Social desirability Format and layout Lenght of survey

Indexes and scales They are composite measures of variables: measurement based on more than one variable (data item)

Political activism index An index is constructed thru simple accumulation of scores assigned to individual attributes Political activism index Signed a political petition Wrote a letter to public officer Gave money to a political cause Gave money to a political candidate Persuaded someone to change their voting plans Wrote a political letter to the editor

Need to have face validity (logical validity) Unidimensionality—a composite measure should represent only one dimension İtems should be correlated with each other

Index Example Tax compliance: Diyelim ki bir gelir elde ettiniz ve bu gelirden ödemeniz gereken bir vergi miktarı var. Ama ödemeseniz de hiçbir şekilde fark edilmeyecek. Ödemeniz gereken bu vergiyi, ödemezseniz fark edilmeyeceği halde, yalnızca vatandaşlık göreviniz olduğu için öder misiniz? Diyelim ki 100 liralık bir malı nakit ödeyerek almaya karar verdiniz. Satıcı size fiş istemezseniz malı 90 liraya vereceğini söyledi. Biliyorsunuz vergi iadesi de kalktı. Siz ne yaparsınız? Diyelim ki bir ev kiralamak üzeresiniz. Ev sahibi daha az vergi vermek için kira miktarını kontratta(sözleşmede) düşük göstermeyi planlıyor. Bunun karşılığında, kira miktarında indirim yapacak. Ev sahibinin kira bedelini kontratta düşük göstermesini kabul eder misiniz?

Hiç kimse vergisini ödemese bile, biz aile olarak üzerimize düşen vergiyi ödemeliyiz. Bu düşünceye... Bildiğiniz gibi Türkiye’de kimileri vergilerini ödüyor, kimileri ödemiyor. Ailenizin ödediği vergileri göz önünde bulundurduğunuzda şu sayacaklarımdan hangisinin görüşünüze en uygun olduğunu düşünüyorsunuz? (SEÇENEKLERİ OKU, TEK CEVAP) 1> Insanlar kendi payına düşen vergilerin tamamını mutlaka ödemelidir 2> İnsanlar kendi paylarına düşen vergilerin çoğunu ödemelidir 3> İnsanlar fırsatını bulduklarında vergilerinin çoğunu

SCALES A scale is constructed thru the assignment of scores to patterns of attributes İntensity İtems represent different degrees of intensity Ran for office Worked on a political campaign Conributed money to a political campaign voted

Bogardus Social Distance Scale “ Are you willing to permit Albanians to live in your country? Are you willing to permit Albanians to live in your community? Are you willing to permit Albanians to live in your neighborhood? Are you willing to permit Albanians to live next door to you? Are you willing to permit Albanians to teach in your children’s school? Would you let your child marry an Albanian?

Interview bias Errors by the respondents Unintentional errrors or interviewer slopiness İntentional subversion by the interwiever Influence due to the interviewer’s expactations Failure of an interviewer to probe İnfluence on the answers due to interviewer’s apperance, tone, attitude, reactions to answers

OTHER ISSUES Contingency questions Question order Order effects Context effects non-attidues and middle positions Probing Social desirability Format and layout Lenght of survey

Administration of surveys Face-to-face surveys Self –administered surveys Mail surveys Telephone surveys Internet (online / e-mail) surveys Mixed mode surveys Developes specific items. Complied them into a quesitonnaire format How are you going to access your respondents and make them respond to your questions--- There are a number of options for administering surveys. Again Depends on: audience (education) sensitivity

A Summary of the Strengths and Weaknesses of the Survey Research Design

Survey response rates Response rate: Enhancing response rates # of surveys returned / #of surveys distributed (participants contacted) 50- 60 % acceptable. 70 - 80% better Lower percentages are acceptable for pre-post designs. Enhancing response rates Follow-up Gifts Presentation of the interviewers Survey modes lend themselves to higher/lower response rates

Strentgths and weaknesses of survey research design Strengths: Useful in describing large populations. Make large samples feasible Can ask questions from a variety of topics Easy to code/analyze Strong in reliablity Weaknesses: Standarization Self-administration can be problematic Weak in validity