I need your help to come up with survey questions! - Discussion
I need your help to come up with survey questions!
Dada Kind, modified 5 Years ago at 7/25/19 3:08 PM
Created 5 Years ago at 7/25/19 12:02 AM
I need your help to come up with survey questions!
Posts: 633 Join Date: 11/15/13 Recent Posts
The interest I got last year about this was tepid.
https://www.dharmaoverground.org/discussion/-/message_boards/message/8180069
But, I felt motivated on this again today so I came up with a ton of questions. I would like to get feedback before I make the real survey. Currently it reflects my own biases too heavily. These are just rough notes. I'm still unsure in some places which questions should be multichoice vs scales etc etc.
In the real survey I will organize these somehow, but for now they're pretty much just puked into my notes.
Please tell me which questions you like or dislike. Does anyone have any suggestions on what questions to add or modify?
https://www.dharmaoverground.org/discussion/-/message_boards/message/8180069
But, I felt motivated on this again today so I came up with a ton of questions. I would like to get feedback before I make the real survey. Currently it reflects my own biases too heavily. These are just rough notes. I'm still unsure in some places which questions should be multichoice vs scales etc etc.
In the real survey I will organize these somehow, but for now they're pretty much just puked into my notes.
Please tell me which questions you like or dislike. Does anyone have any suggestions on what questions to add or modify?
Restrict to meditation communities adjacent to DhO: in particular, believe in some notion of progress. Some form of awakening is possible in our lives with purposeful practice. Orientation towards pragmatism.
Question asking what questions they would like to know
Question asking permission to include their raw data point
Community-belonging question:
todo. steal from SSC survey
Meditation Related
Community
Question asking what questions they would like to know
Question asking permission to include their raw data point
Community-belonging question:
- Do you believe that meditation practice can cause radical quasi-permanent changes to consciousness ('enlightenment', 'awakening')?
- Age
- Gender
- Political affiliation stuff
- Political opinion stuff (global warming, trump, etc)
- country
- sexual orientation
- income
- net worth
- charity
- occupation
- left/right hand, ambi
- Rate your childhood: Almost perfect, Basically normal (slightly dysfunctional), Moderately dysfunctional, Extremely dysfunctional (abuse, significant trauma, parent died, homelessness, etc)
- Have you ever been formally diagnosed with a mental illness (major depression, anxiety disorders, bipolar, schizophrenia, etc)?
- Mood scale
- Have you attended more than 3 therapy/counselling sessions in your life?
- Have you ever taken a psychedelic? Which? How many times?
- Select all: LSD, DMT, MDMA, psilocybin mushrooms, 5meo-dmt, ayahuasca, peyote/mescaline, ketamine, Iboga/Ibogaine, Nitrous oxide, Other obscure psychedelic(s) not listed here
todo. steal from SSC survey
Meditation Related
Community
- which of these podcasts do you listen to (have listened to at least two episodes and listen at least once a year)? Deconstructing Yourself, Emerge with Daniel Thorson, Imperfect Buddha, Buddhist Geeks, Waking Up with Sam Harris, 10% Happier With Dan Harris, Unfettered Mind, The Fire Kasina, Heart Wisdom with Jack Kornfield, Secular Buddhism with Noah Rasheta, Dharma Seed, The Secular Buddhist, Against the Stream
- which of these sites do you use (read or post on at least once a year)? DhO, Awakenetwork, SSC, Suttacentral, r/streamentry, r/mindilluminated, r/unifiedmindfulness, r/buddhism, r/meditation, Meaningness/Vividness, Hamilton Project, Wiser By Design/Magia, Dhamma Wheel, Dharma Wheel,
- Do any of your family members meditate (even casually)? y/n
- If you've introduced a friend or family member to meditation, was the experience positive? Scale 1-10 or I haven't recommended.
- Do you attend a local meditation meetup group (at least 3 times a year)? Yes/No and I have no interest/No but I would like to
- Have you ever used a guided meditation smartphone app? Yes/No and I have no interest/No but I would like to
- Do you use a guided meditation smartphone app regularly (at least once a week)?
- Have you operated in a meditation teacher role (in person or through videochat, not text. free or paid) for more than 10 hours in your life?
- Have you ever personally been involved with a meditation teacher who you know has abused students in some way (financially/sexually/psychologically/physically/etc)?
- Have you ever been instructed in a guided meditation at work or school? Yes/No/Not sure or dont remember
- Do you identify as 'Buddhist'? Would you say to someone "I'm a Buddhist"? Yes/No/With caveats/Other
- Do you believe in the standard interpretation of rebirth/reincarnation? y/n/not sure/it's complicated
- To what extent do you believe meditation facilitates or causes positive psychological growth or healing? 0 not at all, 5 somewhat, 10 always.
- Do you believe that it is possible to eliminate all emotions? y/n/not sure/it's complicated
- Assuming yes to the above, do you believe that it is desirable to eliminate all emotions? Not possible/y/n/not sure/it's complicated
- To what extent are meditative skill and morality correlated? 0 for not correlated whatsoever, 10 for maximally correlated.
- To what extent do you agree with this statement, "accomplished meditators have a particular responsibility to be politically active"?
- Do you believe that technological means (drugs, neurofeedback, transcranial magnetic stimulation, virtual reality, etc) will be able to radically accelerate 'awakening' within the next 10 years?
- On a scale from pessimistic to optimistic, what do you think about "mass awakening" scenarios? That is, do you believe it's likely (greater than 1% probability) that in the next 50 years 25% of the population or more will be 'awakened'. (However you define this)
- Do you believe that 'awakening' (however you define that) was completely understood by past masters, or that there is much more room for discovery? Y/n/not sure/it's complicated/i dont define 'awakening'
- Is it safe for most people (80%+ of people) to meditate intensively? Yes/No/Not sure
- If American, do you believe that the influence of Buddhism in America is increasing or decreasing?
- If American, do you believe that the influence of meditation practices in America is increasing or decreasing?
- Do you believe that all traditions of spirituality are basically talking about the same "awakening"? Yes/No/Not sure. Maybe a scale?
- Do you believe psychedelics have a place in spiritual practice?
- Do you believe that there are 'powers' (astral projection, out of body experience, near death experience, telepathy, visions, etc) experiences that interact somehow with 'objective reality' in a way not currently completely understood by 'science'? Yes/No/Not Sure/It's complicated/Disagree with question
- Do you believe that the earlier teachings of Buddhism (sutta) are more valid than latter historical teachings? Yes/No/It's complicated
- Has meditation influenced your need for sleep? 0 need much more sleep, 5 same, 10 need much less sleep
- Has meditation influenced your susceptibility to addiction (not just clinical addiction, but general addictive)? I didn't have any addictive tendencies as far as I know before so I have no basis for comparison. Not sure. Or, scale 0 addictions are worse, 5 addictions same, 10 addictive tendencies completely gone
- How has meditation influenced your ability to empathize with others? 0 worse, 5 neutral, 10 better
- How has meditation influenced your stress levels? 0 worse, 5 neutral, 10 better
- How has meditation influenced your perception of pain? 0 worse, 5 neutral, 10 better
- How has meditation affected your memory? (Worse / Same / Better / Mixed / Not sure)
- Has meditation changed your experience of sex? Basically the same / Little different / Radically different
- Has meditation changed the time it takes you to recover after becoming upset?
- To what extent has meditation improved your intellectual performance (at school or work)? Scale 1-10. 5 for neutral, 1 for it has worsened it, 10 for it has improved it.
- Have you had one or more 'awakening' events after which your experience appears to be permanently changed in a significant way? Yes/No/Not sure/I dont believe in 'awakening'/I don't know what 'awakening' means
- In your childhood did you do something you now identify as "meditation" without being instructed? Yes/No/Not sure/
- Have you ever experienced what you identify as a 'kundalini experience'?
- Have you ever experienced what you identify as 'nirodha samapatti'?
- Have you ever experienced a strong form of what you identify as a 'jhana'?
- Have you ever had a "oneness experience" — one in which it seemed like you were completely identified with the universe, 'God', or some kind of universal consciousness (even if you now disagree with that interpretation)?
- Have you had temporary (ended within a week) negative experiences (depression, anxiety, panic attacks, frightening hallucinations, anger problems, frightening derealization/dissociation, etc) that you believe were caused by or exacerbated by your meditation practice?
- Have you had permanent (occurred more or less continuously for 2 or more years) negative experiences (depression, anxiety, panic attacks, frightening hallucinations, anger problems, frightening derealization/dissociation, etc) that you believe were caused by or exacerbated by your meditation practice?
- Have you ever had a 'powers' experience (astral projection, out of body experience, near death experience, telepathy, visions, etc)? How many?
- Have you ever had a meditation experience during which you couldn't feel your body at all?
- Have you ever had a lucid dream which you could consciously control? How many?
- At what age did you first try a formal meditation technique?
- At what age did you first start meditating intensively? (Age ranges, "I don't meditate intensively")
- How many hours a week do you meditate?
- How many hours a week do you spend on practices involving visualization?
- How many hours a week do you spend on somatic/body-based practices like hatha yoga, Taichi, stretching, Reggie Ray earth breathing, bioenergetics, Reichian therapy, etc?
- Do you consider study of Buddhist texts to be a significant part of your practice?
- Are you significantly influenced by practices from traditions outside of Buddhism (have done at least 10 hours of practice in)? Choose all which apply: Taoism, Western magick, Christian mysticism, Native American rituals, shamanism, Advaita, Sufi, Actualism, others
- Do you purposefully abstain from all drugs (including alcohol) year-round? Yes/No
- Do you purposefully abstain from all drugs (including alcohol) during at least one month a year? Yes/No
- How would you rate your visualization ability on an average day? 0 complete aphantasiac, 5 some images with okay color and clarity, 10 full life-like visualization with vivid colors and details.
- How many 'dharma books' have you read? Number ranges, and 'idk what this is'
- Would you say that you have a natural affinity for concentration (attending to any part of your experience, small or large without becoming distracted)?
- Would you say that you have a natural affinity for investigating the raw sensations of your experience carefully (at fine and/or large scales)? Scale 1-10.
- Would you say that you have a natural affinity for 'heartful' practices which cultivate gratitude, lovingkindness, forgiveness, compassion, devotion, etc? Scale 1-10.
- Would you say that you have a natural affinity for "letting go" or "surrender" practices (involving releasing intentions to do or have an outcome)? Scale 1-10.
- How many (at least semiformal) meditation retreats (of at least 2 days) have you been on in your life?
- Have you attended a retreat of at least 2 days outside your home country?
- Do you regularly (at least once a month) compare your practice, intentionally or not, against a map (Progress of Insight, Tibetan bumis, Culadasa's stages, Zen oxherding)?
- If you do compare with a map, rate its ability to explain your experience of meditation? (1-10) or N/A
- Have you studied Buddhist languages in order to read Buddhist texts? Choose which apply Pali, Sanskrit, Tibetan, Chinese, ?
- Have you extensively studied Buddhism academically (Buddhist college or degree in Buddhist studies or at least 10+ academic books)?
- What major school of Buddhism do you most resonate/identify with? Theravada, Mahayana, Vajrayana, None
- Motivations for meditating? Select small number (3?): 'awakening', improve concentration, fix psychological problems, better heartful qualities, ?
- Have you ever used a sensory deprivation, sensory isolation, or "float tank"? Yes/No but I would like to/No and I have no interest/No and I don't know what that is
- Have you ever fasted for spiritual purposes?
- How many hours a week do you spend meditating while not seated (walking, standing, moving, etc)?
Ben V, modified 5 Years ago at 7/25/19 7:47 AM
Created 5 Years ago at 7/25/19 7:47 AM
RE: I need your help to come up with survey questions!
Posts: 418 Join Date: 3/3/15 Recent Posts
Interesting. I don't see a problem with the questions, as long as the confidentiality of those surveyed is respected.
Concerning the question around sex, I wouldn`'t just ask if meditation changed one's experience of sex, but how so. If you just ask the former the answer could be interpreted in all sorts of distorted way. For example, if one answers "yes". We could interpret that it means the meditator feels less craving toward sex, when in fact in may mean improved sex life due to being more present with sensations, or vice versa.
I find the questions on politics interesting as many seem to assume that if one is a Buddhist then their views will be such or such about society. The facts are probably more variable.
Concerning the question around sex, I wouldn`'t just ask if meditation changed one's experience of sex, but how so. If you just ask the former the answer could be interpreted in all sorts of distorted way. For example, if one answers "yes". We could interpret that it means the meditator feels less craving toward sex, when in fact in may mean improved sex life due to being more present with sensations, or vice versa.
I find the questions on politics interesting as many seem to assume that if one is a Buddhist then their views will be such or such about society. The facts are probably more variable.
Dada Kind, modified 5 Years ago at 7/25/19 9:28 AM
Created 5 Years ago at 7/25/19 9:28 AM
RE: I need your help to come up with survey questions!
Posts: 633 Join Date: 11/15/13 Recent Posts
Hey, thanks for the feedback!
Yes, the survey will be anonymous. The data will be published openly afterwards.
Concerning the sex question: do you have an idea on exactly how to form the question and answer possibilities? It gets tricky. I could make the question specifically about pleasure and then make a scale. But not all changes affect pleasure, even desireable ones.
Yes, I find the demographic/political questions just as interesting as the meditation ones. I honestly have little idea about the distributions on these. The stereotype is "white middle-class liberal". Maybe, maybe not! Also, most internet surveys have few female respondents. The SSC survey had only 11% female! Is that the case here? Also, given that I will be taking a lot of questions from the SSC survey, we will have a decent way to compare the communities . In addition, they paid via Mechanical Turk for workers to answer the survey (for a control group). We can reuse that control group here.
Yes, the survey will be anonymous. The data will be published openly afterwards.
Concerning the sex question: do you have an idea on exactly how to form the question and answer possibilities? It gets tricky. I could make the question specifically about pleasure and then make a scale. But not all changes affect pleasure, even desireable ones.
Yes, I find the demographic/political questions just as interesting as the meditation ones. I honestly have little idea about the distributions on these. The stereotype is "white middle-class liberal". Maybe, maybe not! Also, most internet surveys have few female respondents. The SSC survey had only 11% female! Is that the case here? Also, given that I will be taking a lot of questions from the SSC survey, we will have a decent way to compare the communities . In addition, they paid via Mechanical Turk for workers to answer the survey (for a control group). We can reuse that control group here.
Dada Kind, modified 5 Years ago at 7/25/19 3:09 PM
Created 5 Years ago at 7/25/19 3:09 PM
RE: I need your help to come up with survey questions!
Posts: 633 Join Date: 11/15/13 Recent Posts
I just edited the OP. I added some new questions, rephrased a couple things, and broke it up into categories so it's more digestible.
Jim Smith, modified 5 Years ago at 7/25/19 3:22 PM
Created 5 Years ago at 7/25/19 3:22 PM
RE: I need your help to come up with survey questions!
Posts: 1799 Join Date: 1/17/15 Recent PostsDada Kind:
I just edited the OP. I added some new questions, rephrased a couple things, and broke it up into categories so it's more digestible.
The more questions you ask the more people will quit in the middle of the survey.
Random results from a web search on "how many questions should a survey have":
https://kb.benchmarkemail.com/how-many-questions-should-a-good-survey-have/
"The length of your survey should be short enough that it takes the average user 5 minutes or less to complete. This can be achieved with about 10 questions or less, usually."
https://surveytown.com/how-many-questions-your-online-survey-should-have/
Dada Kind, modified 5 Years ago at 7/25/19 4:58 PM
Created 5 Years ago at 7/25/19 4:58 PM
RE: I need your help to come up with survey questions!
Posts: 633 Join Date: 11/15/13 Recent Posts
Hello, thank you for replying!
The SlateStarCodex from 2019 had 194 questions (if I'm counting correctly). Previous years might have had more (including stuff like personality quizzes that you had to take from a separate site). They had 8171 respondents. Looking at the results, plenty stuck with the entire thing.
r/slatestarcodex has 19,846 members. r/streamentry has 12,679 members. r/themindilluminated has 11,559 members. The Buddhist Geeks Twitter account has 71.2k followers. DhO has 2,889 members (is that right?). Stuart Charles Law's practice thread has 68,750 views. Florian's "Nada" sound thread has 48,488 views.
It's true that SSC readers might be more nerdy and interested in data. But, many "pragmatic dharma" people are nerdy too and I hope have better patience on average!
Not all of the questions above need to go in the survey. But, I would like it to be long. I will continue thinking of questions and will probably eliminate some. Which questions do you personally think are awkward/inessential/frivolous?
Thanks!
The SlateStarCodex from 2019 had 194 questions (if I'm counting correctly). Previous years might have had more (including stuff like personality quizzes that you had to take from a separate site). They had 8171 respondents. Looking at the results, plenty stuck with the entire thing.
r/slatestarcodex has 19,846 members. r/streamentry has 12,679 members. r/themindilluminated has 11,559 members. The Buddhist Geeks Twitter account has 71.2k followers. DhO has 2,889 members (is that right?). Stuart Charles Law's practice thread has 68,750 views. Florian's "Nada" sound thread has 48,488 views.
It's true that SSC readers might be more nerdy and interested in data. But, many "pragmatic dharma" people are nerdy too and I hope have better patience on average!
Not all of the questions above need to go in the survey. But, I would like it to be long. I will continue thinking of questions and will probably eliminate some. Which questions do you personally think are awkward/inessential/frivolous?
Thanks!
Edward, modified 5 Years ago at 7/26/19 1:26 AM
Created 5 Years ago at 7/26/19 1:25 AM
RE: I need your help to come up with survey questions!
Posts: 129 Join Date: 6/10/19 Recent PostsBen V, modified 5 Years ago at 7/26/19 5:21 PM
Created 5 Years ago at 7/26/19 5:21 PM
RE: I need your help to come up with survey questions!
Posts: 418 Join Date: 3/3/15 Recent Posts
Maybe just asking if their experience of sexuality changed and if they want to say a few things about in what way. It's a topic that could take various tangents. You could also ask yourself what you are looking for here, whether it's about their relation to pleasure as you say, or other things.
Dada Kind, modified 5 Years ago at 7/27/19 9:48 AM
Created 5 Years ago at 7/27/19 9:48 AM
RE: I need your help to come up with survey questions!
Posts: 633 Join Date: 11/15/13 Recent Posts
I did some reading yesterday. Here's my current take:
Even if I had a list of everyone in the 'pragmatic dharma movement', i.e. a sampling frame, and used probability-based sampling from it, it still wouldn't particularly help because I can't force people to take it. There would still be bias from -- who checks their messages, who reads their messages, who feels like taking it, who has the patience to complete it, etc.
This will be a voluntary survey or, if you will, "convenience sampling".
Methods to reduce bias in voluntary surveys like poststratification or propensity weighting aren't feasible here either. For the former, we have no reference statistics of the population. For the latter, we have no way of fitting a model predicting someone's propensity to take the survey.
So, I think being an explicitly voluntary survey would be better. There will be bias of course but at least we can maximize the number of people taking the survey. Anyway, biased data is better than no data (imo).
In my reading I did see some softer suggestions which can mitigate bias somewhat. My summary:
1) Encourage participation as much as possible. Community leaders can ask people to take it. Post the link on DhO, AN, r/streamentry, r/mindilluminated, etc.
2) Give some kind of incentive. No idea what this could be. MCTB3 if you complete it. Buddha points.
3) Ask people to pass the survey onto suitable friends, especially ones who might not otherwise see it -- don't check various sites anymore, don't have internet, don't read messages, never used sites at all, etc. This is called "virtual snowball sampling". There is research that this can be effective for reducing bias in internet surveys given in developing countries.
Does this address your concerns?
Even if I had a list of everyone in the 'pragmatic dharma movement', i.e. a sampling frame, and used probability-based sampling from it, it still wouldn't particularly help because I can't force people to take it. There would still be bias from -- who checks their messages, who reads their messages, who feels like taking it, who has the patience to complete it, etc.
This will be a voluntary survey or, if you will, "convenience sampling".
Methods to reduce bias in voluntary surveys like poststratification or propensity weighting aren't feasible here either. For the former, we have no reference statistics of the population. For the latter, we have no way of fitting a model predicting someone's propensity to take the survey.
So, I think being an explicitly voluntary survey would be better. There will be bias of course but at least we can maximize the number of people taking the survey. Anyway, biased data is better than no data (imo).
In my reading I did see some softer suggestions which can mitigate bias somewhat. My summary:
1) Encourage participation as much as possible. Community leaders can ask people to take it. Post the link on DhO, AN, r/streamentry, r/mindilluminated, etc.
2) Give some kind of incentive. No idea what this could be. MCTB3 if you complete it. Buddha points.
3) Ask people to pass the survey onto suitable friends, especially ones who might not otherwise see it -- don't check various sites anymore, don't have internet, don't read messages, never used sites at all, etc. This is called "virtual snowball sampling". There is research that this can be effective for reducing bias in internet surveys given in developing countries.
Does this address your concerns?
Dada Kind, modified 5 Years ago at 7/27/19 9:54 AM
Created 5 Years ago at 7/27/19 9:54 AM
RE: I need your help to come up with survey questions!
Posts: 633 Join Date: 11/15/13 Recent Posts
Hello,
I'm aiming for these questions to be as closed as possible. I.e., little to no free response. This makes the survey easier to complete and easier to analyze.
I feel that the number of ways meditation can influence sex are difficult to contain in a few response options. That's why I went with just changed-at-all. Even though it's broad, it's answerable and gives some information.
Thoughts?
I'm aiming for these questions to be as closed as possible. I.e., little to no free response. This makes the survey easier to complete and easier to analyze.
I feel that the number of ways meditation can influence sex are difficult to contain in a few response options. That's why I went with just changed-at-all. Even though it's broad, it's answerable and gives some information.
Thoughts?