Research Survey Tool Utilizing a Quantitative Design

Subject: Entertainment & Media
Pages: 16
Words: 4487
Reading time:
17 min
Study level: PhD

The construct that is the focus of this research is social media usage in society. It entails the use of wide-ranging Internet-based applications and services for online exchanges or the sharing of user-generated content (Hu & Zhang, 2016). Common social media services include blogs, wikis, social networking and media-sharing sites, virtual gaming, etc. with a combined user population of over 1.97 billion (Hu & Zhang, 2016). Social media sites, e.g., Facebook, Youtube, and Twitter are popular among users. For the youthful population, such platforms are fundamental to their social life. However, for the senior generation, the use of social media is often limited to searching for specific information. Business enterprises are also increasingly using Facebook and Twitter to improve their online brand presence and bolster their marketing efforts. They have integrated these platforms into their business models to develop and sell products and services to a clientele that spends most of their time online. Therefore, the extensive usage can be attributed to the desire to satisfy people’s social needs, commercial interests, and improved access to the Internet, mobile devices, etc. Further, the capacity of social media to appeal to diverse interests, including information exchange, networking, or status updates, has contributed to its rapid uptake.

Social media (SM) usage as a construct essentially captures the behaviors of users of various platforms, including the time spent on multiple sites, content sought, etc. These dimensions are essential for understanding the drivers and impact of social media on individuals, businesses, and society. Hu and Zhang (2016) define SM usage as a “behavioral process” that involves utilization of “an information technology (IT) artifact” to complete specific tasks (p. 152). Therefore, the use of SM involves an interaction between an individual and an IT component. Factors such as the purpose of the SM platform, the social context of usage, and goals of the user are essential for examining this construct. SM usage can be conceptualized as an assortment of interactions among an IT tool, the user, individual or business goal, and socio-cultural context (Hu & Zhang, 2016). Multiple features implemented in the SM platform allow one to perform tasks of interest. Therefore, a social milieu and personal preferences and behavior would influence how people choose the IT functions of a social media platform. Different interactions among IT components and behavioral dimensions establish SM usage as a broad construct. As such, it should be understood as a multidimensional concept, as it involves multiple aspects – behaviors and IT functions.

Given the popularity of social media, it is crucial to examine SM usage trends in the society. It will shed light on behavioral dimensions, such as the time spent on SM, its impact on behavior, connections, information, the nature of relationships, etc. There is no doubt that the emergence of social networking sites, such as Facebook, YouTube, Twitter, Tumblr, and Instagram, has led to the evolution of a new mechanism of socialization that differs from traditional offline interactions. The precise dimensions of usage are varied. According to Jan, Soomro, and Ahmad (2017), social media has enabled people to create relationships, share information, connect with people globally, improve their behaviors, and achieve healthier social lives. Thus, the study of the SM usage would help paint a picture of the nature of relationships, information, connections, behaviors, and experiences of people across age groups. In addition, it would show generational differences in SM utilization and the preferred media by the youth and adults. Towner and Munoz (2016) observe that Baby Boomers are dependent on traditional media, while Millennials are reliant on online media as a source of information, which shapes their knowledge, perceptions, and attitudes. Hence, while undertaking a particular study, researchers ought to understand demographic preferences of the target population so that they can use appropriate media of communication and data collection.

The study of SM usage and its dimensions is critical to understanding how people communicate and interact on social networking sites. Failure to do so would have some consequences. The conceptualization of this construct would be unidimensional, leaving out context-specific dimensions and the interactive component of IT tools. In addition, the explanatory power of this concept would be diminished, as the relationship between SM usage and user knowledge, perceptions, and attitudes would be less apparent (Hu & Zhang, 2016). Its positive and negative impacts on social, emotional, or communication aspects, especially for the younger generation, have been determined through research of this construct. For instance, the harmful use of SM is attributed to excessive usage that affects work/academic outcomes and causes addiction (Hu & Zhang, 2016). In particular, examining the time users spend on SM platforms would help quantify behavioral dimensions, such as preferred sites, individual tasks by users, etc. A quantitative research design would give insights into the multidimensionality of SM usage and help measure the time used on SM sites. Without such a study, it would be difficult to quantify SM use across ages, generational preferences, and usage trends, and effects and value to users.

Method

Participants

In developing the survey instrument, responding participants would be drawn from users of social media networks, such as Facebook, Twitter, YouTube, Instagram, and Tumblr. The demographic characteristics of the respondents would include 20-50 years age bracket, computer literacy, and the regular use of diverse social media networks for at least a year. The critical considerations made in selecting the participants were the literacy level, accessible population, age, gender, and rural and urban population.

Only the passive recruitment method would be used in sampling the respondents. This approach differs from active recruitment concerning the researcher’s role in the sampling process. While in passive recruitment no proactive participation in enlisting potential participants is required, in the active method the investigator contacts and requests respondents to take part in the study. Since the study would involve an online questionnaire instrument (SurveyMonkey), the participant selection process would be passive. Using the university database of staff and students who are social media users as a sampling frame would ensure that a representative sample is selected from the total population.

Instrument

Questions in the four-item questionnaire focused on the number of hours a user spends on SM sites. This variable was found to be a suitable metric as it quantifies SM usage on a daily, weekly, or even monthly basis. The items for the survey instrument were developed from the review of literature that shows that time influences the duration and frequency of social media usage. Further, generational differences exist in SM use with younger users being active during weekdays, evenings, and mornings, whereas older ones visit these sites in the morning and evening of any day of the week (Quinn, Chen, Mulvenna, & Bond, 2016). Therefore, the number of hours per day, week, or month that a person spends on SM is a valid and reliable variable for measuring social media usage.

The instrument had four items rated on a scale of 1-8. Responding users had to indicate how often one logs into the preferred SM site to determine the frequency of SM usage. The measurement of this item involved a Likert scale where the respondent indicates his/her preference from eight categories ranging from hourly to monthly. The second item measured the amount of time spent on social media after logging into the preferred site. In this case, measurement involved an interval scale (in minutes). Items 3 and 4 ask about the number of hours spent on SM daily and weekly, respectively.

Procedure

The list of four items included in the survey instrument was developed through two approaches. First, a review of the literature on survey tools (SurveyMonkey) and SM usage behavior across different demographics was conducted. It was established that web-based survey methods are better than a traditional paper-based questionnaire because of the reduced transmission period, advanced customization features, lower delivery costs, and shorter data entry time (Saleh & Bista, 2017). Further, the time spent on social media was found to be a suitable variable for measuring SM usage, i.e., frequency and behavior as well as differences between SM users of different ages (Quinn et al., 2016).

Demographic differences exist in the consumption of social media. For instance, there is a strong correlation between age and SM usage. According to Perrin (2015), presently, 65% of American adults (>65 years) use these platforms, up from 7% a decade ago. They use social media for communication, politics, health, dating, etc. In contrast, 90% of young adults (18-29 years) are regular SM users. From the review of survey tools and relevant studies, it is clear that ‘time spent’ daily, weekly, or monthly would be an appropriate measure of social media usage. This metric includes the duration a user spends logged into the site and the frequency of SM consumption.

In the second approach, the researcher contacted 20 individuals active on social media and requested their help in this research project. These SM users responded to exploratory questions about the average amount of time they spend on their preferred SM accounts. Subsequently, from the two approaches, a list of items was developed to measure SM consumption. The screening of the measures helped to reduce redundancy and served as a quality control measure. The results of the pilot review indicated that respondents preferred a Likert-type scale that does not involve a personal comment section. Further, the items were subjected to a second pilot review including people outside the sampling frame, i.e., those who do not use social media. From their critical views, the researcher was able to revise the instrument to achieve content validity and provide it in an easy-to-use format.

Respondents were asked on a scale of 1-8 to rate how often they log into their SM accounts, the average time spent online after logging-in, and the number of hours they spend daily and weekly. Common Likert scales (categorical) were not used, as they were considered subjective. According to Olufadi (2016), a response of ‘sometimes’ may involve a higher or lower frequency than stated. Further, such an ordinal scale may not be suitable for quantitative analyses.

Steps

Reliability and Validity

Instrument reliability after factor analyses will involve the test/retest procedure to assess its consistency over time. According to Olufadi (2016), the test-retest approach measures the stability of questionnaire items at two different times using a correlation coefficient. A similarity in repeat test scores will indicate that the metric is stable, and therefore, reliable. First, respondents meeting the inclusion criteria – aged 20-50 years, computer literate, and regular SM users – will be invited to take part in a test-retest study through a web-based survey. The pilot sample will be 10 participants drawn from the student and teaching population of a college or university. Respondents will also include those who do not use social media. Second, they will retake the same survey after two weeks. Codes will be used to match initial and subsequent responses before determining the reliability of the items using the coefficient of stability. The scores in time 1 and 2 will not be different from each other because the variable (time spent) is unlikely to change over a two-week period.

Content and face validity will be used when validating the draft instrument. Olufadi (2016) defines validity as the “amount of systematic or built-in error” in a questionnaire (p. 457). Content validity will ensure that the items capture the full extent of the construct (social media) being studied, which is social media usage. The instrument should represent the frequency of SM consumption daily, weekly, and monthly across different age groups. In the first step, three reviewers (i.e., faculty staff from various institutions that are familiar with the construct) will be consulted to examine the four items. They will rate them on a dichotomous scale as either favorable or unfavorable.

Secondly, they will be supplied with the study’s conceptual framework to help them assess the relevance of the measures. Third, items with a lower content validity index (CVI) score (<.80) will be reviewed, while those with a CVI value of >.8 will be retained as they are (Mohamad, Sulaiman, Sern, & Salleh, 2015). After revising the instrument based on CVI results and reviewers’ comments, face validity will be done through a pilot study involving a sample of 20 SM users. This construct entails determining if the items (time spent) on the instrument represent a valid metric of social media usage. The respondents will indicate if the measures reflect any of the domains illustrated in the conceptual framework. Further, they will rate how well they understand each metric with regard to grammar, clarity, relevance, and presentation. From pilot results, the researcher will determine if the instrument has face validity.

Avoiding Bias

In research, prejudice consideration of instrument items can affect the reliability and validity of the results. Researcher bias may occur at any stage of the study, including planning, sampling, data collection, and analysis. It describes the systematic error inherent in the design due to a deliberate choice or outcome (Mohamad et al., 2015). It can be avoided through a proper research process from sampling to publication. For this study, researcher bias will be controlled through three steps. First, data collection will involve a web-based survey tool (SurveyMonkey) to minimize direct contact with the respondents. This approach will reduce the risk of bias related to acquiescence or social desirability (Dodou & de Winter, 2014). Second, closed questions with validated scales will be used to establish the reliability and validity of the quantitative results. The study will not include open-ended questions, which are subject to a different interpretation of the qualitative data. Third, in the research design, selection bias will be reduced through random sampling from a known sample frame of social media users.

Ethical Considerations

Since the study involves human subjects, the protection of participant rights will be required. The first step is ensuring that participation is voluntary. Social media users meeting the inclusion will voluntarily take part in the study without coercion or inducements. A letter inviting them to participate in a web-based survey will be included in the instrument. It will reveal to the participants their rights, including the freedom to withdraw, confidentiality of participating SM users, and possible risks and benefits (Mohamad et al., 2015). The study will not draw its sample from vulnerable populations. Therefore, the participants will not be exposed to more harm than they would go through in their day-to-day activities. Further, they will receive no personal benefit or reward for participation; however, they will be informed that their responses will enrich our understanding of the behavioral dimensions associated with SM usage in society.

Further, the instrument will not capture the respondents’ names or locations during the survey. Questions on demographic details will focus on age and the computer literacy level. The researcher will use a checklist containing random codes to identify the recipients of the survey instrument and facilitate follow-up. Sensitivity to linguistic and socio-cultural dissimilarities is another ethical consideration in this research (Mohamad et al., 2015). Although the questionnaire will be in English, communication with participants will be precise and respectful of culture, gender, and ethnic background. IRB approval will be sought prior to data collection.

Issues

Before embarking on data collection, the researcher will address a range of survey administration issues related to time and resources. The research will involve a self-administered web-based survey. One of the pertinent problems is the fashion of self-administration, i.e., through e-mail or web server. A mix-mode approach will be appropriate to ensure a higher response rate. However, the researcher will have to send a pre-notification to the respondents via e-mail to allow them to select their desired mode. A hyperlink in the e-mail will direct users to the web-based survey (SurveyMonkey). This page will contain a letter introducing a respondent to the study, survey details, a researcher’s statement on confidentiality, and the process of completing the questionnaire.

A second issue relates to the accessibility of the respondents. The study will use a random sampling method and passive recruitment to select participants meeting the inclusion criteria. As stated, the target population includes students and teaching staff in a university between the ages of 20 and 50. Contacting these respondents would require access to their emails from the institutional database. The researcher will have to request the institution to inform the staff and students of the research through an e-mail announcement. The information will also be displayed on the notice board. Without accessing the target population, it will be hard to avoid sampling pitfalls due to a lack of address frame and invite respondents to participate in the study.

Another potential survey administration issue is the follow-up process. A higher non-response rate may introduce bias due to unrepresentative data collected after a large number of respondents did not answer the questions. This error will also affect the descriptive analysis and reduce the generalizations that can be made from the sample (Mohamad et al., 2015). Non-response is also ultimately costly, as the researcher will have to strive to improve response rates while exercising restraint and respecting the participant’s right to withdrawal from the study at any time. Therefore, minimizing non-response is critical for this study.

The SurveyMonkey’s List Management feature will be used to monitor non-respondents who received the announcement but have not indicated that they will participate in the project. These details will be stored in a data field to help send e-mail reminders. It is highly likely that the target population will be willing to take part in this study and give responses to questions about the amount of time spent on social media sites. However, the researcher must provide an assurance of anonymity to the respondents. Therefore, to overcome this challenge and improve participation, unique identification codes will be linked to each e-mail address in a respondent inventory. The aim is to maintain confidentiality throughout the follow-up process.

Another issue is the respondent burden that stems from the perception that the survey is complicated or time-consuming. Wordy and tedious questions could lead to fatigue that will affect the efficacy of a study (O’Neil & Schutt, 2013). The respondent burden also leads to non-response and low-quality data. The researcher will overcome this problem by providing questions in an easy-to-answer format. A pretest will be done to establish how long it would take to do the survey. On average, completing the four items takes less than a minute. The letter of introduction will contain this information along with response options included in the instrument to encourage participation. A thank you note at the beginning of the survey will also motivate respondents to take part in the study.

Results

Types of Analysis

The study would involve a descriptive correlational design. Independent variables (time spent) would not be manipulated; instead, they would be used to describe the attributes of social media usage in the target population. The correlational component of the design would be used to evaluate relationships between these variables (O’Neil & Schutt, 2013). In other words, it would explain SM usage behaviors without the effect of experimental manipulation. First, the researcher would review returned web-based surveys for mistakes and identify them with participants’ codes. The aim is to ascertain the completeness of data and check for errors before doing the analysis. The SurveyMonkey tool would support the fast validation of responses and accuracy tests. Initial data handling would involve recording the data in an Excel spreadsheet. Subsequently, a research assistant would enter downloaded dataset into the SPSS software for analysis.

The analysis of the quantitative data from the four-item instrument would involve testing scale reliability and descriptive statistics to illustrate social media usage in different age groups. Internal consistency between the four items would be evaluated using Cronbach’s alpha (O’Neil & Schutt, 2013). The comparisons would involve responses about the frequency of logging into social media and hours spent on a preferred site after accessing it. Also, compared would be the amount of time the respondents’ use daily and weekly on SM and usage behavior between age groups (young versus older adults). A quantitative nonparametric analysis would be done in SPSS using frequency, descriptive statistics, and correlational coefficients. Correlations (Spearman’s rank correlation coefficient) would indicate bivariate relationships (O’Neil & Schutt, 2013). In addition, the Fisher r-to-z transformation would be used to determine how significantly different the correlations are from each other (O’Neil & Schutt, 2013). One-way ANOVA would be performed to compare age and login frequency, the time spent logged in, and daily and weekly hours on SM sites. The confidence interval for these analyses would be set at p<0.05.

The Likely Results

When comparing SM usage between youth (20-29) and adults (30-50), the hypothesized results of this study would show a significant difference in login frequency and the time spent on social media (p<0.05). It is likely that the 20-29 years age group will spend more hours in SM sites than the 30-50 years cohort. The rationale for this difference is that Millennials rely on online sites for information and communication, while Baby Boomers prefer traditional media (Towner & Munoz, 2016). A majority of younger generations is on social media; hence, this group is likely to spend more time logged in than adults do. Therefore, usage behaviors are likely to be significantly different between the two demographic groups.

The ANOVA results for SM usage regarding how often users logged into their preferred SM site will show a significant difference between younger and older adults. Login frequency (monthly, fortnightly, weekly, etc.) would be higher in the 20-29 years group than in the 30-50 years cohort. The results would show that the frequency of logging into SM sites decreases with age. In addition, younger users would spend more hours on a preferred social media platform after each login than older ones would. The ANOVA results would show that there is a significant difference in these two cohorts regarding the time spent on SM sites. Younger users are likely to spend 121-150 minutes online after logging in, while older ones would use 10-20 minutes in each session.

The results for daily hours spent on SM sites would also be significantly different between the 20-29 and 30-50 age groups. Younger users have most of their friends on social media, and thus, they are active users (Olufadi, 2016). It is hypothesized that they will spend an average of 8-10 hours daily on their preferred SM sites chatting, sharing and seeking information and photos. On the other hand, users aged 30-50 years are likely to use less than one hour daily on social media. They are reliant on traditional media, including texting, phone calls, etc., for communication. The ANOVA results for hours spent on SM weekly would also be significantly different between the two demographic groups. Based on item 4 of the instrument, the younger users are likely to utilize more time (21-30 hrs) on their preferred social media site than older ones (2-5 hrs).

Bivariate correlations would show the relationship between the SM usage level and the time spent on SM sites. A positive correlation would be found between social media consumption and the frequency of logging into the preferred networking sites. Active users reporting that they log in more often would have the highest Spearman’s rho score. Similarly, a positive correlation would exist between usage and the time spent online after logging into SM. In this case, the rho score would be highest for those who browse longer per session. The daily and weekly hours spent on social media would positively correlate with SM consumption. Users indicating that they spend less than 1 hour and below 2 hours daily and weekly (most probably those in the 30-50 age bracket), respectively, would have the lowest rho value.

Discussion

Properties of the Instrument

The proposed instrument comprises four items that measure the SM usage level of respondents. Its principal properties include reliable and validated scales, short-format, and straightfoward questions. The researcher developed it through a literature review on the measurement of social media consumption. The items included in this instrument allowed the investigator to construct scales to measure the variables of SM usage. They have the desirable properties of reliability and validity. The stability of the scales was ascertained through the test/retest method on a small sample (n=10) of users of social media. In this view, the reliability of the instrument must be interpreted carefully. It is unclear if the scales would produce the same coefficient of stability value when used with different samples. Overall, the questionnaire has excellent test-retest reliability. The self-administered questionnaire is also short and easy to complete. Further, the questions are readable, clear, precise, and well-worded. On taking the survey, the researcher discovered that due to the accessible format and fewer items, a respondent would take less than a minute to complete the survey.

Revisions or Improvements

The reliability and validity of the instrument could be improved by using diverse samples, i.e., SM users from different demographic characteristics and locations. A replication of pilot reviews would also support the testing of the scales. This approach would enhance the external validity of the instrument due to the large and diverse samples on which the questionnaire is tested. Further, a sample that cuts across demographic attributes larger than the one used in the current study (n=10) would give more reliable results. It should include users from multiple regions of the country to enhance representativeness and generalization. Correlations between the scales would help refine them based on the computed differences in internal consistency and coefficients of determination. This investigation was not performed in the present study.

Factor analysis may be required to review and improve this instrument. Additional items could be included to measure other independent variables not quantified in this research. In this study, the time spent on social media was operationalized through login frequency, the duration a user spends online after logging in, and hours used on SM sites daily and weekly. This definition could be expanded to include a self-reported diary in addition to the four indices. This approach would ensure that respondents do not overestimate or underestimate the time spent on SM.

Recommendations for Further Study

This exploratory research has given insight into the time spent on social media across different age groups. The instrument appears to be a promising tool for measuring social media consumption levels. The multidimensionality of this construct (time used) as a measure of SM usage is reflected in the four items and related scales embedded in the instrument. Nevertheless, further research should explore the statistical aspects of the proposed scales in other locations or institutions to bolster its reliability. In addition, the items developed in this instrument were drawn from empirical studies published in English. Future research should consider literature written in other languages on the time spent on social media to enrich the development of the scales. Further, narrowing research down to a specific SM platform, e.g., Facebook or Twitter, would give results that are more informative. Such an approach would also lead to an accurate estimation of the time utilized for a particular SM site.

References

Dodou, D., & de Winter, J. C. F. (2014). Social desirability is the same in offline, online and paper surveys: A meta-analysis. Computers in Human Behavior, 36, 487–495. Web.

Hu, T., & Zhang, P. (2016). Social media usage as a formative construct: Conceptualization, validation and implication. Journal of Information Technology Management, 27(4), 151-168. Web.

Jan, M., Soomro, A., & Ahmad, N. (2017). Impact of social media on self-esteem. European Scientific Journal, 13(23), 329-341. Web.

Mohamad, M. M., Sulaiman, N. L., Sern, L. C., & Salleh, K. M. (2015). Measuring the validity and reliability of research instruments. Procedia-Social and Behavioral Sciences, 204, 164-171. Web.

Olufadi, Y. (2016). Social networking time use scale (SONTUS): A new instrument for measuring the time spent on the social networking sites. Telematics and Informatics, 33, 452–471. Web.

O’Neil, C., & Schutt, R. (2013). Doing data science: Straight talk from the frontline. Newton, MA: O’Reilly Media.

Perrin, A. (2015). Social media usage: 2005-2015. Washington, DC: Pew Research Center.

Quinn, D., Chen, L., Mulvenna, M. D., & Bond, R. (2016). Exploring the relationship between online social network site usage and the impact on quality of life for older and younger users: An interaction analysis. Journal of Medical Internet Research, 18(9), 1-16. Web.

Saleh, A., & Bista, K. (2017). Examining factors impacting online survey response rates in educational research: Perceptions of graduate students. Journal of MultiDisciplinary Evaluation, 13(29), 63-75. Web.

Towner, T., & Munoz, C. (2016). Boomers versus Millennials: Online media influence on media performance and candidate evaluations. Social Sciences, 5(56), 1-20. Web.

Appendix

Actual Measure

In scoring the instrument, four measures are used. The item scores are added together to give an aggregate value indicating social media usage level.

Measure 1: Login frequency

Frequency Monthly Fortnightly Weekly Daily Twice a day Thrice a day 2-5 hours Hourly
Scale 1 2 3 4 5 6 7 8

Measure 2: Hours spent logged in

Minutes <10 10-20 21-30 31-60 61-90 91-120 121-150 >150
Scale 1 2 3 4 5 6 7 8

Measure 3: Hours spent on social media daily

Hours Less than 1 2-3 4-6 8-10 11-12 12-14 14-16 More than 16
Scale 1 2 3 4 5 6 7 8

Measure 4: Hours spent on social media weekly

Hours <2 2-5 6-10 11-15 26-20 21-30 31-40 >40
Scale 1 2 3 4 5 6 7 8

Scholarly Rationale

Assignment 8 has not been graded yet, and therefore, there was no feedback that I could incorporate into the current paper.