Processing math: 100%
Kanozia, R., Arya, R., Singh, S., Ganghariya, G., & Narula, S. (2021). A Study on Fake News Subject Matter, Presentation Elements, Tools of Detection, and Social Media Platforms in India. Asian Journal for Public Opinion Research, 9(1), 48–82. https://doi.org/10.15206/ajpor.2021.9.1.48
Download all (6)
  • Figure 1: Subject Matter of Fake News
  • Figure 2: Presentation Elements of Fake News
  • Figure 3: Debunking Tool(s)/Technique(s) Used
  • Figure 4: Social Media Site on Which Fake News Was Shared
  • Table 1: Coding for Table 2
  • Table 2: Characteristics of Fake News Stories

Abstract

This research article attempts to understand the current situation of fake news on social media in India. The study focused on four characteristics of fake news based on four research questions: subject matter, presentation elements of fake news, debunking tool(s) or technique(s) used, and the social media site on which the fake news story was shared. A systematic sampling method was used to select a sample of 90 debunked fake news stories from two Indian fact-checking websites, Alt News and Factly, from December 2019 to February 2020. A content analysis of the four characteristics of fake news stories was carefully analyzed, classified, coded, and presented. The results show that most of the fake news stories were related to politics in India. The majority of the fake news was shared via a video with text in which narrative was changed to mislead users. For the largest number of debunked fake news stories, information from official or primary sources, such as reports, data, statements, announcements, or updates were used to debunk false claims.

Background

Although fake news has existed since the dawn of journalism, the problem has been exacerbated now and has become a major issue for most countries around the world. According to researchers, with the advancements in technology and the Internet, news travels rapidly across the world (Gruhl et al., 2004). Social media sites have now become a major source for news consumption (Nielsen & Schrøder, 2014). However, one of the major problems of using social media for news consumption is the rapid proliferation and dissemination of fake news with just a single click (Allcott & Gentzkow, 2017). It has been found in a study that fake news travels faster than actual news items (Vosoughi et al., 2018). Hence, it has become hard to detect and curb fake news, but several organizations, independent fact-checkers, and media houses aim to debunk fake hoaxes and make people aware of the phenomenon of fake news (Apuke & Omar, 2020).

People started using social networking sites for news consumption due to its low cost, easy access, and rapid dissemination of information. However, social media has now become a double-edged sword. Despite having its advantages, it is responsible for the dissemination of fake news. Individuals and society are experiencing many negative effects of this proliferation of false information (Shu et al., 2017). According to Paris and Donovan (2019), advancements in artificial intelligence technologies have also made the fake news crisis direr. The authors observe that with various social media platforms, it has become difficult to curb the dissemination of or fact-check the manipulated AV content.

According to statista.com, India is the world’s second largest digitally populated country with 688 million active users as of January 2020 out of which around 400 million are active social media users (Keelery, 2020). Facebook is the most used social media platform in the country with around 290 million users as of July 2020, making India the leading country in the world for the highest number of users on the platform (Clement, 2020a). It is followed by YouTube with around 265 million active users in India in 2019 (Laghate, 2019). Twitter has over 13 million users in India as of April 2020 (Tankovska, 2021). WhatsApp, the Facebook-owned social media platform, has penetrated India to such an extent that the country has become the biggest market for the platform in the world with 400 million active users in 2019 (Banerjee, 2020). The rampant usage of social media platforms by such a big population is one of the major factors for the proliferation of fake news in the country.

In India, there have been many instances when rumors on social media, especially on WhatsApp, led to dire consequences, even loss of lives (Farooq, 2018). In July 2018, two men were injured and one was killed by a mob in the Indian state of Karnataka that thought the men were child kidnappers, because such rumors were being spread on social media platforms during that time (Samuels, 2020). Such incidents led to 24 deaths in 2018 owing to misinformation on child kidnappings and cow slaughter spread on social media (Mishra, 2019). A substantial rise in the spread of fake news on social media was also seen in 2019 related to general elections, the Pulwama attack on the security personnel in Jammu and Kashmir state of India, and the abrogation of Article 370 in the same state, Citizenship Amendment Bill (CAB) and Citizenship Amendment Act (CAA), etc. (Chaturvedi, 2019). A BBC report also revealed that fake news on WhatsApp is proving deadly to the country (PTI, 2018). The present study aims to understand the current scenario of fake news on social media in India.

Apart from the creators of fake news, users also play a major role in the dissemination of fake news (Vosoughi et al., 2018). One of the major effects of fake news is it has worsened the psychological phenomenon of confirmation bias in a negative way and led to online collectives among users. In addition, fake news fed into online echo chambers and self-reinforcing algorithmic and cognitive systems has further helped misinformation circulation and led to the formation of filter bubbles (Bakir & McStay, 2018).

Nielsen and Graves (2017) attempt to uncover the audience’s perspective of fake news and argue that people cannot clearly differentiate between fake and real news but see both of them differing by varying degrees; the difference is not absolute but gradual. The authors further argue that the fake news discussion has taken its roots due to two structural shifts – the public’s rising distrust in news media, politicians, and other civic institutions; and the shift from traditional media to digital, mobile, and social media. Their study finds that people perceive fake news as: satire, poor journalism, propaganda, misleading advertising, and false news. Wagner and Boczkowski (2019), during their study on the reception of perceived fake news and misinformation, found that people have become more attentive to the media they are consuming because the majority have the perception that the quality of news reporting has deteriorated. Although people use social media for news consumption, they particularly distrust these platforms.

There are third party fact-checkers that include many fact-checking and debunking organizations such as politifact.com, snopes.com, etc. These debunking websites have the primary goal to test the facts of a news article and debunk it if it is false. These sites depend on manual verification for which journalists, researchers, users, and media people use many tools like Hoax, Twitter Trails, etc. (Akers et al., 2018). In India, some fact-checking initiatives have also emerged over the last few years such as factchecker.in, Alt News, Factly, SMHoaxSlayer, BoomLive, etc. Although these fact-checking websites are helpful to some extent, the “illusory truth effect” also increases over time with repetition and increasing exposure to fake news on the Internet (Pennycook et al., 2018). The study observed that perceived accuracy increased with a single prior exposure to both kinds of news – fake and real news headlines on social media platforms - despite the fake ones being labeled as “Disputed” by fact-checkers or being inconsistent with the participant’s political ideology. Although the warning did have a negative impact on the perceived accuracy of the fake news, it was canceled out due to the positive effect of repetition.

It has been observed that users play a major role in the proliferation of fake news. Hence, apart from technological initiatives to curb the circulation of fake news, media literacy initiatives are required (Bulger & Davison, 2018).

The concept of fake news has developed over the years and has its roots far back in history to 44 B.C. when Octavian started a propaganda campaign in Rome against Antony with slogans etched onto coins to smear Antony’s reputation. Wars and revolutions over the years have seen many examples of powerful propaganda and fake news circulation, such as during World War I from 1914-1918, the Russian Revolution in 1917, and the infamous Nazi propaganda in Germany during and the years leading up to World War II (Posetti & Matthews, 2018).

As information and communication technology has advanced over the years, incidents resulting from fake news have become more prominent. The US presidential elections of 2016 became a hot bed for fake news when teenagers in Macedonia created fake websites and circulated completely fabricated stories supporting Donald Trump. While campaigning for the elections, Trump used the term “fake news” very often, commenting on journalists and news organizations (Allcott & Gentzkow, 2017). The term became so popular that it was named the “Word of the Year” for 2016 by Macquarie Dictionary (Lavoipierre, 2017) and in 2017, Collins Dictionary rewarded the term with the same tag (Hunt, 2017).

Literature Review

The incidents discussed above have led to many research studies trying to define the concept of fake news in different ways. Egelhofer and Lecheler (2019) give two dimensions of fake news: the fake news genre and the fake news label. The former refers to the deliberate creation and dissemination of false journalistic disinformation while the latter to the labeling of news as fake or delegitimizing of news media by political parties if it contradicts their political agendas. Farkas and Schou (2018) discuss fake news as a “floating signifier” used in different ways by political actors within diverging discourses to tell people what is “right” and what is “fake.” Wardle (2017) has given the simplest concept of fake news by dividing it into two categories – disinformation and misinformation – where the former is deliberately created and shared despite the knowledge that the information is false, while the latter is shared unintentionally without the awareness of its falseness. However, fake news is not such a simple term, as discussed by Tandoc et al. (2018) who have given six definitions or types of fake news: news satire, parody, fabrication, manipulation, advertising, and propaganda. These types differ in having varying levels of facticity. Furthermore, Morgan and Hoax (2019) have found that the motivations behind misleading news are found to be passion, to earn money, for politics or to gain power, to make fun of something or someone, or to (mis)inform an audience. Their study further argues that the term “fake news” is a misleading simplification in itself.

The first step to curb the fake news crisis is to identify or detect such news stories. Detection of fake news has become important. It aims to identify any false information effectively by checking the truthfulness of a viral message and what crux and context have been communicated in a news article (Shu et al., 2020). Sharma et al., (2019) divided fake news identification methods into three types: content-based, feedback-based, and intervention based. The first type is based on the content of the information that needs to be verified, the second on the user responses the news article receives on social media, and lastly, the third type provides computational solutions for fake news detection. After a particular piece of fake news has been identified, it needs to be debunked. Detailed debunking messages correlate positively with the debunking effect and creating conditions by engaging audiences in scrutiny, whereas counter-arguing of misinformation also helps in the acceptance of debunking messages (Chan et al., 2017). In many studies, it has been found that the process of active inoculation induced by a fake news game reduced the perceived reliability and persuasiveness of fake news articles among the audience. Therefore, educating people through games can inoculate them against fake news (Roozenbeek & Van Der Linden, 2019). Although attaching warnings to news stories that have been disputed by third-party fact-checkers helps, Pennycook et al. (2020) suggest that attaching “verified” tags to stories which have been approved by fact-checkers, as well as warnings to disputed stories, might improve the effectiveness of the warnings. There are third party fact-checkers that include many fact-checking and debunking organizations across the world (Akers et al., 2018) that use many tools. This process is time-consuming and risky in today’s world where information spreads quickly. Thus, it has become very important to come up with new technologies to efficiently and quickly debunk fake news (Popat et al., 2018).

Recently, substantial research has been done on new technological tools and models that can detect fake news. Natural language processing and machine learning techniques are helping us to speed the process of fake news detection. Artificial intelligence is proving to be a promising solution to this problem. For example, Popat et al. (2018) propose a neural network model that works by taking into account the credibility of the sources, the article’s language, and external evidence articles. According to Qayyum, et al. (2019), using a block chain-based framework is the best solution to tackle fake news. New datasets have also been developed to keep an account of articles that have been individually labeled for veracity by experts. In their research article, Torabi Asr and Taboada (2019) have proposed a new dataset named MisInfoText as a resource for text classification efforts. Guha (2017) has developed a new service called “Related Fact Checks,” which helps in checking the veracity of an article by bringing up fact-checks that are relevant to the article. Deep learning models are also being developed for big data.

Objective and Research Questions

The present research aims to find out different characteristics of fake news circulating on social media. The study mainly has four research questions:

RQ1. What subject matter areas are discussed by fake news reports of the two Indian fact-checking websites?

RQ2. What presentation elements (text, still photographs, videos, other images, or a combination of these) comprise these fake news reports?

RQ3. What tools and techniques were used by fact-checkers to detect and debunk fake news?

RQ4. In which social media outlets (Facebook, Twitter, WhatsApp, and YouTube) were fake news circulated more frequently?

Research Methodology

For the present study, the research was carried out with quantitative content analysis. This research method helps to systematically categorize and record features of textual, visual, or aural content for further analysis (Krippendorff, 2018). With this technique, the study systematically analyzed, categorized, and recorded the characteristics of debunked fake news stories published on the fact-checking websites. The data obtained from the analysis of all the stories was then coded and quantified to answer the four research questions and reach to the conclusion of the study.

Sampling Frame

The present study selected debunked fake news stories from two fact-checking websites: Alt News and Factly. These websites were selected because they are the leading fact-checking websites in India. They are the pioneers of fact-checking in India. Alt News was founded by a software engineer and activist, Pratik Sinha, in February 2017. Alt News is a not-for-profit initiative that aims at debunking all types of fake news from social as well as mainstream media. It is an independent initiative working on donations and grants and covers important news stories that are neglected by the mainstream media. Alt News has also been recognized internationally at Google News Lab Asia-Pacific Summit 2017 in Singapore, where Alt News’s founder was invited to discuss solutions for combating fake news. The second fact-checking website, Factly, also checks the veracity of news stories on social and mainstream media by using data from credible and official sources or with the help of various tools, including the Right to Information Act of 2005 (RTI). It is a for-profit public information portal founded by Rakesh Dubbudu in 2015. The International Fact-Checking Network (IFCN) certified the website in November 2018 (Patil, 2018).

Both of the fact-checking sites were started by technocrats, not by media houses (Patil, 2018). Certainly, media houses are more likely to be under the influence of certain schools of thought, ideologies, political parties, and advertisers as compared to technocrats. The revenue model of these two fact-checking websites is crowd funding, which further reduces the chances of bias. Both of the websites are IFCN certified and have a high number of people following their social media accounts. As of June 2020, Alt News had over 165K likes on Facebook, 265K followers on Twitter, and over 120K subscribers on YouTube. Factly had over 120K likes on Facebook, around 3K followers on Twitter, and 35K subscribers on YouTube. Both of the websites also have WhatsApp contact numbers through which people can contact them for sharing social media stories to fact-check. Both of them were also a part of the Google News Initiative launched in India in 2018 to train journalists in fact-checking.

The time period sampled for the study was three months, from December 2019 to February 2020, i.e., 90 days. The reason behind choosing this time period was the protests against the Citizenship Amendment Bill (CAB) and Citizenship Amendment Act (CAA), which erupted across India after the bill was made into an act. The bill was passed in both the houses of the Indian Parliament, Lok Sabha and Rajya Sabha, on December 9, 2019 and December 11, 2019, respectively. The bill was finally turned into an act on December 12, 2019 with Indian President Ram Nath Kovind’s approval (Latief, 2019). The selected three months, from December to February, were a crucial time period when the country saw a number of protests in various parts related to the CAB, CAA, and National Registry of Citizens (NRC). The CAB or CAA allowed illegal migrants from Pakistan, Afghanistan, and Bangladesh belonging to six religious minority communities, Hindu, Christian, Jain, Parsi, Sikh, and Buddhist communities, to apply for citizenship if they had begun living in India before December 31, 2014. The act excluded illegal migrants belonging to the Muslim community, the largest minority group in India, and other refugees belonging to other countries or communities which were not mentioned in the act. This led to mass protests across the country as people thought the act to be discriminating on the basis of religion (Rajalakshmi, 2020). Similarly, the NRC, which was updated in August 2019 only in the Northeastern Indian state of Assam to identify illegal migrants from neighboring countries, was potentially being implemented countrywide (National Population Register, NPR), which would lead to the detention and deportation of those who would be unable to prove their Indian citizenship (India Today Web Desk, 2019). These issues led to massive protests along with an upsurge in fake news on social media (Goel, 2019) in the three months chosen for the present study.

Sampling Design and Size

For the present study, a systematic random probability sampling method was used to select the sample. With this sampling method, the first unit was randomly selected from the sampling frame and then the rest of the units were selected by drawing calculations based on constant interval size (Lavrakas, 2008). The sampling frame for the study constituted a total of 180 news stories, one story from each day from each website, i.e., 90 stories from Alt News and 90 from Factly in 90 days. With the help of systematic random probability sampling method, the desired sample size of 100 or less for ease and convenience was obtained. Therefore, out of the sampling frame (N) i.e., from the total 180 stories, 90 debunked fake news stories (45 stories from each website) were selected as sample size (n) from 90 days. Hence, the interval (i) was calculated as follows:

Interval (i)=Sampling frame/Sampling size=180/90=2

Hence, every second (2nd) element or date was selected from the sampling frame starting with the first randomly selected sample element. The first randomly selected unit of the present sampling frame was the first published story of a fake news item debunked by the website on December 2, 2019. With the calculated interval size, the following published news stories were selected from December 4, 6, and 8, 2019 and so on. Hence, even number date news stories were selected for the months of December 2019 and February 2020 while odd number date news stories were selected for the month of January 2020. This criterion was used to select sample units from both of the fact-checking websites.

The first debunked news story of a day was selected for the sample. If no debunked fake news was found on any of the sampled dates then, an additional news story from the following date in the sample sequence was selected. For example, if no debunked fake news was found on December 6, 2019, then an additional news story was taken from December 8. The present study only selected fake news from India.

Exclusion Criteria: The following three types of news stories were excluded from the sample:

  • News stories with no link to India

  • News stories in languages other than English

  • Fake news story found in the second website that was the same as found on the first one

This sample selection method was used in both fact-checking websites. The sample size taken from each website was 45 debunked fake news stories. Thus, the overall sample size of the present study was 90 debunked fake news stories.

Instrument: Coding and Classification

The collected sample data was analyzed and categorized based on the works of Jabbour (2013), Seuring (2013), and Krippendorff (2018). The four major categories of the classification were solely based on the four research questions of the present research – subject-matter, presentation elements, debunking tool/technique, and social media outlet. These major categories were further divided into subcategories. The subcategories were arrived at after a careful analysis of the characteristics of the sampled debunked fake news stories that fell into a particular major category. Alphabetic codes were developed for all the subcategories accordingly with the help of an inductive coding approach in which the data itself was used as a basis for creating the codes (Boyatzis, 1998; Thomas, 2006). The content of the codes, categories, and classifications was validated by three senior researchers (experts) of the related field specializing in media literacy and misinformation. All of the experts are in substantial agreement regarding the adequacy and validity of the scale for assessing the data. Table 1 presents the classification according to which the sample data was analyzed and coded. Table 2 shows the categorization, classification, and coding of all the 90 debunked fake news stories. Both the tables are presented in the Appendix section.

Results and Discussion

The following are the results for the different categories and subcategories. In case of percentages, the results were rounded to nearest figures. Also, the percentage total of all the sub-categories under a major category was more than 100% because some news stories belonged to more than one sub-category.

RQ1. Subject Matter of Debunked News

The first major category involved the identification of the subject matter of a fake news story that one of the fact-checking websites debunked. It was divided into 11 sub-categories having alphabetic codes from A to K. The time period of the present study saw protests against the ruling government due to the CAB, CAA, NPR, and NRC across India. Fake news related to these issues was widely shared on social media. In the study, 35 fake news stories i.e., 39% of the 90 sample stories were related to these topics in some way. It was further observed that these protests were used as a backdrop to spread fake news in almost all sub-categories.

The results showed that the majority i.e., more than half, of the debunked fake news stories (71%) were related to politics (B). This hinted that in India, politics could be a major root cause of fake news. It was observed that most of these fake news stories revolved around different political parties or their leaders, and in many cases, leaders from one political party disseminated fake news about other parties. The main intention was to demean a particular leader or its party among the public through fake news. It was observed that when party leaders shared fake news, most of the users having the same political ideology believed it. This audience liked, commented, and further circulated the misleading news on social media, thereby proliferating fake news. This is consistent with the findings of Van Bavel and Pereira (2018) who have proposed a belief model in their study to understand the effect of partisanship on cognitive processes and explained it as the reason behind people being blindly loyal to a political party to the extent of even disregarding truth. This was observed ahead of Assembly Elections in Delhi (the capital of India) in February 2020 (“Delhi Assembly Election 2020: Important Dates, Full Schedule and Voting Timings, a Quick Guide to 8 February Polls,” 2020) when prominent politicians circulated several fake news stories about different parties on social media. Some fake news stories revolved around US President Donald Trump’s visit to India in the same month (Lakshman, 2020). These observations clarified that media as well as political literacy is very important to educate the public about fake news. After politics, the next most common subject matter of the fake news (22%) was communal issues (issues related to conflicts between ethnic or religious communities) (A). One reason for such a high number of communal fake news could be the aforementioned protests. Many communal fake news stories were observed to be related to the protests. Fake news related to the Jafrabad communal riots in Delhi during the end of February 2020 (“13 Dead, over 250 Injured in Delhi’s Worst Communal Riot in Three Decades,” 2020) also added to this category. It was observed that such fake news was shared to falsely pit one community against another and create disturbances. Some fake news stories were observed to be circulated with the intention of disrupting the harmony between Muslims and Hindus. Ten of the 64 political fake news stories, i.e., 15.6% of the political stories, were directly related to communal issues. This hinted to the fact that communal issues were often used to spread fake news for a political leader’s or party’s benefit. The next subject matter with the highest number of fake news stories was internationally linked to India (I) with 16.7% of fake news stories. Among them, 11 stories i.e., 73% of the international stories were related to politics. In such stories, images and videos from other countries were used to falsely narrate incidents that happened in India.

Figure 1
Figure 1:Subject Matter of Fake News

Educational (E) fake news stories were the third most common. All these stories revolved around two Indian universities – Jawaharlal Nehru University (JNU) and Jamia Millia Islamia, both situated in Delhi. Of the 14 stories, 15.6% of educational fake news stories, about 57% i.e., 8 stories, were related to JNU, about 28.6% i.e., 4, to Jamia, and only 7% i.e., 1, was linked to Aligarh Muslim University in Uttar Pradesh state (AMU) and IIT each. Fake news related to JNU protests against a hostel fee hike, CAA, and campus violence at the time of university admission at the beginning of January 2020 were widely shared on social media (Prasad, 2020). In Jamia and AMU, students started protests against CAA and NRC in December 2019 (Ravi, 2019). Jamia students also witnessed the brutality of police when police thrashed unarmed students in the university’s reading room (The Wire Staff, 2019). Fake news related to all these student protests and police violence became rampant on social media. The main intention behind such educational fake news was observed to be to discredit the students’ protests or to further agitate them by falsely blaming the student protesters.

Crime (D) fake news stories were the next most extensively circulated online to mislead users. Out of total 10% of crime stories, 8 stories i.e., about 88.9%, were related to rape or sexual harassment and 5 stories i.e., about 5.6%, had some link to politics. Most of this fake news was spread against the backdrop of the November 2019 Hyderabad rape and murder case of a veterinary doctor (“Hyderabad Veterinarian Rape and Murder: Killers Burnt Body at Underpass,” 2019). The main intention behind such fake news was observed to be to provoke the public’s anger against such crimes and criminals. Subjects with fewer less fake news stories were: economy-related (C, 6.7%), historical (J, 4%), entertainment (F, 3%), health (G, 3%), and sports (H, 1%). In some fake news, historical events like Mahatma Gandhi’s assassination were used with false narratives to mislead users. All 3 health-related fake news stories were related to coronavirus. Such a low number of coronavirus stories could be because it had not yet spread to India. The coronavirus stories were from February. The months after February observed a rise in fake news related to coronavirus in India. Further research in this field is recommended. In addition, research related to fake news in a particular subject matter is also recommended to have a better understanding of the subject area with respect to that particular field. A total of 5.6% of stories belonged to the Other (K) subcategory as they didn’t fall into any of the other mentioned subcategories.

RQ2. Presentation Elements of Debunked Fake News

We identified the presentation element of fake news stories that the fact-checking websites had debunked. This category was divided into 6 sub-categories with alphabetic codes from A to F.

Figure 2
Figure 2:Presentation Elements of Fake News

In all of the sampled stories except four, only one presentation element was used. Here, “image” means still images such as photographs and art. In the case of the sub-categories “image(s) with text” and “video with text,” text refers to the content which was written (like a caption) in the social media post as an explanation of the image or video. Half of the sample of fake news stories (50%) used a video with text (B). In most cases, the context of the video was changed in the shared post using a false narrative to mislead users. Some fake news stories used clipped or manipulated videos to distort facts. This hinted that users more often believed and spread fake news having a video. The next widely used presentation element was image with text (A, 31%). Many fake news stories used one or more photographs to spread misinformation. The photograph was accompanied by a false narrative to change its context. In some cases, photographs were photo shopped or digitally manipulated. In most of the fake news stories, old and unrelated images and videos were used with false claims to spread misinformation. A smaller number of fake news stories (12%) used only text (C). A new presentation element, which was observed in the study of fake news stories, was the infographic (D, 4%). In such stories, an image showing information through a combination of text, data graphs, symbols, and other graphics were used (Merriam Webster, n.d.). This presentation element was given a separate sub-category because social media posts that used an infographic did not need a caption or narrative (text) as the infographic gave all the information itself. In 4% of news stories, text images (F) were used. Here, “text image” refers to screenshot of a fake post from one social media platform that was posted on another platform to spread misinformation. These posts used two elements, one kind of presentation element (only text) in one social media site and another element (the screenshot image of the text) of another social media site. A negligible number of fake news stories (2%) used text with a link or an attachment (E) to spread misinformation. Such posts had text (usually a little caption) accompanied by a link or attachment which users can click to get more information about what was being said in the social media post.

RQ3. What are the prominent debunking tool(s) and technique(s) used by fact-checkers?

The third major category identified the tool(s) or technique(s) used by the fact-checking website to debunk a fake news story. Here, tools refers to the software used for debunking, like Google Reverse Image Search, InVid, etc., while technique refers to the manual analysis done by the fact-checkers, like contacting someone to get information or careful examination of images and videos, etc. It was observed that rarely was only one tool or technique used for debunking. In most of the cases, a combination of tools and techniques were used. This category had the largest number of subcategories. It was divided into 15 sub-categories having alphabetic codes from A to O.

Figure 3
Figure 3:Debunking Tool(s)/Technique(s) Used

Information from official or primary sources (I) was used most frequently to debunk fake news stories (36.7%). In this technique, reports, data, statements, announcements, or any updates from official or primary sources were used to debunk false claims. Official social accounts, websites, or media reports were sought to obtain information from primary sources to uncover facts. The next most commonly used tool was Google reverse image search (A), which was used in 25.6% of the debunked stories. This tool was used primarily for images and for videos, after making screenshots (Roozenbeek & Van Der Linden, 2019). It helped in finding the origin of an image or a video. The original context of the shared image or video was revealed through this tool once the original was found. It also helped in identifying if the shared image or video was manipulated in any way to mislead users. At this step, the original and shared images were carefully compared (K, 9%) to know the truth. In the case of some fake news stories using video, this tool was used after using InVid. Fact-checkers used InVid to split a video into multiple keyframes (InVid Verification Application, n.d.). One of the keyframes was then subjected to Google reverse image search to identify the origin of the video. InVid (G) was used in 10% of the stories. Alt News used InVid more than Factly. InVid was used for just one news story checked by Factly. A screenshot or screengrab of the shared video was subjected to Google reverse image search in Factly.

Keyword searches on Google (E, 21%), YouTube (D, 17.8%), Twitter (B, 3%) and Facebook (O, 2%) were also used to debunk fake news stories. Shared videos were also carefully observed or examined (J, 17.8%) to identify any discrepancies. For fake news stories that had some international link, photographs, keyframes, or screenshots of videos were subjected to a reverse image search on a Russian search engine, Yandex (H, 9%). In some cases, a simple Internet search on Google with search filters (F, 6.7%) was used to identify about the exact origin of a claim. The person to whom the claim was attributed in the social media post was also directly contacted (L, 4%) by the fact-checking website to know about the authenticity of the claims. In a few cases, officials from an organization or body related to a fake news story (C, 3%), reputed experts on the subject matter (M, 2%), and police (N, 2%) were directly contacted to debunk shared social media claims. Both the fact-checking websites mostly used a combination of the aforementioned tools and techniques to thoroughly analyze a fake news story and debunk it with all the pieces of evidence found during the analysis.

RQ4. Fake News Platforms

The fourth and last major category identified the social media site where a fake news story was widely circulated and received many likes, shares, and comments. This category was divided into only 4 subcategories using alphabetic codes from A to D.

Most often, a particular fake news story was found to have been circulated on more than one social media platform. Facebook (A) was the site on which the highest number of fake news stories, more than half (81%), were found. This could be due to the fact that Facebook has the highest number of active users in the world (Clement, 2020b). It could be a reason for the proliferation of fake news on Facebook. The site with the next highest number of fake news stories was Twitter (B), with 49% of the total fake stories. This micro-blogging platform is becoming very popular among social media users. This could be a reason for the high number of fake news stories found on Twitter (Smith, 2020). Very few fake news stories were found to be shared on WhatsApp (C, 9%) or YouTube (D, 4%). Despite the present findings, WhatsApp groups have also emerged as a creator and propagator of fake news stories in India (Farooq, 2018) as well as other countries, like Brazil (Resende et al., 2019) and Nigeria (Apuke & Omar, 2020). In their study, Resende et al. (2019) have analyzed the flow of mis/information within publicly accessible WhatsApp groups in Brazil. They have further designed a web-based system called WhatsApp Monitor which helps fact-checkers and journalists to know about the most popular content being shared in the groups. Many misleading videos are also uploaded and shared on YouTube (Hussain et al., 2018). Apart from these four social media sites, fake news is also disseminated on other social networking sites. Further studies can be conducted on fake news spread and developing strategies to combat fake news on these less frequently used social media platforms. Apart from the above results, it was observed during the analysis that five misleading news stories i.e., about 5.6%, were found to be spread through national media platforms. This hinted that even mainstream media could become a victim of fake news. Thus, it is very important for media professionals to carefully observe a news story to check its authenticity and truthfulness before publishing or broadcasting it. Media professionals also need to be made more aware and educated about fake news. Organizing more workshops and training programs on identifying fake news for media professionals can help. Further research on misleading news reported by media is recommended in future.

Figure 4
Figure 4:Social Media Site on Which Fake News Was Shared

Conclusion and Recommendations

Political fake news stories were the most commonly found online. Of these political stories, majority were international or had a communal angle. This finding implies that in India, most of the fake news originates from the political field or has some link to it. The reason could be propaganda or conspiracy theories used by political parties to target the people who share their political ideologies, which often lead to polarization in a society. Unverified and false political, communal, or xenophobic content may lead to communal clashes, lynching, hate crimes, and prove dangerous for democracy. The present study recommends that people should cross verify any claim before believing it. Most of the fake news stories were circulated using video with a false narrative to mislead users. Users and fact-checkers need to be more alert when viewing social media posts with videos. The most used debunking tool/technique was referring to information from official or primary sources. Fact-checkers used a combination of debunking tools and techniques. These can also be used by social media users who must apply critical thinking and use tools like InVid data viewer, YouTube data viewer, and Google reverse image search to check the claims. After the evaluation of this data, source verification or consulting the officials could be another option to debunk false claims in a shared story. Facebook and Twitter are two of the most popular social networking sites and were found to have the highest number of fake news stories. Users need to be more alert when coming across content on these two social media outlets. Fact-checkers should more often check the content of these two social media sites.

In almost all the stories, it was found that most of the social media users believed the false claims blindly. This shows that there is an urgent need of media and information literacy, which should be taught to students beginning in elementary school. Users, as well as media professionals, must be educated on how not to believe everything that is on social media. Social media users need to think analytically while coming across fake news stories (Pennycook & Rand, 2019). In their study, Pennycook and Rand (2019) found that lazy thinking is the reason behind people falling prey to political fake news. They further suggest that analytic thinking can play a major role in self-inoculating people against such disinformation. Social media platforms can warn users by highlighting a particular story that needs a fact-check. That would simply help people to believe the claim only after its verification. Fact-checking initiatives should be increased. Non-Government Organizations (NGOs) and other institutions should also promote these initiatives along with media literacy. Transparency in the working of governments and other organizations is necessary. Data should be made available freely and transparently to help counteract fake information. Technology is not the only solution to curb fake news. Active and collaborative participation of social media platforms, users, governments, and media professionals can help to fight fake news not just in India but also around the world.

Factly mostly analyzed fake news from Facebook. Fake news from other social media sites were not analyzed much. This could be a limitation of the present study, and future studies can focus more on other social media sites, too. Although the present study has selected technocrat-operated fact-checking websites as they tend to not have much bias towards certain ideologies and advertisers, they are not immune to bias. Thus, future studies can be conducted on the performance of technocrat-operated vs. media-operated fact-checking agencies. The time period selected for the study was only three months; future studies can be done for longer time periods to get a better understanding of fake news trends and the current situation in India.


Accepted: February 06, 2021 KST

References

13 dead, over 250 injured in Delhi’s worst communal riot in three decades. (2020, February 26). Hindustan Times. https://www.hindustantimes.com/india-news/communal-riot-rages-on-in-capital-13-dead/story-x9d9fYOo7WTIdGmCT5l4VK.html
Akers, J., Bansal, G., Cadamuro, G., Chen, C., Chen, Q., Lin, L., Mulcaire, P., Nandakumar, R., Rockett, M., Simko, L., Toman, J., Wu, T., Zeng, E., Zorn, B., & Roesner, F. (2018). Technology-enabled disinformation: Summary, lessons, and recommendations. arXiv Preprint. https://arxiv.org/abs/1812.09383
Allcott, H., & Gentzkow, M. (2017). Social media and fake news in the 2016 election. Journal of Economic Perspectives, 31(2), 211–236. https://doi.org/10.1257/jep.31.2.211
Google Scholar
Apuke, O. D., & Omar, B. (2020). Fake news proliferation in Nigeria: Consequences, motivations, and prevention through awareness strategies. Humanities & Social Sciences Reviews, 8(2), 318–327. https://doi.org/10.18510/hssr.2020.8236
Google Scholar
Bakir, V., & McStay, A. (2018). Fake news and the economy of emotions: Problems, causes, solutions. Digital Journalism, 6(2), 154–175. https://doi.org/10.1080/21670811.2017.1345645
Google Scholar
Banerjee, P. (2020, February 12). WhatsApp announces 2 billion users worldwide. Livemint. https://www.livemint.com/technology/tech-news/whatsapp-announces-2-billion-users-worldwide-11581516342061.html
Boyatzis, R. E. (1998). Transforming qualitative information: Thematic analysis and code development. Sage.
Google Scholar
Bulger, M., & Davison, P. (2018). The promises, challenges, and futures of media literacy. Data & Society Report. https://datasociety.net/library/the-promises-challenges-and-futures-of-media-literacy/
Chan, M.-P. S., Jones, C. R., Jamieson, K. H., & Albarracín, D. (2017). Debunking: A meta-analysis of the psychological efficacy of messages countering misinformation. Psychological Science, 28(11), 1531–1546. https://doi.org/10.1177/0956797617714579
Google Scholar PubMed Central PubMed
Clement, J. (2020a). Leading countries based on Facebook audience size as of July 2020 (in millions). https://www.statista.com/statistics/268136/top-15-countries-based-on-number-of-facebook-users/
Clement, J. (2020b). Number of monthly active Facebook users worldwide as of 1st quarter 2020 (in millions). https://www.statista.com/statistics/264810/number-of-monthly-active-facebook-users-worldwide/
Delhi Assembly Election 2020: Important dates, full schedule and voting timings, a quick guide to 8 February polls. (2020, January 22). Firstpost. https://www.firstpost.com/politics/delhi-assembly-election-2020-important-dates-full-schedule-and-voting-timings-a-quick-guide-to-8-february-polls-7940591.html
Egelhofer, J. L., & Lecheler, S. (2019). Fake news as a two-dimensional phenomenon: A framework and research agenda. Annals of the International Communication Association, 43(2), 97–116. https://doi.org/10.1080/23808985.2019.1602782
Google Scholar
Farkas, J., & Schou, J. (2018). Fake news as a floating signifier: Hegemony, antagonism and the politics of falsehood. Javnost - The Public, 25(3), 298–314. https://doi.org/10.1080/13183222.2018.1463047
Google Scholar
Farooq, G. (2018). Politics of fake news: How WhatsApp became a potent propaganda tool in India. Media Watch, 9(1), 106–117. https://doi.org/10.15655/mw/2018/v9i1/49279
Google Scholar
Goel, K. (2019, December 18). Here’s your round-up of all the fake news around CAA protests. The Quint. https://www.thequint.com/news/webqoof/round-up-of-fake-news-around-caa-protests-students-police
Gruhl, D., Guha, R., Liben-Nowell, D., & Tomkins, A. (2004). Information diffusion through blogspace. WWW ’04: Proceedings of the 13th International Conference on World Wide Web, 491–501. https://doi.org/10.1145/988672.988739
Google Scholar
Guha, S. (2017). Related Fact Checks: A tool for combating fake news. arXiv Preprint. https://arxiv.org/abs/1711.00715
Hussain, M. N., Tokdemir, S., Agarwal, N., & Al-Khateeb, S. (2018). Analyzing disinformation and crowd manipulation tactics on YouTube. 2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), 1092–1095. https://doi.org/10.1109/asonam.2018.8508766
Google Scholar
Hyderabad veterinarian rape and murder: Killers burnt body at underpass. (2019, November 29). The Hindu. https://www.thehindu.com/news/national/telangana/hyderabad-veterinarian-rape-and-murder-killers-burnt-body-at-underpass/article30119040.ece
India Today Web Desk. (2019, December 18). What is NRC: All you need to know about National Register of Citizens. India Today. https://www.indiatoday.in/india/story/what-is-nrc-all-you-need-to-know-about-national-register-of-citizens-1629195-2019-12-18
InVid Verification Application. (n.d.). https://www.invid-project.eu/invid-verification-application/
Jabbour, C. J. C. (2013). Environmental training in organisations: From a literature review to a framework for future research. Resources, Conservation and Recycling, 74, 144–155. https://doi.org/10.1016/j.resconrec.2012.12.017
Google Scholar
Keelery, S. (2020, July 7). Number of social network users in India from 2015 to 2018 with a forecast until 2023 (in millions). https://www.statista.com/statistics/278407/number-of-social-network-users-in_india/
Krippendorff, K. (2018). Content analysis: An introduction to its methodology. Sage publications.
Google Scholar
Laghate, G. (2019, December 10). YouTube in India has over 265 mn monthly active users 1200+ channels with 1 mn+ subs. The Economic Times. https://economictimes.indiatimes.com/industry/media/entertainment/youtube-in-india-has-over-265-mn-monthly-active-users-1200-channels-with-1-mn-subs/articleshow/72456212.cms?from=mdr
Lakshman, S. (2020, February 11). U.S. President Donald Trump to visit India on February 24 and 25: White House. The Hindu. https://www.thehindu.com/news/international/us-president-donald-trump-announces-visit-to-india-from-february-24-25/article30788848.ece
Latief, S. (2019, December 17). What is citizenship law and why people are protesting against it. The Times of India. https://timesofindia.indiatimes.com/india/what-is-citizenship-amendment-act-and-why-has-it-triggered-protests/articleshow/72759793.cms
Lavoipierre, A. (2017). Fake news’ named 2016 Word of the Year by Macquarie Dictionary. ABC News, 25. https://www.abc.net.au/news/2017-01-25/fake-news-named-2016-word-of-the-year/8211056
Lavrakas, P. J. (2008). Encyclopedia of survey research methods. Sage Publications. https://doi.org/10.4135/9781412963947
Google Scholar
Merriam Webster. (n.d.). Infographic. In Merriam Webster’s Online Dictionary. https://www.merriam-webster.com/dictionary/infographic
Mishra, V. (2019, October 18). India’s fake news problem is killing real people. Asia Times. https://asiatimes.com/2019/10/indias-fake-news-problem-is-killing-real-people/
Morgan, D., & Hoax, T. S. (2019, November 13). Beyond fake news – 10 types of misleading news. Infographic. Eavi. https://eavi.eu/beyond-fake-news-10-types-misleading-info/
Nielsen, R. K., & Graves, L. (2017). “News you don’t believe”: Audience perspectives on fake news. Reuters Institute for the Study of Journalism. https://reutersinstitute.politics.ox.ac.uk/our-research/news-you-dont-believe-audience-perspectives-fake-news
Nielsen, R. K., & Schrøder, K. C. (2014). The relative importance of social media for accessing, finding, and engaging with news: An eight-country cross-media comparison. Digital Journalism, 2(4), 472–489. https://doi.org/10.1080/21670811.2013.872420
Google Scholar
Paris, B., & Donovan, J. (2019). Deepfakes and cheap fakes: The manipulation of audio and visual evidence. Data & Society. https://datasociety.net/wp-content/uploads/2019/09/DS_Deepfakes_Cheap_FakesFinal-1-1.pdf
Pennycook, G., Bear, A., Collins, E. T., & Rand, D. G. (2020). The implied truth effect: Attaching warnings to a subset of fake news headlines increases perceived accuracy of headlines without warnings. Management Science, 66(11), 4921–5484. https://doi.org/10.1287/mnsc.2019.3478
Google Scholar
Pennycook, G., Cannon, T. D., & Rand, D. G. (2018). Prior exposure increases perceived accuracy of fake news. Journal of Experimental Psychology: General, 147(12), 1865–1880. https://doi.org/10.1037/xge0000465
Google Scholar PubMed Central PubMed
Pennycook, G., & Rand, D. G. (2019). Lazy, not biased: Susceptibility to partisan fake news is better explained by lack of reasoning than by motivated reasoning. Cognition, 188, 39–50. https://doi.org/10.1016/j.cognition.2018.06.011
Google Scholar
Popat, K., Mukherjee, S., Yates, A., & Weikum, G. (2018). DeClarE: Debunking fake news and false claims using evidence-aware deep learning. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. https://doi.org/10.18653/v1/d18-1003
Google Scholar
Posetti, J., & Matthews, A. (2018). A short guide to the history of ‘fake news’ and disinformation: A New ICFJ Learning Module. International Center For Journalists. https://www.icfj.org/sites/default/files/2018-07/A%20Short%20Guide%20to%20History%20of%20Fake%20News%20and%20Disinformation_ICFJ%20Final.pdf
Prasad, S. K. (2020, January 10). India is cracking down on university protests. Here’s what you need to know. The Washington Post. https://www.washingtonpost.com/politics/2020/01/10/india-is-cracking-down-university-protests-heres-what-you-need-know/
PTI. (2018, June 24). Fake news proving deadly, leading to loss of lives in India: BBC. The Quint. https://www.thequint.com/news/india/bbc-fake-news-proving-deadly-leading-to-loss-of-lives-india
Qayyum, A., Qadir, J., Janjua, M. U., & Sher, F. (2019). Using blockchain to rein in the new post-truth world and check the spread of fake news. IT Professional, 21(4), 16–24. https://doi.org/10.1109/mitp.2019.2910503
Google Scholar
Rajalakshmi, T. K. (2020, January 17). Countrywide CAA & NRC protests: Secular unity. Frontline. https://frontline.thehindu.com/cover-story/article30431601.ece
Ravi, S. (2019, December 15). Anti-Citizenship Act protests: Violence hits Delhi, over 50 injured. The Hindu. https://www.thehindu.com/news/national/police-enter-jamia-millia-islamia-campus-in-delhi-as-anti-citizenship-act-protest-turns-violent/article30312133.ece
Resende, G., Melo, P., Sousa, H., Messias, J., Vasconcelos, M., Almeida, J., & Benevenuto, F. (2019). (Mis)Information Dissemination in WhatsApp: Gathering, Analyzing and Countermeasures. The World Wide Web Conference, 818–828. https://doi.org/10.1145/3308558.3313688
Google Scholar
Roozenbeek, J., & Van Der Linden, S. (2019). The fake news game: Actively inoculating against the risk of misinformation. Journal of Risk Research, 22(5), 570–580. https://doi.org/10.1080/13669877.2018.1443491
Google Scholar
Samuels, E. (2020, February 21). How misinformation on WhatsApp led to a mob killing in India. The Washington Post. https://www.washingtonpost.com/politics/2020/02/21/how-misinformation-whatsapp-led-deathly-mob-lynching-india/
Seuring, S. (2013). A review of modeling approaches for sustainable supply chain management. Decision Support Systems, 54(4), 1513–1520. https://doi.org/10.1016/j.dss.2012.05.053
Google Scholar
Sharma, K., Qian, F., Jiang, H., Ruchansky, N., Zhang, M., & Liu, Y. (2019). Combating fake news: A survey on identification and mitigation techniques. ACM Transactions on Intelligent Systems and Technology (TIST), 10(3), 1–42. https://doi.org/10.1145/3305260
Google Scholar
Shu, K., Sliva, A., Wang, S., Tang, J., & Liu, H. (2017). Fake news detection on social media: A data mining perspective. ACM SIGKDD Explorations Newsletter, 19(1), 22–36. https://doi.org/10.1145/3137597.3137600
Google Scholar
Shu, K., Wang, S., Lee, D., & Liu, H. (2020). Mining disinformation and fake news: Concepts, methods, and recent advancements. arXiv Preprint. https://arxiv.org/abs/2001.00623
Smith, K. (2020, January 2). 60 Incredible and Interesting Twitter Stats and Statistics [Blog post]. Brandwatch. https://www.brandwatch.com/blog/twitter-stats-and-statistics/
Tandoc, E. C. Jr., Lim, Z. W., & Ling, R. (2018). Defining “Fake News”: A typology of scholarly definitions. Digital Journalism, 6(2), 137–153. https://doi.org/10.1080/21670811.2017.1360143
Google Scholar
Tankovska, H. (2021). Leading countries based on number of Twitter users as of January 2021 (in millions). https://www.statista.com/statistics/242606/number-of-active-twitter-users-in-selected-countries/
The Wire Staff. (2019, December 16). After a day of police violence, Jamia students detained for protesting CAA released. The Wire. https://thewire.in/rights/citizenship-amendment-jamia-millia-islamia-police-violence
Thomas, D. R. (2006). A general inductive approach for analyzing qualitative evaluation data. American Journal of Evaluation, 27(2), 237–246. https://doi.org/10.1177/1098214005283748
Google Scholar
Torabi Asr, F., & Taboada, M. (2019). Big data and quality data for fake news and misinformation detection. Big Data & Society, 6(1), 205395171984331. https://doi.org/10.1177/2053951719843310
Google Scholar
Van Bavel, J. J., & Pereira, A. (2018). The partisan brain: An identity-based model of political belief. Trends in Cognitive Sciences, 22(3), 213–224. https://doi.org/10.1016/j.tics.2018.01.004
Google Scholar
Vosoughi, S., Roy, D., & Aral, S. (2018). The spread of true and false news online. Science, 359(6380), 1146–1151. https://doi.org/10.1126/science.aap9559
Google Scholar
Wagner, M. C., & Boczkowski, P. J. (2019). The reception of fake news: The interpretations and practices that shape the consumption of perceived misinformation. Digital Journalism, 7(7), 870–885. https://doi.org/10.1080/21670811.2019.1653208
Google Scholar
Wardle, C. (2017). Fake news. It’s complicated. First Draft News, 16. https://medium.com/1st-draft/fake-newsits-complicated-d0f773766c79
Google Scholar

Appendix

Table 1:Coding for Table 2
Major Category Subcategory with respective Codes
Subject matter of fake news A. Communal
B. Political
C. Economy related
D. Crime
E. Education
F. Entertainment
G. Health
H. Sports
I. Internationally linked to India
J. Historical
K. Others
Presentation element of fake news A. Image(s) with text
B. Video with text
C. Only text
D. Infographic
E. Text with a link/attachment
F. Text image
Debunking tool/ technique used A. Reverse image search on Google
B. Keyword search on Twitter
C. Directly contacting official from the concerned body/authority
D. Keyword search on YouTube
E. Keyword search on Google
F. Simple Internet search on Google using search filters (date and time filters, etc.)
G. InVid
H. Reverse image search on Yandex
I. Referring to reports/data/information from official or primary sources
J. Careful observation and examination of video
K. Comparing images
L. Referring to the person to whom the fake news is attributed
M. Directly contacting reputed experts on the subject matter
N. Directly contacting police
O. Keyword search on Facebook
Social Media Site/s on which fake news was shared A. Facebook
B. Twitter
C. WhatsApp
D. YouTube
Table 2:Characteristics of Fake News Stories
Story No. Subject matter Presentation element(s) Debunking tool/ technique used Social media site on which news was shared
1. D B A, G A, B
2. C C C, I A, B, C
3. B, D B D, J A
4. D D A A, B
5. A, E A C, E, I A, B
6. B B A, G C
7. K B F A, B
8. B, I A E, H B, C
9. B A I, K A, B
10. B A A B
11. A, B, E F F, L A, B
12. B, I, J B D, F, J A, B
13. A B E A, B, C
14. K B E B
15. B, I B G, H, J B
16. B, I B D, J B, C
17. A, B, I B D A, B
18. B, E B J, K, L B
19. B, E A I, L A, B
20. A, B B H, G A, B
21. B, E B A, G A, B
22. B, E, J A A, I, M B
23. B B D, I, J A
24. B, C, D A A A, B
25. A, D B E, I, N A, B
26. B A A A, B
27. B A K, L A, B
28. B B B, G, H A, B, C
29. F B A, G, K A, B
30. A, B B D, O A, B
31. A, B C I A, B, C
32. B, C B E A, B
33. B B E, I, J A, B
34. A, B B D, J B
35. B, I E I B
36. G, I A, B I, M A, D
37. B A I, K B
38. B B D, G, H A, B
39. B, E B I, J D
40. B, E B D, I, J D
41. B A A B
42. C, E A B, E C
43. B B C, D B, D
44. K A I A, B
45. A A A A
46. B, C D E, I A
47. B, D C I A
48. B, H C, F I A, B
49. A A A A
50. D D A A
51. B B H A
52. B, E A E, I A
53. A, B, E B D, J A
54. B, I E I, J A
55. B, C C, F B A, B
56. A B N A
57. B A A A
58. B A A, K A
59. A C E, I A
60. B, I A A A
61. A, B, I B A, J A
62. B, I B H, D A
63. B, F C, F I A, B
64. A, F A I A
65. B B A, G A
66. B, E A A, E, K A
67. K B E A
68. B A A B
69. B, E B F, J A
70. B B E A
71. B B E A
72. A, B B E A
73. G, I B H, I A
74. B, D, E A F A
75. G C I A, B
76. B, J A A A
77. A B E, I A
78. B B F A
79. B, I A A A
80. B B D, J A
81. B B D, J A
82. B C I A
83. A, B D I A
84. B, D C I A
85. I, J A I A
86. A, I B A A
87. K C E, I A, B
88. B B D A
89. B A E, I, K, O A
90. K B D, I A, B