Monday, February 04, 2013

“Victims of Our Own Narratives?”

Portrayal of the “Other” in Israeli and Palestinian School Books
Initiated by the Council of Religious Institutions of the Holy Land
Funded by a grant from the United States Department of State, Bureau of Democracy, Human Rights and Labor awarded to A Different Future
Study Report, February 4, 2013  

Pursuant to promoting development of a culture of peace and mutual respect in the Holy Land, the Council of Religious Institutions of the Holy Land commissioned this study of how Israeli and Palestinian school books portray each other, the three Abrahamic faiths and themes related to conflict and peace. A joint Israeli/Palestinian research team developed and applied a standardized, manualized and multi-rater research method to maximize objectivity. Study methods and findings were reviewed by a Scientific Advisory Panel of international experts in text book analysis and leading Israeli and Palestinian academics. 

Israeli books were from the State Secular and Religious Tracts and from independent Ultra-Orthodox schools. Palestinian books were nearly all from the Ministry of Education’s text books, but a small number of books from the few independent religious schools (Al-Waqf) were included when relevant to study themes. Israeli books were on average 2.5 times as long as Palestinian books and included more photographs and illustrations. Israelis have produced their own school books for over 60 years, and they have gone through repeated cycles of review and revision. Palestinians began producing their own books for the first time only in 2000 and completed the full set of books for grades 1-12 in 2006. Several of the first-produced books are now in a second or third edition. 

There were four main findings of the study. First, dehumanizing and demonizing characterizations of the other as seen in textbooks elsewhere and of concern to the general public are rare in both Israeli and Palestinian books. Second, both Israeli and Palestinian books present unilateral national narratives that present the other as enemy, chronicle negative actions by the other directed at the self-community, and present the self-community in positive terms with actions aimed at self-protection and goals of peace. Descriptions of the other as enemy in Israeli books were generally related to violent attacks against Israelis, and Israeli books more often described Palestinians as aiming to destroy than to dominate Israel. Descriptions of the other as enemy in Palestinian books were generally related to the Jews being given (by international powers) or Israel appropriating Palestinian land and resources (e.g., water), and Palestinian books more often described Israelis as seeking to dominate than to destroy Palestinians. Historical events, while not false or fabricated, are selectively presented to reinforce each community’s national narrative. Third, there is a lack of information about the religions, culture, economic and daily activities of the other, or even of the existence of the other on maps. The absence of this kind of information about the other serves to deny the legitimate presence of the other. This important problem can be addressed by the addition of information about the culture, religions, and everyday activities of the other. Fourth, while present and problematic in all three
school systems, the negative bias in presentation of the other, the positive bias in presentation of the self, and the absence of images and information about the other are all statistically significantly more pronounced in the Israeli Ultra-Orthodox and Palestinian books than in the Israeli State books. Israeli State books provide some more information about the other, less negative overall characterization of the other, and multiple examples of actions by Israelis against Palestinians that were criticized as wrong by Israeli citizens and leaders. The Israeli Ultra-Orthodox and Palestinian books do not differ significantly from each other in these regards. Within the two tracks of the Israeli State schools, self-critical presentations were significantly more common in the secular than in the religious track books; books from the religious track included some self-critical examples but overall were similar in presentation of the self-community to the Israeli Ultra-Orthodox and Palestinian books.
This general profile of portrayals of the “other” is typical of school books of societies in conflict. Each society creates a national narrative based on repeated descriptions of the other and its acts in negative terms, recounting of historical events from the perspective of the self- community’s legitimate struggles for self-preservation in relation to threats of destruction or domination by the other, and the absence of information that legitimizes the presence and humanizes the other. These narratives help sustain the community as it deals with the violence, losses and deprivations of the conflict. However, they can also be obstacles to peace as they engender fear, mistrust, misunderstanding and dehumanization of the other. Social scientists have described steps through which conflict-related national narratives can be modified, and some governments have taken these steps in deliberate efforts to reduce conflict and promote peace.
It is recommended that the Israeli and Palestinian Ministries of Education each establish a committee of Ministry staff and community experts to review current and future books in light of the study findings and prepare a plan of action based upon the review. Additional steps will be necessary to facilitate similar processes regarding the books of the Israeli Ultra-Orthodox communities.
The Israeli-Palestinian conflict continues over self-determination, territory, natural resources, holy places and security. Contradictory goals and interests in different domains have to be addressed in conflict resolution. Resolution of these disagreements is made more difficult by powerful socio-psychological forces which fuel distrust and hostility. These forces include beliefs, perceptions, images, myths or attitudes about the rival, the collective self and the conflict. Such beliefs and images are often part of each society’s national narrative, and these narratives can be important as societies continue to marshal human and material resources demanded by the conflict. The narratives are propagated through many years by various channels of communication and various institutions in each involved society, including the education system. However, these collective narratives often leave little room for acknowledgement of the historical past, culture, and future aspirations of the other collective. Thus, while these narratives help sustain societies during conflicts, they can stand as a major obstacle to any peace-making process and later processes of reconciliation. The narratives may need to be modified in order to facilitate building a new reality of peace. In this endeavor there can be a need to modify school textbooks which may serve as one agent among others in socializing new generations.
The current study analyzed descriptions of the other side and the conflict in general that appear in the school textbooks of the Israeli and Palestinian educational systems. The Israeli- Palestinian conflict is long standing, and multiple observers have commented that negative stereotypic and dehumanizing views of the “other” create fear, hatred and enmity, contribute to
mistrust, and obstruct diplomatic efforts to resolve conflict. Explicit attempts to address these issues have been made, especially after the Oslo Accords were signed in 1993. But the national narratives have strong roots due to the long duration of the conflict. Moreover, accusations by each side about the current content of the other’s school books add to the mistrust and animosity.
Peace negotiations have included efforts to deal with this aspect of the conflict. In the Taba Agreement (Oslo 2, signed in 1995) under chapter four, Cooperation, article 22 says each side:
...shall accordingly abstain from incitement, including hostile propaganda, against each other...that their respective educational systems contribute to the peace between the Israeli and Palestinian peoples and to peace in the entire region, and will refrain from the introduction of any motifs that could adversely affect the process of reconciliation
Similarly, the Wye River Memorandum (signed in 1998) includes in Section A (Security Actions) an explicit statement about incitement and called for a committee with education specialists (Point 3 Prevention of Incitement):
(a) The Palestinian side would issue a decree prohibiting all forms of incitement to violence or terror, and establishing mechanisms for acting systematically against all expressions or threats of violence or terror. This decree would be comparable to the existing Israeli legislation which deals with the same subject. (b) A U.S.-Palestinian- Israeli committee would meet on a regular basis to monitor cases of possible incitement to violence or terror and to make recommendations and reports on how to prevent such incitement. The Israeli, Palestinian and U.S. sides would each appoint a media specialist, a law enforcement representative, an educational specialist and a current or former elected official to the committee.
School textbooks figure prominently in these discussions for two reasons. First, beliefs and attitudes about the other among the young generation are in part shaped by school textbooks and discussions based upon them. Second, the content of textbooks are public statements by the elders, leaders and governments of how they view one another and the conflict. While the effects on children can sustain fear and aggression across generations, the effects of such public depictions of the other can impact trust between those currently charged with ending the conflict.
Israelis and Palestinians make at least six primary charges and countercharges about the role of textbooks in preserving the negative and inappropriate representation of the other and the conflict. The first is that each side as a nation, society, community, or religion is unfavorably and/or inaccurately depicted in the textbooks of the other and even its legitimacy is disregarded. The second is that the textbooks misrepresent the description of the conflict by omitting, marginalizing or magnifying certain events or processes in order to present them in line with the society’s own national narrative. The third is that the textbooks neglect to teach the history, culture, religion and tradition of the other, and therefore miss the opportunity to recognize the humanity of the other (PRIME project: 2001-2009). The fourth is that both sides school textbooks fail to include examples of the peaceful periods of coexistence between the two sides and especially do not portray fairly the nature of relationship between Jews and Arabs through the years of a long history (Firer and Adwan:2004). The fifth claim is that the shared beliefs of the three Abrahamic faiths that could promote trust and familiarity are rarely covered in textbooks, including the fact that each community's own scriptural sources include passages that emphasize the equality of all people under God/Allah, and the value of treating all people accordingly. The sixth claim is that the textbooks do not prepare their own students to live in peace through active peace education.
The present project employs a new methodology to produce a transparent, simultaneous, comprehensive, and scientifically rigorous analysis of both current Israeli and current Palestinian school books. The research team consists of researchers fluent in Arabic and Hebrew, led jointly by senior Israeli and Palestinian researchers widely acknowledged for their expertise in text book analysis, and supported by an international advisory panel of leading Israeli and Palestinian academics and experts in school book analysis from Europe and the United States (Appendix A).
Key Features of the Study
  1. 1-  The research team is a joint Israeli/Palestinian team led by an Israeli professor (Daniel Bar- Tal) and a Palestinian professor (Sami Adwan), both internationally recognized for their expertise in text book analysis, and includes Israeli and Palestinian research assistants fluent in Arabic, Hebrew and English.
  2. 2-  Development and use of a standardized, manualized and multi-rater research method to maximize objectivity.
  3. 3-  Simultaneous evaluation of both Israeli and Palestinian books by the same research team using the same methods for both sets of books.
  4. 4-  Demonstrated inter-rater reliability: Israeli-Israeli, Palestinian-Palestinian, and Israeli- Palestinian.
  5. 5-  Remote data entry to a database at Yale so that no one on the research team knows how the data are adding up as the study progresses (e.g., no one sees how the results are looking after the first 10 books are entered and then has that knowledge influence, consciously or unconsciously, how they analyze subsequent books).
  6. 6-  Creation of an international scientific advisory panel of world experts in textbook analysis (see Appendix A) who met in Jerusalem with the research team to review all study methods before data acquisition began, monitored subsequent work including data analytic methods, reviewed study findings, and reconvened in Jerusalem to meet with the research team to help interpret study findings.
  7. 7-  Direct communication with the Israeli and Palestinian Ministries of Education to inform them of the study, invite discussion of planned study methods, and encourage their future participation in modifying the curriculum if study findings suggest modifications are desirable
    Overview of the School Systems Israeli School System.
    Israel's educational system consists of Jewish schools (study language is Hebrew) and
Arab schools (study language is Arabic). Hebrew education is divided into three tracks: State- secular, State-religious and Independent Ultra-Orthodox. The Arabic books used in schools for Arab children in Israel were not included in the study since they do not provide information about how each community portrays the other; the other in this case is the Israeli Jew but the books are prepared and approved by the Israeli Ministry of Education and not by the Arab community in Israel. State-secular schools provide non-religious education with curriculum approved and supervised by the Ministry of Education. State-religious schools are similarly supervised but offer religious education with intensive Jewish studies for children of the Orthodox Jewish sector. The Ultra-Orthodox schools, in contrast, operate independently and their books are not subject to approval by the Ministry of Education. Some UO schools formally
adhere to at least a portion of the core curriculum determined by the Ministry of Education as part of a variety of different funding relationships with the government (funding of the Ultra- Orthodox schools range from 55-100% of that provided per student in the State schools).
The law outlines the objectives of State education with regard to teaching universal values; the values of the State of Israel as both Jewish and democratic; history and heritage of the land of Israel and of the Jewish people; remembrance of the Holocaust and heroism; development of the child’s personality, creativity, talents and intellectual competencies; and acquaintance with the culture and heritage of the Arabs and of other minorities in Israel.
The total number of pupils who attended institutions in the Israeli educational system in the 2010/11 school year was 1,917,893. The number of schools in the Israeli educational system in 2010/11 was 4,385 (preschool and higher education not included), of which 2,399 were State- secular schools, 724 State-religious schools and 1,262 Ultra-Orthodox. State education consisted of 1,530 schools of Hebrew education and 869 schools of Arab Education (Ministry of Education, 2012).
The Ultra-Orthodox education system reflects the complex make-up of the Ultra-Orthodox community. Each sect, and even sub-group has its own education system. Hence, there are more than 1,200 schools in the system and the number keeps growing, though the number of pupils in each school can be small. In 2000, 19.2% out of all students in Israel were recorded as being part of the Ultra-Orthodox system, whilst in 2004 that number had increased to 26.2% (approximately 500,000 children).

Palestinian School System
After the 1948 war and until 1967, Palestinian education in the West Bank and Gaza Strip was under the auspice of Jordon and Egypt respectively. Palestinian children used textbooks that were produced according to Jordanian and Egyptian education philosophies and ideologies, and objectives and goals, teaching and learning methods, as well as teachers’ qualification and training were all in accordance with these ideologies. After the 1967 war, the Israeli Military Authority assumed control over the Palestinian education system. The same Jordanian and Egyptian textbooks continued to be used in Palestinian schools but were censored by the Israeli Military Education Commander. Specific passages were blackened out or deleted, and some books banned (e.g., the "Palestinian Cause" textbook for grade 12 was banned from use in mid 70’s). School development and teacher training were very limited, and textbooks became outdated.
The Palestinians assumed control of their own education system in 1994. The Palestinian Ministry of Education (PMoE) was one of the first four ministries to be established after the signing of the Oslo Accords. PMoE inherited a collapsed educational system and started a comprehensive plan to improve its quality and standards. The plan focused on structuring of the Ministry of Education itself and on forming the Palestinian education regulations and laws. It also included building new schools, unifying the education systems of Gaza Strip and the West Bank, producing Palestinian school textbooks, and upgrading teachers’ and school principals’ skills and knowledge through in-service training.
The stated overall aims of the Palestinian education system are to: prepare Palestinian children professionally and scientifically; enhance Palestinians understanding of their history, culture, heritages, aspirations and identity; emphasize human values such as freedom, justice, and equality; develop children’s technological and communicative abilities; and finally, to widen their global understanding and willingness to live in the world
There are three kinds of Palestinian schools, according to supervising authority: Public (governmental), UNRWA (the United Nations Relief and Work Agency) and Private. Governmental schools are submitted to direct supervision of the PMoE through Directorates of Education at the district level. This supervision applies both to schools in the West Bank and Gaza Strip, and all use the same books and take the same formal Tawjihee examinations (Jehad Zakarneh, Assistant Deputy Head of the Curriculum Center, Palestinian National Authority). UNRWA's schools and post-secondary institutions were opened in the 1950’s mainly for Palestinian refugees, up to grade nine. More than half of the schools in Gaza Strip where most refugee camps are located, are run by UNRWA. By law, UNRWA schools use the textbooks approved by the Palestinian Ministry of Education. UNRWA also provides supplemental teaching materials that include units on human rights and official UN maps. Private schools are located mainly in the major cities, and supervised by religious affiliations, private individuals or societies.
In year 2012,there were 2707 Palestinian schools in West Bank and Gaza Strip. Of these, 2005 (74%) are governmental schools are under the direct supervision of the Ministry of education through the 22 Educational Directorate offices (16 in the West Bank and 6 in the Gaza Strip). UNRWA schools are mainly open for the children of Palestinian refugees. According to the agreement signed between host countries and UN in early 1950s, UNRWA schools are obliged to use the school books produced by the host countries and only offer teaching for the compulsory period (grade 1-9). The number of UNRWA schools is 343 forming about 13% of the school system. Students in Governmental and UNRWA schools do not pay any fees/tuitions and school textbooks are given free of charge to their students. The number of Private schools is 359 forming about 13%. Students have to pay fees/tuition to enroll in these schools and pay for their textbooks. Pupils who study in this type of school are mostly from families in the middle and high socio-economic classes. There are 8 Palestinian schools with an Islamic Religious curriculum; 2 schools located in Gaza Strip and 6 schools located in the West Bank. Three schools are for female pupils (one school in Gaza Strip and two schools in West Bank) and 5 for male pupils (one in Gaza Strip and 4 in West Bank). About 800 students are enrolled in these schools. In addition to other subjects, they study Islamic religious schoolbooks developed by the Jordanian Ministry of Religious and Endowment beginning in the 7th grade. At the end of the 12th grade they sit for the Tawjihee exam as any other student. The same schoolbooks are used in all schools in the Palestinian National Authority (PNA) regardless of their supervising authorities, but different English textbooks are used in private schools and different books on religion are used in the very small number of schools with Islamic religious curriculum as just described. The total number of Palestinian students in Gaza and West Bank is 1,129,538. About 760,891(70%) are in Public schools, 253,116(23%) are in UNRWA schools, 7811(7%) are in Private schools, and 800 (0.1%) attend the schools with Islamic Religious curriculum. Within Gaza, approximately 67% of students are in UNRWA schools. The total number of Palestinian teachers is 52172, 36553( 70 %) teach in public schools, 9908 (19 %) in UNRWA schools and 5872 (11 %) in Private schools ( Palestinian Census Center, 2012).
Research Methods
The present study aimed to document the ways Palestinians and Israelis and the conflict between them are portrayed in each other's school textbooks. The study began with the goal of compiling a comprehensive list of books currently used in the Israeli and Palestinian school systems. While all of the Palestinian books were identified, this goal could not be fully attained for the Israeli books because of diversity and complexities in the Israeli Ultra-Orthodox schools.
With regard to the state educational system in Israel, the decision was taken to analyze only the Ministry of Education approved school textbooks.
Once appropriate books were obtained, those rich in content related to study themes were identified. Then a large enough number of passages from a large enough sample of books across disciplines and grade levels were content analyzed to produce a highly reliable characterization of content related to study themes. The study evaluated six thematic areas: 1) the “Other” Group; 2) One’s Own Group; 3) Religion; 4) Peace; 5) the Conflict; and 6) Values;. Photographs, maps, illustrations, tables and figures, and student activities were considered as well as written text.
Selection of the books
All books approved by the Israeli and Palestinian Ministries of Education for the year 2009, and books purchased from school bookstores in the Ultra-Orthodox communities, constituted the initial book list. From these lists, books in the following subject areas were considered for study: Literature, History, Languages (Arabic and Hebrew), Geography, Social studies, Civic/national education, Religion. In religion, the scriptures themselves were not included. Religious textbooks that use religious prophets’ Hadith and religious codes were included. Natural sciences textbooks which include biology, math, physics, chemistry, geology, etc. were not included. Language textbooks were not included, except Arabic language books in the Jewish Israeli educational system (there were no comparable Hebrew language books in the Palestinian system). The official list of books from the Israeli Ministry of Education was revised during the course of the study in 2011, with 69 books of the relevant subject matters dropped from the list and 214 added. Of these new books, 8 were relevant to study themes and entered into the analysis. The official 2009 list of books from the Palestinian Ministry of Education was reviewed for changes in 2011 (numbers of words and lines, study questions), and the few noted were not relevant to study themes. These considerations led to a potential study list of 381 Israeli and 142 Palestinian books from the two Ministries of Education. Books from the Ultra-Orthodox schools in appropriate topic areas were purchased directly from book stores, with 55 books used by the school system of Agudat Israel schools and 66 used by the school system of Maayan (Shas party). Together there were 492 Israeli books. Twenty-four books used in the Palestinian religious schools were also obtained, with only six proving relevant to study themes. This created a total of 148 Palestinian books.
General Process
Study methods proceeded in two phases. The first phase reviewed the books to determine the level of relevancy of each for the study, identify the relevant pages and accompanying additional material. The second phase was a standardized content analysis relevant to study themes.
Determining the Unit of Analysis and Analytical Procedures
The Unit of the Analysis was a literary piece (LP) that provides a holistic presentation
of the topic in a book. Literary piece can be a poem, story, chapter, essay, part of a book, etc. The unit can differ in the number of pages. In the case of readers, the unit can be very short even few sentences, while in the case of a history books, the unit can be a chapter containing dozens of pages.
Phase I: Identification of Books for In-depth Content Analysis
The analysis of each book began with counting the number of pages devoted to the general themes of the study in comparison to all the pages of the book, and in relation to the
number of chapters or pieces in the book (for example, a number of pieces in readers). All books were listed, identifying authors, year of publication, number of chapters and pages relevant to the study questions, and including a very short description indicating which study themes were present.
The books were classified to five levels of relevancy:
Level 0- Books that did not have any pages devoted to the themes of the study;
Level 1- books that had up to 10% of the pages devoted to the themes of the study;
Level 2- books that had between 11% up to 30% of the pages devoted to the themes of the study; Level 3- books that had between 31% up to 50% of the pages devoted to the themes of the study; Level 4- books that had between 51% up to 70% of the pages devoted to the themes of the study; Level 5- books that had over 70% of the pages devoted to the themes of the study.

Relevancy Ratings of Israeli and Palestinian Books: Total Books and Analyzed Books
Palestinian Books: Total
Palestinian Books: Analyzed
Israeli Books: Total
Israeli Books: Analyzed
Level 0
Level 1
49 (6 Waqf)
Level 2
Level 3
Level 4
Level 5
A total of 74 Israeli books and 94 Palestinian books were analyzed, yielding 2188 LPs from Israeli books and 960 from Palestinian books. Nearly all the Palestinian books were analyzed. Given the greater length of the Israeli books, focus was placed on books with highest relevancy (levels 3-5) and then some books from levels 1 and 2 were randomly selected for analysis. This produced large enough numbers of LPs from both Israeli and Palestinian books to ensure a very high level of reliability in the analysis. Among Israeli books, 21 were used in State Secular schools, 20 in State Religious schools, 18 in both State Secular and Religious schools and 15 in Ultra-Orthodox schools. Israeli research assistants analyzed approximately 1/3 of the Palestinian and 2/3 of the Israeli books. Palestinian research assistants analyzed approximately 1/3 of the Israeli and 2/3 of the Palestinian books.
Phase II: Content Analysis of the Textbooks
Highly specific sets of evaluation questions related to each study theme and an accompanying implementation manual were used to structure and standardize evaluations of the LPs. The six study themes were decided upon in a method development and review meeting of the leaders of the Scientific Research Team (SRT) and the Scientific Advisory Panel (SAP) in Jerusalem At the same time SAP members suggested specific questions to answer in evaluating passages relevant to each of the thematic areas of study. Professors Adwan and Bar-Tal then created initial rating forms with the specific evaluation questions and the instruction manual that accompanies them. The rating forms were created in English which is the common scientific language for Professors Adwan, Bar-Tal and Wexler, and the SAP. The forms were then translated into Arabic and Hebrew by the bilingual research team. The research team met together several times to practice using the forms and the manual, and evaluated LPs together to identify areas needing further clarification and modification. Over the same period, the forms and manual went through additional cycles of review and modification by the SAP before being finalized for use.
Initial questions provide a general description and context of the literary piece. These questions ask about the historical period and geographic location, the general Nature of the events (War; Violent confrontation/resistance; Occupation; Nonviolent Conflict; Neutral; Cooperation; Peaceful context), the relationship portrayed between the two groups (Very Violent; Violent; Competitive without violence; Neutral; Cooperative; Peaceful), whether there is reference to previous events or whether this is an isolated case, and whether larger political/societal processes and other parties are involved.
The next set of questions evaluates the ways in which the “other” is portrayed. In the Palestinian books, the other refers to the Jews in general, Jews in Israel, Israelis, Zionists, and Jews in other countries. In the Israeli books the other refers to Arabs in general, Muslims, Christians and Palestinians. Evaluation of material related to this primary study theme was structured by a series of specific queries. Who is described, how are they characterized and what are they doing? Are they individual(s), leader(s), group(s), nation? Do they have names or other identifying features? What is their gender and socioeconomic status? Are they characterized in terms of de-legitimization (of which dehumanization is one type), negative stereotyping, neutral presentation, positive presentation or very positive presentation? Are their acts described as evil, negative, neutral or positive? (This evaluation of every literary piece that contained the description of the other allowed later quantitative analysis). What is the purported intention of their acts and are they represented as victims, perpetrators or bystanders?
In order to provide an internal comparison or reference, all the queries about passages portraying the other were also asked regarding passages portraying the self-group. Since the purpose of this section was to provide a general comparison profile, and because LPs describing the self-collective were quite common, this section was completed on approximately the first half of the books analyzed only. In the Palestinian books presentation of the self-group refers to Palestinians, Arabs, Muslims, and Arab Christians. In the Israeli books presentation of the self- group refers to Jews, Israelis, and Zionists.
The next section evaluated LPs related to Christianity and Islam in Israeli books and Christianity and Judaism in Palestinian books. The general characterization from negative to positive as well as the degree of information provided about the beliefs, religious leaders, holidays and holy places of each religion were noted. The section on religions was followed by a section evaluating LPs about the conflict, with specific questions about the basis of the conflict. The next thematic category includes descriptions of peace, peace-making, conflict resolution and reconciliation. Specific queries include the possibility of peace, bases for peace, role of external entities and the nature of the possible peace (warm/cold, temporary/lasting). Research assistants were next asked to identify explicit references to values of the other community and of the self-community, including such things as respect for life, respect for the other, critical thinking, jihad, martyrdom, human rights, security, freedom, dignity and non-violence. The final sections of the analysis evaluate photographs, illustrations and maps.
Analysis was done in the language of the analyzed textbooks; i.e., using the form in the language of the book being analyzed independent of whether the rater was Israeli or Palestinian since the raters were fluent in both languages. Researchers entered quotes from the LPs to illustrate the basis of their rating of a LP. Quotes were then translated into English and Arabic or Hebrew as appropriate. The full set of quotes in three languages is provided in a separate document. Selected quotes are included in the body of this report to illustrate how the quantitative ratings were obtained and provide a qualitative aspect to the study report.
Phase II: Data Entry and Management
Research assistants entered their evaluations of LPs from their own work locations via the internet directly into a data base managed at Yale University by Professor Wexler and the data analysis team. The data base and entry system was developed for this project by American IT Solutions and can be interactively viewed by following the instructions at
Phase II: Inter-rater Reliability
Over 670 LPs in Stage Two were analyzed by two research assistants. The second rater evaluated the LPs identified by page numbers for analysis by the first rater, but had no idea what thematic areas had been identified as present in the LPs specified by the first rater or how the LP had been rated as per specific evaluation questions by the first rater. There was 93% agreement between raters on the decisions of which thematic areas were related to each LP. To evaluate overall inter-rater agreement on the ratings of individual LPs with ordered categorical answer choices along the numeric rating scales (e.g., very negative portrayal of the other, negative, neutral, positive, very positive; or characterization of the nature of an imagined peace as cold, neutral or positive), counts were done of the number of instances of complete agreement, the number of instances of the two raters providing adjacent ratings on the scale (e.g., very negative and negative) and the number of instances where ratings were more disparate. Overall, 63% of the time the two raters were in complete agreement, and 91% of the time they were in complete agreement or only one step apart (Figure 1). Figure 2 shows plots of each individual rater’s scores compared to all others with whom that individual co-rated LPs. Inter-rater reliability was similar for all members of the research team; none of the individual research assistants were outlier raters or different from all the others. A complete description of statistical methods and results of assessing inter-rater reliability is provided in Appendix E. The distribution of agreement scores for Israelis rating Israeli books compared to the ratings of Palestinians rating the same books, and vice versa, are presented in Figures 7 and 8 in Appendix E. Israelis tended to rate both sets of book very slightly more negatively than did Palestinians. There was also a very slight tendency for each group to rate their own books more positively than the “other’s” books. Both of these differences are so small that they could be due to chance alone and have no meaningful impact on the overall assessments.

To read entire document, click here

No comments: