library(reticulate)
library(dplyr)
library(purrr)
library(tidyr)
library(stringr)
library(ggplot2)LLMs for Political Text with R
A tutorial for R and reticulate with Gemini and ChatGPT
Artificial Intelligence
Data Visualization
Text as Data
This tutorial shows how to connect to LLMs via APIs in R with the reticulate package.
Introduction
LLMs are everywhere now! Increasingly they are being used in social science research and can help scholars and students annotate texts much faster and cheaper than humans (Laurer et al. 2024). However, connecting to these models via API natively from R is challenging. Here I show a basic framework for classifing text and getting structured output from two of the most popular LLMs, ChatGPT and Google’s Gemini. This tutorial assumes a working knowledge of R and some familiarity reticulate package for interfacing with Python APIs.
To begin, I load the appropriate Python environment using reticulate::use_condaenv("gptenv", required = TRUE), which ensures that my code runs inside a Conda environment where the required Python packages are installed. The gemini_key is used to authenticate requests to Google’s Gemini API, while the openai_key is used for accessing OpenAI’s GPT models. Replace the text with your own keys. Each call costs money so make sure to never share/show them!
use_condaenv("gptenv", required = TRUE)
gemini_key <- "[YOUR-KEY-HERE]"
openai_key <- "[YOUR-KEY-HERE]"Custom Functions
To run structured prompts through both OpenAI and Google models, I wrote two helper functions: gpt_get_structured_output() and gemini_get_structured_output(). Each function takes a string (in this case a sentence) and a shared prompt, sends them to the respective API using Python code embedded via reticulate::py_run_string(), and returns the model’s text output. I also use safely() wrappers (safe_gpt and safe_gemini) to ensure that if a particular sentence fails (due to API rate limits or formatting issues), the rest of the pipeline continues.
After collecting the model responses, I parse them into structured key–value pairs using parse_kv_block(), which extracts results like framing: justice into a clean named list.
# Sends a prompt and input text to OpenAI's GPT-3.5 and returns the structured output.
gpt_get_structured_output <- function(text,
prompt,
api_key) {
# Concatenate the system prompt and user input
ai_prompt <- prompt
# Construct and run Python code that uses the OpenAI client API (v1.0+)
py_run_string(paste0(
"import openai\n",
"client = openai.OpenAI(api_key='", api_key, "')\n",
"sys_prompt = '''", ai_prompt, "'''\n",
"user_prompt = '''", text, "'''\n",
"response = client.chat.completions.create(\n",
" model = 'gpt-3.5-turbo',\n",
" temperature = 0,\n", # deterministic output
" messages = [\n",
" { 'role': 'system', 'content': sys_prompt },\n",
" { 'role': 'user', 'content': user_prompt }\n",
" ]\n",
")\n",
"result = response.choices[0].message.content\n"
))
# Return the raw text output from GPT
py$result
}
# Wrap the GPT function in a safely() wrapper to handle errors gracefully
safe_gpt <- safely(gpt_get_structured_output)
# Sends a prompt and text to Google’s Gemini 1.5 Pro model and returns the structured result.
gemini_get_structured_output <- function(text,
prompt,
api_key) {
# Combine the prompt and user input into one string
ai_prompt <- paste0(
prompt,
text
)
# Construct and run Python code that uses the Gemini client
py_run_string(paste0(
"import google.generativeai as genai\n",
"genai.configure(api_key='", api_key, "')\n",
"model = genai.GenerativeModel('models/gemini-1.5-pro-latest')\n",
"response = model.generate_content(\"\"\"\n", ai_prompt, "\n\"\"\")\n",
"result = response.text\n"
))
# Return the raw text output from Gemini
py$result
}
# Wrap the Gemini function in a safely() wrapper to handle errors gracefully
safe_gemini <- safely(gemini_get_structured_output)
# Parses key-value outputs into a named list (e.g. `multilateralism: true`)
parse_kv_block <- function(text_block) {
# Split text into lines
lines <- stringr::str_split(text_block, "\n")[[1]]
# Extract key-value pairs using a regular expression
kv <- stringr::str_match(lines, "^([a-zA-Z_]+):\\s*(.*)$")
# Identify valid rows where both key and value are present
valid <- which(!is.na(kv[,2]) & !is.na(kv[,3]))
# Return as a named list
out <- setNames(kv[valid, 3], kv[valid, 2])
as.list(out)
}Prompt Design
The prompt is designed to extract structured metadata from sentences drawn from UN General Assembly speeches (Baturo, Dasandi, and Mikhaylov 2017).1 Specifically, we ask the models to assess whether a sentence is about multilateralism or multipolarity, and if it uses a normative framing (justice, injustice) or not (neither, unknown). I explicitly tell the model not to explain itself—just to return these labels, one per line, exactly in the format I specify. This makes it easy to parse the output later using a regular expression (regex) in R.
The same prompt will be passed to both models. The output is parsed into structured key–value pairs for later comparison.
prompt <- paste0(
"You are reading sentences from the UN General Assembly speeches. ",
"Infer if the sentence is about systemic change in the context of multilateralism or multipolarity. ",
"Return only the following key-value pairs, each on a new line:\n",
"multilateralism: true or false\n",
"multipolarity: true or false\n",
"framing: justice, injustice, or neither\n",
"If the sentence is not about international relations, set framing to 'unknown'. ",
"Do not return any explanation or formatting. Only the two lines of key-value pairs.\n\n"
)Examples
Let’s try out our custom functions and prompt with some example text:
gemini_get_structured_output("The BRICs countries gaining ground in global affairs.", prompt, gemini_key)[1] "multilateralism: false\nmultipolarity: true\nframing: neither\n"gpt_get_structured_output("The BRICs countries gaining ground in global affairs.", prompt, openai_key)[1] "multilateralism: false\nmultipolarity: true\nframing: neither"And with a nonsensical sentence to see what it does:
gemini_get_structured_output("Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam.", prompt, gemini_key)[1] "multilateralism: false\nmultipolarity: false\nframing: unknown\n"gpt_get_structured_output("Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam.", prompt, openai_key)[1] "framing: unknown\nmultilateralism: false"Raw Text Data
| sentence_id | sentence |
|---|---|
| MNG_71_2016_57 | We urge multilateral institutions to take the lead in those important efforts. |
| SGP_77_2022_60 | The United Nations-led multilateral system will be key in pulling everyone together to row in the same direction and not leave anyone behind. |
| JPN_17_1962_85 | On the problem of trade, we should-remind ourselves that the General Agreement on Tariffs and Trade has been making, since its inception, major contributions to the furtherance of free trade on a multilateral, non-discriminatory basis. |
| KOR_68_2013_3 | The Republic of Korea is pleased that the United Nations, in partnership with its Member States and under the stewardship of Secretary-General Ban Ki-moon, is strengthening multilateralism by successfully responding to the diverse challenges that the international community faces. |
| VNM_71_2016_23 | We are seeing multilateral institutions expand, both in numbers and in strength. |
| TLS_72_2017_48 | Only through dialogue, negotiation, multilateral cooperation and respect for democracy can we meet the ambitious goals we have established for 2030. |
| MYS_44_1989_107 | The international economic front, very few encouraging steps have been taken to achieve the stated goals of multilateralism, interdependence and mutual co-operation. |
| TUV_75_2020_17 | Concretizing the Multilateral response in a UN we need and a future we want. |
| NRU_75_2020_65 | We need a global financial system that is more responsive to the urgent needs of developing countries, and I strongly urge you to take on this issue as part of your effort to reaffirm multilateralism. |
| NZL_33_1978_61 | I recognize that the principles of an open multilateral trading system raise difficult domestic political problems, but economic development is about change and innovation and we need to look at today's problems with our eyes very firmly fixed on the longer term. |
| MNG_71_2016_156 | We reaffirm our firm commitment to the multilateral system with the United Nations at its core. |
| VUT_76_2021_13 | This demands a reaffirmation of faith in multilateralism since no national capabilities, however large, can on their own be adequate to effect meaningful change or transition. |
| KHM_74_2019_79 | I would like to conclude by sharing our belief that in a multipolar world, multilateralism can succeed only if it rejects all forms of interference. |
| KIR_77_2022_43 | However, the lack of solidarity even through multilateralism continues to be the stumbling block to addressing the global climate change emergency. |
| PLW_74_2019_38 | We also look forward to the successful conclusion of negotiations and the adoption of a new treaty on high seas biodiversity that will set the multilateral framework for us to effectively protect the high seas and complement our national efforts. |
| SGP_63_2008_96 | In fact, smaller countries can turn the emerging multipolarity to advantage if we combine our strengths in regional and international institutions. |
| MNG_45_1990_97 | We have undertaken the work of bringing our national legislation into conformity with our international obligations and of withdrawing reservations made earlier in regard to some multilateral treaties and conventions. |
| LAO_36_1981_64 | Bilateral or multilateral cooperation among the two groups of countries of IndoChina and ASEAN, and also other countries of the region with those outside the region, must in no case prejudice the security and interests of other countries of the area, nor should it be directed against any other country. |
| IDN_77_2022_74 | It is why we need a renewed multilateralism that is fit for purpose and fit for its time. |
| IDN_49_1994_40 | It is essential that serious and concerted efforts be undertaken in the Conference on Disarmament in conjunction with the broader multilateral endeavours in the Amendment Conference on the partial test-ban Treaty to ensure the conclusion of a universal and effectively verifiable treaty within a fixed time-frame. |
| NLD_43_1988_114 | The spectacular emergence of new economic centres in various parts of the third world is only one indication that we are witnessing a development towards greater multipolarity. |
| FRA_74_2019_75 | That is not to say that our multilateralism is worn out, that we no longer listen to ourselves or that we are no longer effective. |
| RUS_66_2011_19 | Its goal is to enhance productive multilateral collaboration to address the urgent problems of the contemporary world. |
| ITA_62_2007_34 | It is only through multilateralism, by marshalling everyone’s energies, that we can hope to do good. |
| TUR_32_1977_133 | The principles enshrined in the Charter must always guide our actions in our bilateral and multilateral relations. |
| CYP_74_2019_8 | For some, alternatives to multilateralism may seem attractive. |
| AUT_74_2019_61 | At the outset, I shared with the Assembly my concern about the state of multilateralism. |
| HRV_72_2017_84 | Croatia’s commitment to multilateralism is unwavering. |
| DEU_56_2001_121 | The future belongs to responsible governance for one world, governance based not on hegemonic claims, but on cooperation, solidarity and multilateralism. |
| HUN_23_1968_105 | Our co-operation with the Czechoslovak Socialist Republic, the consolidation and strengthening of our common international activity will create a new and more solid basis for bilateral and multilateral negotiations with a view to building up a real and peaceful system of security in Europe. |
| ROU_28_1973_106 | Romania itself follows the process of developing and modernizing its national economy The Romanian people make consistent efforts for the building up of a multilaterally developed society through the more effective use of the country's human and material potential for the acceleration of the development and modernization of the productive forces, the improvement of the management and organization of the society, the deepening of democracy and the laying of new premises for raising the standards of life and civilization of the whole people. |
| AND_76_2021_1 | Mr President, Mr Secretary General, Your Excellencies, Ladies and Gentlemen, I would like to begin by congratulating His Excellency Mr Abdulla Shahid on his election as President of this General Assembly and His Excellency Mr Antonio Guterres on his reappointment for a second term as Secretary-General of the United Nations (UN), this great assembly of all peoples and for all peoples, whose main objective is to build a system of values and a code of conduct based on multilateralism and cooperation. |
| LIE_65_2010_14 | Past experience has shown time and again that multilateral action can be very effective when it is based on a broad political consensus, which is not the same as giving everyone a right to veto. |
| NLD_75_2020_32 | Critical evaluation and continuous efforts to improve are key elements of multilateral cooperation. |
| AUT_64_2009_2 | We have all witnessed something new and different: a genuine opportunity for a real renaissance of multilateralism. |
| PRT_42_1987_53 | The climate of dialogue achieved during the conference allowed for a sound analysis of multilateral economic co-operation. |
| ITA_76_2021_57 | The fight against climate change requires multilateral engagement and pragmatic cooperation among all major global players — both rich and emerging economies. |
| DEU_59_2004_43 | To make this multilateral cooperation sustainable and capable, we need a courageous and comprehensive reform of the United Nations that faces up to the challenges we are facing. |
| AND_72_2017_85 | That is why much of our foreign policy is focused on multilateral forums, as we showed four years ago during the Andorran presidency of the Council of Europe. |
| DEU_63_2008_63 | My proposal to multilateralize the fuel cycle showed how those risks can at the very least be minimized. |
| CRI_62_2007_64 | Neither the chronic pessimism of some nor the ungrateful egoism of others will check the slow but determined march forward of those of us who believe in multilateralism and in a future of greater shared welfare for all. |
| BRA_63_2008_46 | Its distorted form of representation stands between us and the multilateral world to which we aspire. |
| PER_73_2018_41 | We will continue to promote initiatives to help restore democracy in our sister country within the framework of the Organization of American States, the Lima Group and in other multilateral forums. |
| URY_35_1980_86 | Now we must go from words to action and allocate the necessary funds, negotiate bilateral and multilateral financial aid and at the same time offer UNEP and the International Union for Conservation of Nature and Natural Resources the sustained support that will permit the practical and progressive adoption of their plans. |
| JAM_70_2015_31 | The next major step for multilateralism will come in a few weeks when we hope to conclude negotiations in Paris on a global agreement on climate change. |
| BRA_64_2009_20 | No one is yet clearly willing to confront serious distortions of the global economy in the multilateral arena. |
| LCA_77_2022_6 | We have arrived at this watershed moment in history because, we, the Members of the United Nations, have not adhered to the rules and principles of the Organization that we created seventy-six years ago as a multilateral answer to the human propensity to use arms against our fellow humans, instead of joining with them to turn those weapons into tools for peace and development. |
| ARG_61_2006_21 | We believe that, in order to face this criminal threat successfully, we must carry out a sustained multilateral and legitimate response. |
| JAM_49_1994_7 | This is an exciting and daunting task, particularly for a small island State like Jamaica, whose faith and commitment to multilateralism and the United Nations remain unwavering. |
| GTM_64_2009_4 | My presence highlights our commitment to multilateralism in general and to the United Nations in particular. |
| JAM_58_2003_16 | These are real concerns that underscore the need to strengthen 2 multilateralism, to restore confidence in the United Nations system, to buttress its centrality in decisions that affect us all and to thereby enhance its capacity to enforce. |
| COL_76_2021_1 | We meet again in this global forum, which has historically been a vital space for the development of multilateralism, peacebuilding and responses to threats to our shared home, against the backdrop of a cruel pandemic that is battering our health systems, our economies, our gains in equity and progress on fulfilling the 2030 Agenda for Sustainable Development. |
| KNA_49_1994_116 | Above all, the concept of the vulnerability index must become part of the economic language and armamentarium of all multilateral and national development institutions. |
| CHL_77_2022_80 | We need a modernized United Nations, where we all have the same goals, Based on multilateralism, justice and peace at all times and in all places, we must commit to taking the necessary actions, not just making statements, so as to stop Russia’s unjust war on Ukraine and put an end to all abuses by the powerful everywhere in the world. |
| SLV_25_1970_59 | Moreover, the principles and norms the violation of which constitute aggression should be regarded as part of a system, and in this sense such a definition cannot be taken to mean the indication of limits for isolated cases; rather the definition must recognize the multilateral conditioning of the major international principles. |
| GTM_76_2021_76 | I also draw the Assembly’s attention to the fact that Taiwan could provide its experience, capacities and knowledge to the strengthening of multilateralism, taking into account the challenges we face today. |
| CHL_70_2015_26 | We are calling for an architecture of multilateral institutions that will provide backing for the national policies proposed under the 2030 Agenda. |
| PAN_44_1989_36 | Persecution at the international level, not excluding multilateral agencies of which we are a member, is intended to put an end to national efforts to achieve economic recovery and meet the basic needs of the Panamanian people. |
| BRA_56_2001_51 | In the field of trade, it is high time that multilateral negotiations were translated into greater access for goods from developing countries to the more prosperous markets. |
| DOM_52_1997_35 | Also, together with other countries, it is formulating strategies to take advantage of funds from multilateral sources. |
| MLT_64_2009_63 | In his report on the work of the Organization, the Secretary-General has laid emphasis on the need to embark on a multilateral effort of immense magnitude “that draws upon the strengths and contributions of all the countries of the world, as well as their citizens” (A/64/1, para. |
| DJI_70_2015_25 | We will have the opportunity to return to this, but allow me now to address the implications of the macroeconomic fragility in the global economy, the many and varied challenges facing our country and the importance of establishing a credible and beneficial multilateral trading system for all. |
| DZA_73_2018_47 | The rejection of policies based on force, with the dangers that they entail, requires that we constantly seek, through dialogue and consensus, to strengthen multilateral action. |
| BHR_71_2016_71 | We still hear the same irresponsible sectarian discourse from Iran and have witnessed the damage done to our bilateral and multilateral relations. |
| MAR_43_1988_72 | At the same time, we must take advantage of the forthcoming multilateral talks, especially the Uruguay Round, to improve the international economic environment in the spheres of the flow of .net resources, trade and the stabilization and rising of commodity prices. |
| DZA_32_1977_140 | Real reform of the international monetary system: consideration of the legitimate interests of the developing countries in multilateral trade negotiations: the creation of a joint commodity fund; the adoption of a code of conduct on the transfer of technology; the solution of the problems of indebtedness of the developing countries; the promotion of the industrialization of the third world -this list is not exhaustive—are so many objectives which the international community must set for itself within a reasonable period of time. |
| MAR_74_2019_54 | Multilateral action is not just an organizational structure of international relations; it is also a state of mind. |
| QAT_74_2019_74 | Despite the different situations and circumstances among countries, we need international cooperation today more than ever and must ensure the credibility of multilateral action to counter the challenges of climate change. |
| DZA_64_2009_12 | Multilateral institutions, also undermined by contradictions and a lack of coherence, are unable to overcome impasses in negotiations on vital questions that have a direct impact on our populations. |
| MLT_38_1983_36 | Many countries have today recognized and expressed support for our status of neutrality, both at the bilateral level and in the multilateral forums of the Movement of Non-Aligned Countries, the Commonwealth and the Conference on Security and Co-operation in Europe. |
| MLT_76_2021_3 | If anything, COVID only reaffirmed the priority of multilateralism, and that it is the key to a strong and sustainable future. |
| MAR_73_2018_66 | The multilateral system, despite the criticism it faces, will remain essential to addressing the challenges and issues facing the international community. |
| MAR_76_2021_22 | The COVID-19 pandemic has shown that there is an urgent need for pragmatic multilateral action whose legitimacy is rooted in the effective realization of the rights of citizens to security, health and development. |
| QAT_76_2021_60 | Qatar’s bet on international institutions and multilateral cooperation is a strategic one. |
| BHR_61_2006_69 | Today, all of us need to join in efforts to achieve the objective of a collective security system through multilateral diplomacy, for which the United Nations is the main arena. |
| KWT_48_1993_15 | Kuwait welcomes the accession to United Nations membership by Eritrea, the Czech Republic, the Slovak Republic, Monaco, Andorra and The Former Yugoslav Republic of Macedonia and trusts that their accession to the Organization’s membership will consolidate further the efficacy and universality of multilateralism. |
| IRQ_68_2013_72 | While the United Nations is our agreed forum for international cooperation to address the problems and challenges that we face through multilateral negotiations, we believe that the time has come to reform its bodies, in particular the Security Council, as the primary body responsible for international peace and security, in order to make them consistent with the aspirations of the peoples of the world in the twenty- first century and more representative, transparent and able to meet the challenges they face. |
| SAU_61_2006_7 | The world today faces a host of global problems that can be successfully addressed only within the context of multilateral cooperation under the umbrella of the United Nations. |
| OMN_66_2011_2 | We are confident that his diplomatic experience, especially in the field of multilateral diplomacy, will have a great impact on the success of this session. |
| MAR_76_2021_23 | Mr President, The bid to deliver vaccines, our last line of defence against the pandemic, is in fact an opportunity to assert our will and demonstrate our ability to infuse multilateral action with a new impetus. |
| USA_41_1986_121 | We applaud the success of the meeting of the General Agreement on Tariffs and Trade (GATT) trade ministers last week in Uruguay, where agreement was reached to launch a new round of multilateral trade negotiations covering a wide range of topics important to economic growth, with over 90 other countries members of GATT, the United States is working to maintain the free flow of international trade. |
| USA_15_1960_58 | We should also look forward to appropriate and timely financial assistance from these two multilateral financial sources as the emerging countries qualify for their aid. |
| CAN_39_1984_49 | We shall encourage super-Power and multilateral discussion on all outer space weapons and shall commission further studies on how a space-weapons ban might be verified. |
| CAN_74_2019_12 | Strengthening multilateralism means involving all stakeholders. |
| USA_63_2008_34 | Other multilateral organizations have spoken clearly as well. |
| CAN_39_1984_65 | In his most recent report he asks this: "Why has there been a retreat from internationalism and multilateralism at a time when actual developments both in relation to world peace and to the world economy would seem to demand their strengthening?" |
| USA_34_1979_132 | In addition, it could recommend how current institutions could be strengthened and whether new multilateral ones should be created. |
| CAN_74_2019_75 | We are fully committed to bringing innovative ideas to prevent and respond to crises, and we firmly believe in the power of multilateral solutions and partnership to address the most intractable of global challenges. |
| CAN_76_2021_80 | However, the benefits of multilateralism have not always been evenly or fairly distributed, and the potential for peace is yet to be realized in far too many places. |
| USA_63_2008_15 | Multilateral organizations have responsibilities. |
| CAN_49_1994_35 | As the leader of the multilateral Working Group on Refugees, we are tackling with determination that task entrusted to us by the international community. |
| CAN_40_1985_93 | Moreover we shall work towards the conclusion of a multilateral agreement prohibiting the possession and use of radiological weapons. |
| CAN_40_1985_31 | The challenges facing the multilateral system reach beyond this institution. |
| CAN_56_2001_19 | We in Canada, as long- standing proponents of multilateralism and the United Nations system, have strongly welcomed the close collaboration evident between the United States Administration and the United Nations over these last two months. |
| USA_20_1965_212 | We would be prepared to increase the amount of capital flowing through multilateral channels. |
| CAN_38_1983_194 | They have not lost faith in the multilateral institutions we have so painstakingly constructed. |
| CAN_59_2004_32 | It is always preferable to have multilateral authority for intervention in the affairs of a sovereign State. |
| CAN_40_1985_125 | Bilateral relations between countries on the one hand, and the multilateral system on the other, ought to be mutually complementary and reinforcing. |
| USA_33_1978_43 | The facilities of IMF have been expanded, and discussions are under way to expand the facilities of the multilateral development banks. |
| CAN_49_1994_102 | Canada is ready to proceed with a comprehensive re- evaluation of its multilateral system as it applies to economic and social issues. |
| PAK_60_2005_108 | It is in the same spirit of constructive engagement that Pakistan wishes to pursue cooperative multilateralism and to strengthen the United Nations — this unique forum that is indispensable, in our interdependent world, for all States, large or small, powerful or weak. |
| AFG_41_1986_34 | The imperialist policy of using economic assistance as a means of exerting political pressure on the developing countries is no longer confined within the limits of bilateral relations; it has also been extensively employed to undermine the capabilities of multilateral institutions, with a view to curtailing or completely stopping the flow of international development assistance to developing countries, particularly those that dare refuse to submit to imperialist diktat. |
| PAK_21_1966_212 | The purpose of the conference, among other things, would be to consider, first, how the security of States without nuclear weapons can best be assured, preferably through multilateral guarantees; secondly, how they may co-operate among themselves in preventing further proliferation of nuclear weapons; and thirdly, the development and use of nuclear energy for exclusively peaceful purposes, through mutual co-operation and for mutual benefit, including the carrying out of explosions of nuclear devices for peaceful purposes under appropriate international supervision. |
| LKA_32_1977_110 | Existing national insurance agencies in developed countries could pool their risks through a reinsurance agency that might be set up on a multilateral basis. |
| BGD_49_1994_168 | Bangladesh looks forward to consolidating our relations with South Africa in more concrete ways, bilaterally and in all multilateral forums. |
| PAK_27_1972_102 | Pakistan attaches the highest priority to both bilateral and multilateral efforts for disarmament. |
| PAK_66_2011_23 | It is the United Nations and multilateralism that will safeguard that future. |
| BTN_67_2012_56 | As a small State, we have always attached the highest importance to multilateralism and the primacy of an effective United Nations that serves the interests of all its Member States. |
| AFG_64_2009_9 | In our increasingly interdependent world and the multilaterally oriented international system, the United Nations must assume greater responsibility for finding collective solutions to our challenges. |
| LKA_47_1992_81 | The successful conclusion of the convention on chemical weapons demonstrates the competence of the United Nations in multilateral negotiations. |
| LKA_77_2022_12 | In multilateralism, we talk to each other, we develop a relationship of trust and confidence and, if something were to come up, you have the base to work from. |
| NPL_32_1977_77 | Similarly, there has been little progress in the multilateral trade negotiations in GATT on the reduction of trade barriers of developed countries, which restrict the exports of primary and processed commodities from developing countries. |
| IND_36_1981_154 | Total bilateralism at the expense of multilateral cooperation would run into alliances that may not be in the interest of harmonious relations and relaxation of tensions. |
| PAK_71_2016_4 | Today, three decades after the end of the Cold War, our multipolar world is more free and vibrant, yet still chaotic and turbulent; more interdependent, but more unequal; more prosperous, yet still afflicted by poverty. |
| PAK_66_2011_13 | Each September we return to this great city and this grand stage so that we can restate and reaffirm the principles and values of multilateralism. |
| PAK_59_2004_94 | Today, there is welcome resurgence of support for multilateralism. |
| MDV_68_2013_63 | SIDS need to be given full recognition within the global governance regimes and multilateral and financial institutions, and adequately integrated and institutionalized within the United Nations system. |
| NPL_37_1982_17 | International economic co-operation and the system of multilateral co-operation continue to erode at an alarming rate. |
| BTN_38_1983_94 | Recently there has been much concern expressed at the deterioration of the resource position of some multilateral development organizations, UNDP in particular. |
| PAK_53_1998_146 | This suggests the need for concerted action by the international community, which should include, first, the 14 strengthening of the capacity of multilateral institutions to address the issues of trade, finance and development in an integrated and coherent manner, thereby ensuring the effective governance of globalization. |
| CMR_74_2019_81 | This occasion gives me the opportunity to warmly acknowledge the contribution of bilateral and multilateral partners that have always reaffirmed Cameroon’s unity and territorial integrity. |
| CMR_39_1984_157 | As a corollary, multilateralism is dangerously declining; there are clear signs of this, mainly the current decrease in the resources made available to institutions such as UNDP, the fact that many States, particularly the more powerful ones, do not hasten to resort to the machinery offered by the United Nations to resolve issues of world-wide interest and, to some extent, the difficulties experienced now by UNESCO. |
| ZWE_45_1990_61 | Zimbabwe therefore wishes to appeal to the international community and to the various donors and multilateral economic institutions for support of our national efforts. |
| ZWE_52_1997_7 | Africa has consistently underlined the need for reform of the United Nations and other multilateral bodies, to promote the democratization and effectiveness of the international decision-making process. |
| ZWE_73_2018_40 | My country remains committed to strengthening multilateralism and the peaceful resolution of differences. |
| SWZ_63_2008_61 | The Kingdom of eSwatini reaffirms its commitment to the purpose and preservation of the central role of the United Nations in multilateral affairs. |
| SOM_74_2019_6 | In this globalized world, no nation, no matter how wealthy, strong or prepared, can individually stand alone against the tide of global challenges, which requires common action and coordinated multilateral responses. |
| ZMB_72_2017_5 | The theme for this session, “Focusing on people: Striving for peace and a decent life for all on a sustainable planet”, provides us with an opportunity to evaluate the existing multilateral approaches to address the challenges affecting our peoples. |
| CPV_71_2016_10 | As a small island State, Cabo Verde bases its foreign policy on the principles enshrined in the Charter of the United Nations and is certain that multilateralism is the most appropriate way to approach issues on the international agenda. |
| SEN_75_2020_44 | Furthermore, in implementing the Declaration on the commemoration of the seventy-fifth anniversary of the United Nations, which Senegal fully endorses, we must renew our commitment to the agenda for UN reform in order to restore multilateralism to its rightful place with the United Nations at its centre. |
| LSO_59_2004_75 | That must be done against the background of multilateralism, upon which international peace and security is premised. |
| GNQ_77_2022_7 | Equatorial Guinea is today launching an appeal on the need and importance of enhancing multilateralism and international cooperation, which are so necessary to address those global challenges and to revive development and sustainable economic growth in our countries, especially developing countries. |
| BFA_76_2021_7 | The strategic vision you presented for your second term confirms your commitment and determination to work towards strengthening multilateralism and the United Nations in solving the multiple problems facing our world, especially in these difficult times caused by the coronavirus disease (COVID-19) pandemic. |
| KEN_77_2022_104 | It is time for multilateralism to reflect the voices of the farmers, represent the hopes of villagers, champion the aspirations of pastoralists, defend the rights of fisherfolk, express the dreams of traders, respect the wishes of workers and indeed protect the welfare of all the peoples of the global South. |
| GIN_50_1995_90 | The United Nations and its specialized agencies, as the paramount instrument for cooperation and multilateral agreement, see their responsibilities increased today more than ever before. |
| GIN_28_1973_13 | The problem of international peace and security, the defence of national sovereignties, decolonialization and the struggle against imperialism continue to be the focus of both bilateral and multilateral meetings. |
| GNB_75_2020_44 | We cannot talk about multilateralism while allowing the economic and financial embargo to continue to be imposed on one of the most supportive members of our Organization — Cuba. |
| SLE_57_2002_57 | Never since the end of the Second World War has multilateral cooperation become such a necessary means for resolving international disputes and for ensuring the well-being of people everywhere. |
| SLE_67_2012_70 | We are confident that with sustained support from our bilateral and multilateral partners, we shall promote socioeconomic progress and provide a better standard of living for our people in an atmosphere of peace and security. |
| ZMB_44_1989_9 | We meet today against a background of great hopes and expectations for the success of multilateralism, which is so vital for the well-being of mankind. |
Apply Functions to Sentence Data Frame
In this step, I apply the GPT and Gemini models to each sentence in my dataset using purrr::map(). The safe_gpt and safe_gemini functions call the APIs with a shared prompt and safely handle any errors. I extract the results using map_chr(), replacing failures with "FAILED". Finally, I drop the intermediate result columns to keep the dataframe clean. Depending on the size of the corpus, this might take a long time!
classified_df <- df %>%
mutate(
gpt_result = map(sentence, ~ safe_gpt(.x, prompt, openai_key)),
gpt_output = map_chr(gpt_result, ~ ifelse(is.null(.x$error),
.x$result,
"FAILED")),
gemini_result = map(sentence, ~ safe_gemini(.x, prompt, gemini_key)),
gemini_output = map_chr(gemini_result, ~ ifelse(is.null(.x$error), .
x$result,
"FAILED"))
) %>%
select(-gpt_result, -gemini_result)Parse the Output into Columns
Next, I parse the structured outputs from both models using parse_kv_block() and expand them into separate columns with unnest_wider(), adding _gpt and _gemini suffixes. I then remove the raw output columns and create a new variable, framing_overlap, which captures agreement between models on the framing label or marks it as "conflict" when they differ.
classified_df <- classified_df %>%
mutate(
parsed_gpt = map(gpt_output, parse_kv_block),
parsed_gemini = map(gemini_output, parse_kv_block)
) %>%
unnest_wider(parsed_gpt, names_sep = "_gpt") %>%
unnest_wider(parsed_gemini, names_sep = "_gemini") %>%
select(-gpt_output, -gemini_output) %>%
mutate(
framing_overlap = case_when(
parsed_gemini_geminiframing == parsed_gpt_gptframing ~ parsed_gemini_geminiframing,
TRUE ~ "conflict"
)
)Classified (Parsed) Data
| sentence_id | sentence | parsed_gpt_gptmultilateralism | parsed_gpt_gptmultipolarity | parsed_gpt_gptframing | parsed_gemini_geminimultilateralism | parsed_gemini_geminimultipolarity | parsed_gemini_geminiframing | framing_overlap |
|---|---|---|---|---|---|---|---|---|
| MNG_71_2016_57 | We urge multilateral institutions to take the lead in those important efforts. | true | false | justice | true | false | neither | conflict |
| SGP_77_2022_60 | The United Nations-led multilateral system will be key in pulling everyone together to row in the same direction and not leave anyone behind. | true | false | justice | true | false | justice | justice |
| JPN_17_1962_85 | On the problem of trade, we should-remind ourselves that the General Agreement on Tariffs and Trade has been making, since its inception, major contributions to the furtherance of free trade on a multilateral, non-discriminatory basis. | true | false | justice | true | false | neither | conflict |
| KOR_68_2013_3 | The Republic of Korea is pleased that the United Nations, in partnership with its Member States and under the stewardship of Secretary-General Ban Ki-moon, is strengthening multilateralism by successfully responding to the diverse challenges that the international community faces. | true | false | justice | true | false | neither | conflict |
| VNM_71_2016_23 | We are seeing multilateral institutions expand, both in numbers and in strength. | true | false | neither | true | false | neither | neither |
| TLS_72_2017_48 | Only through dialogue, negotiation, multilateral cooperation and respect for democracy can we meet the ambitious goals we have established for 2030. | true | false | justice | true | false | neither | conflict |
| MYS_44_1989_107 | The international economic front, very few encouraging steps have been taken to achieve the stated goals of multilateralism, interdependence and mutual co-operation. | true | false | justice | true | false | injustice | conflict |
| TUV_75_2020_17 | Concretizing the Multilateral response in a UN we need and a future we want. | true | false | neither | true | false | neither | neither |
| NRU_75_2020_65 | We need a global financial system that is more responsive to the urgent needs of developing countries, and I strongly urge you to take on this issue as part of your effort to reaffirm multilateralism. | true | false | justice | true | false | justice | justice |
| NZL_33_1978_61 | I recognize that the principles of an open multilateral trading system raise difficult domestic political problems, but economic development is about change and innovation and we need to look at today's problems with our eyes very firmly fixed on the longer term. | true | false | justice | true | false | neither | conflict |
| MNG_71_2016_156 | We reaffirm our firm commitment to the multilateral system with the United Nations at its core. | true | false | neither | true | false | neither | neither |
| VUT_76_2021_13 | This demands a reaffirmation of faith in multilateralism since no national capabilities, however large, can on their own be adequate to effect meaningful change or transition. | true | false | justice | true | false | neither | conflict |
| KHM_74_2019_79 | I would like to conclude by sharing our belief that in a multipolar world, multilateralism can succeed only if it rejects all forms of interference. | true | true | justice | true | true | justice | justice |
| KIR_77_2022_43 | However, the lack of solidarity even through multilateralism continues to be the stumbling block to addressing the global climate change emergency. | true | false | justice | true | false | injustice | conflict |
| PLW_74_2019_38 | We also look forward to the successful conclusion of negotiations and the adoption of a new treaty on high seas biodiversity that will set the multilateral framework for us to effectively protect the high seas and complement our national efforts. | true | false | justice | true | false | neither | conflict |
| SGP_63_2008_96 | In fact, smaller countries can turn the emerging multipolarity to advantage if we combine our strengths in regional and international institutions. | false | true | neither | true | true | neither | neither |
| MNG_45_1990_97 | We have undertaken the work of bringing our national legislation into conformity with our international obligations and of withdrawing reservations made earlier in regard to some multilateral treaties and conventions. | true | false | justice | true | false | justice | justice |
| LAO_36_1981_64 | Bilateral or multilateral cooperation among the two groups of countries of IndoChina and ASEAN, and also other countries of the region with those outside the region, must in no case prejudice the security and interests of other countries of the area, nor should it be directed against any other country. | true | false | justice | true | false | neither | conflict |
| IDN_77_2022_74 | It is why we need a renewed multilateralism that is fit for purpose and fit for its time. | true | false | justice | true | false | neither | conflict |
| IDN_49_1994_40 | It is essential that serious and concerted efforts be undertaken in the Conference on Disarmament in conjunction with the broader multilateral endeavours in the Amendment Conference on the partial test-ban Treaty to ensure the conclusion of a universal and effectively verifiable treaty within a fixed time-frame. | true | false | justice | true | false | justice | justice |
| NLD_43_1988_114 | The spectacular emergence of new economic centres in various parts of the third world is only one indication that we are witnessing a development towards greater multipolarity. | false | true | neither | false | true | neither | neither |
| FRA_74_2019_75 | That is not to say that our multilateralism is worn out, that we no longer listen to ourselves or that we are no longer effective. | true | false | neither | true | false | neither | neither |
| RUS_66_2011_19 | Its goal is to enhance productive multilateral collaboration to address the urgent problems of the contemporary world. | true | false | justice | true | false | neither | conflict |
| ITA_62_2007_34 | It is only through multilateralism, by marshalling everyone’s energies, that we can hope to do good. | true | false | justice | true | false | justice | justice |
| TUR_32_1977_133 | The principles enshrined in the Charter must always guide our actions in our bilateral and multilateral relations. | true | false | justice | true | false | neither | conflict |
| CYP_74_2019_8 | For some, alternatives to multilateralism may seem attractive. | true | false | neither | true | false | neither | neither |
| AUT_74_2019_61 | At the outset, I shared with the Assembly my concern about the state of multilateralism. | true | false | neither | true | false | neither | neither |
| HRV_72_2017_84 | Croatia’s commitment to multilateralism is unwavering. | true | false | neither | true | false | neither | neither |
| DEU_56_2001_121 | The future belongs to responsible governance for one world, governance based not on hegemonic claims, but on cooperation, solidarity and multilateralism. | true | false | justice | true | true | justice | justice |
| HUN_23_1968_105 | Our co-operation with the Czechoslovak Socialist Republic, the consolidation and strengthening of our common international activity will create a new and more solid basis for bilateral and multilateral negotiations with a view to building up a real and peaceful system of security in Europe. | true | false | justice | true | false | justice | justice |
| ROU_28_1973_106 | Romania itself follows the process of developing and modernizing its national economy The Romanian people make consistent efforts for the building up of a multilaterally developed society through the more effective use of the country's human and material potential for the acceleration of the development and modernization of the productive forces, the improvement of the management and organization of the society, the deepening of democracy and the laying of new premises for raising the standards of life and civilization of the whole people. | true | false | justice | true | false | neither | conflict |
| AND_76_2021_1 | Mr President, Mr Secretary General, Your Excellencies, Ladies and Gentlemen, I would like to begin by congratulating His Excellency Mr Abdulla Shahid on his election as President of this General Assembly and His Excellency Mr Antonio Guterres on his reappointment for a second term as Secretary-General of the United Nations (UN), this great assembly of all peoples and for all peoples, whose main objective is to build a system of values and a code of conduct based on multilateralism and cooperation. | true | false | justice | true | false | justice | justice |
| LIE_65_2010_14 | Past experience has shown time and again that multilateral action can be very effective when it is based on a broad political consensus, which is not the same as giving everyone a right to veto. | true | false | justice | true | false | neither | conflict |
| NLD_75_2020_32 | Critical evaluation and continuous efforts to improve are key elements of multilateral cooperation. | true | false | justice | true | false | neither | conflict |
| AUT_64_2009_2 | We have all witnessed something new and different: a genuine opportunity for a real renaissance of multilateralism. | true | false | neither | true | false | justice | conflict |
| PRT_42_1987_53 | The climate of dialogue achieved during the conference allowed for a sound analysis of multilateral economic co-operation. | true | false | neither | true | false | neither | neither |
| ITA_76_2021_57 | The fight against climate change requires multilateral engagement and pragmatic cooperation among all major global players — both rich and emerging economies. | true | false | justice | true | true | neither | conflict |
| DEU_59_2004_43 | To make this multilateral cooperation sustainable and capable, we need a courageous and comprehensive reform of the United Nations that faces up to the challenges we are facing. | true | false | justice | true | false | justice | justice |
| AND_72_2017_85 | That is why much of our foreign policy is focused on multilateral forums, as we showed four years ago during the Andorran presidency of the Council of Europe. | true | false | neither | true | false | neither | neither |
| DEU_63_2008_63 | My proposal to multilateralize the fuel cycle showed how those risks can at the very least be minimized. | true | false | justice | true | false | neither | conflict |
| CRI_62_2007_64 | Neither the chronic pessimism of some nor the ungrateful egoism of others will check the slow but determined march forward of those of us who believe in multilateralism and in a future of greater shared welfare for all. | true | false | justice | true | false | justice | justice |
| BRA_63_2008_46 | Its distorted form of representation stands between us and the multilateral world to which we aspire. | true | false | justice | true | false | justice | justice |
| PER_73_2018_41 | We will continue to promote initiatives to help restore democracy in our sister country within the framework of the Organization of American States, the Lima Group and in other multilateral forums. | true | false | justice | true | false | neither | conflict |
| URY_35_1980_86 | Now we must go from words to action and allocate the necessary funds, negotiate bilateral and multilateral financial aid and at the same time offer UNEP and the International Union for Conservation of Nature and Natural Resources the sustained support that will permit the practical and progressive adoption of their plans. | true | false | neither | true | false | neither | neither |
| JAM_70_2015_31 | The next major step for multilateralism will come in a few weeks when we hope to conclude negotiations in Paris on a global agreement on climate change. | true | false | justice | true | false | neither | conflict |
| BRA_64_2009_20 | No one is yet clearly willing to confront serious distortions of the global economy in the multilateral arena. | true | false | injustice | true | false | injustice | injustice |
| LCA_77_2022_6 | We have arrived at this watershed moment in history because, we, the Members of the United Nations, have not adhered to the rules and principles of the Organization that we created seventy-six years ago as a multilateral answer to the human propensity to use arms against our fellow humans, instead of joining with them to turn those weapons into tools for peace and development. | true | false | justice | true | false | injustice | conflict |
| ARG_61_2006_21 | We believe that, in order to face this criminal threat successfully, we must carry out a sustained multilateral and legitimate response. | true | false | justice | true | false | justice | justice |
| JAM_49_1994_7 | This is an exciting and daunting task, particularly for a small island State like Jamaica, whose faith and commitment to multilateralism and the United Nations remain unwavering. | true | false | justice | true | false | neither | conflict |
| GTM_64_2009_4 | My presence highlights our commitment to multilateralism in general and to the United Nations in particular. | true | false | neither | true | false | neither | neither |
| JAM_58_2003_16 | These are real concerns that underscore the need to strengthen 2 multilateralism, to restore confidence in the United Nations system, to buttress its centrality in decisions that affect us all and to thereby enhance its capacity to enforce. | true | false | justice | true | false | injustice | conflict |
| COL_76_2021_1 | We meet again in this global forum, which has historically been a vital space for the development of multilateralism, peacebuilding and responses to threats to our shared home, against the backdrop of a cruel pandemic that is battering our health systems, our economies, our gains in equity and progress on fulfilling the 2030 Agenda for Sustainable Development. | true | false | justice | true | false | neither | conflict |
| KNA_49_1994_116 | Above all, the concept of the vulnerability index must become part of the economic language and armamentarium of all multilateral and national development institutions. | true | false | justice | true | false | neither | conflict |
| CHL_77_2022_80 | We need a modernized United Nations, where we all have the same goals, Based on multilateralism, justice and peace at all times and in all places, we must commit to taking the necessary actions, not just making statements, so as to stop Russia’s unjust war on Ukraine and put an end to all abuses by the powerful everywhere in the world. | true | false | justice | true | false | justice | justice |
| SLV_25_1970_59 | Moreover, the principles and norms the violation of which constitute aggression should be regarded as part of a system, and in this sense such a definition cannot be taken to mean the indication of limits for isolated cases; rather the definition must recognize the multilateral conditioning of the major international principles. | true | false | justice | true | false | justice | justice |
| GTM_76_2021_76 | I also draw the Assembly’s attention to the fact that Taiwan could provide its experience, capacities and knowledge to the strengthening of multilateralism, taking into account the challenges we face today. | true | false | justice | true | false | neither | conflict |
| CHL_70_2015_26 | We are calling for an architecture of multilateral institutions that will provide backing for the national policies proposed under the 2030 Agenda. | true | false | justice | true | false | justice | justice |
| PAN_44_1989_36 | Persecution at the international level, not excluding multilateral agencies of which we are a member, is intended to put an end to national efforts to achieve economic recovery and meet the basic needs of the Panamanian people. | true | false | injustice | true | false | injustice | injustice |
| BRA_56_2001_51 | In the field of trade, it is high time that multilateral negotiations were translated into greater access for goods from developing countries to the more prosperous markets. | true | false | justice | true | false | justice | justice |
| DOM_52_1997_35 | Also, together with other countries, it is formulating strategies to take advantage of funds from multilateral sources. | true | false | neither | true | false | neither | neither |
| MLT_64_2009_63 | In his report on the work of the Organization, the Secretary-General has laid emphasis on the need to embark on a multilateral effort of immense magnitude “that draws upon the strengths and contributions of all the countries of the world, as well as their citizens” (A/64/1, para. | true | false | justice | true | false | neither | conflict |
| DJI_70_2015_25 | We will have the opportunity to return to this, but allow me now to address the implications of the macroeconomic fragility in the global economy, the many and varied challenges facing our country and the importance of establishing a credible and beneficial multilateral trading system for all. | true | false | neither | true | false | neither | neither |
| DZA_73_2018_47 | The rejection of policies based on force, with the dangers that they entail, requires that we constantly seek, through dialogue and consensus, to strengthen multilateral action. | true | false | justice | true | false | justice | justice |
| BHR_71_2016_71 | We still hear the same irresponsible sectarian discourse from Iran and have witnessed the damage done to our bilateral and multilateral relations. | true | false | injustice | true | false | injustice | injustice |
| MAR_43_1988_72 | At the same time, we must take advantage of the forthcoming multilateral talks, especially the Uruguay Round, to improve the international economic environment in the spheres of the flow of .net resources, trade and the stabilization and rising of commodity prices. | true | false | justice | true | false | neither | conflict |
| DZA_32_1977_140 | Real reform of the international monetary system: consideration of the legitimate interests of the developing countries in multilateral trade negotiations: the creation of a joint commodity fund; the adoption of a code of conduct on the transfer of technology; the solution of the problems of indebtedness of the developing countries; the promotion of the industrialization of the third world -this list is not exhaustive—are so many objectives which the international community must set for itself within a reasonable period of time. | true | false | justice | true | false | justice | justice |
| MAR_74_2019_54 | Multilateral action is not just an organizational structure of international relations; it is also a state of mind. | true | false | justice | true | false | neither | conflict |
| QAT_74_2019_74 | Despite the different situations and circumstances among countries, we need international cooperation today more than ever and must ensure the credibility of multilateral action to counter the challenges of climate change. | true | false | justice | true | false | justice | justice |
| DZA_64_2009_12 | Multilateral institutions, also undermined by contradictions and a lack of coherence, are unable to overcome impasses in negotiations on vital questions that have a direct impact on our populations. | true | false | injustice | true | false | injustice | injustice |
| MLT_38_1983_36 | Many countries have today recognized and expressed support for our status of neutrality, both at the bilateral level and in the multilateral forums of the Movement of Non-Aligned Countries, the Commonwealth and the Conference on Security and Co-operation in Europe. | true | false | neither | true | false | neither | neither |
| MLT_76_2021_3 | If anything, COVID only reaffirmed the priority of multilateralism, and that it is the key to a strong and sustainable future. | true | false | justice | true | false | justice | justice |
| MAR_73_2018_66 | The multilateral system, despite the criticism it faces, will remain essential to addressing the challenges and issues facing the international community. | true | false | justice | true | false | neither | conflict |
| MAR_76_2021_22 | The COVID-19 pandemic has shown that there is an urgent need for pragmatic multilateral action whose legitimacy is rooted in the effective realization of the rights of citizens to security, health and development. | true | false | justice | true | false | justice | justice |
| QAT_76_2021_60 | Qatar’s bet on international institutions and multilateral cooperation is a strategic one. | true | false | neither | true | false | neither | neither |
| BHR_61_2006_69 | Today, all of us need to join in efforts to achieve the objective of a collective security system through multilateral diplomacy, for which the United Nations is the main arena. | true | false | justice | true | false | justice | justice |
| KWT_48_1993_15 | Kuwait welcomes the accession to United Nations membership by Eritrea, the Czech Republic, the Slovak Republic, Monaco, Andorra and The Former Yugoslav Republic of Macedonia and trusts that their accession to the Organization’s membership will consolidate further the efficacy and universality of multilateralism. | true | false | justice | true | false | neither | conflict |
| IRQ_68_2013_72 | While the United Nations is our agreed forum for international cooperation to address the problems and challenges that we face through multilateral negotiations, we believe that the time has come to reform its bodies, in particular the Security Council, as the primary body responsible for international peace and security, in order to make them consistent with the aspirations of the peoples of the world in the twenty- first century and more representative, transparent and able to meet the challenges they face. | true | false | justice | true | false | justice | justice |
| SAU_61_2006_7 | The world today faces a host of global problems that can be successfully addressed only within the context of multilateral cooperation under the umbrella of the United Nations. | true | false | justice | true | false | neither | conflict |
| OMN_66_2011_2 | We are confident that his diplomatic experience, especially in the field of multilateral diplomacy, will have a great impact on the success of this session. | true | false | neither | true | false | neither | neither |
| MAR_76_2021_23 | Mr President, The bid to deliver vaccines, our last line of defence against the pandemic, is in fact an opportunity to assert our will and demonstrate our ability to infuse multilateral action with a new impetus. | true | false | justice | true | false | justice | justice |
| USA_41_1986_121 | We applaud the success of the meeting of the General Agreement on Tariffs and Trade (GATT) trade ministers last week in Uruguay, where agreement was reached to launch a new round of multilateral trade negotiations covering a wide range of topics important to economic growth, with over 90 other countries members of GATT, the United States is working to maintain the free flow of international trade. | true | false | justice | true | false | neither | conflict |
| USA_15_1960_58 | We should also look forward to appropriate and timely financial assistance from these two multilateral financial sources as the emerging countries qualify for their aid. | true | false | justice | true | false | neither | conflict |
| CAN_39_1984_49 | We shall encourage super-Power and multilateral discussion on all outer space weapons and shall commission further studies on how a space-weapons ban might be verified. | true | false | justice | true | false | neither | conflict |
| CAN_74_2019_12 | Strengthening multilateralism means involving all stakeholders. | true | false | justice | true | false | neither | conflict |
| USA_63_2008_34 | Other multilateral organizations have spoken clearly as well. | true | false | neither | true | false | neither | neither |
| CAN_39_1984_65 | In his most recent report he asks this: "Why has there been a retreat from internationalism and multilateralism at a time when actual developments both in relation to world peace and to the world economy would seem to demand their strengthening?" | true | false | justice | true | false | injustice | conflict |
| USA_34_1979_132 | In addition, it could recommend how current institutions could be strengthened and whether new multilateral ones should be created. | true | false | neither | true | false | neither | neither |
| CAN_74_2019_75 | We are fully committed to bringing innovative ideas to prevent and respond to crises, and we firmly believe in the power of multilateral solutions and partnership to address the most intractable of global challenges. | true | false | justice | true | false | neither | conflict |
| CAN_76_2021_80 | However, the benefits of multilateralism have not always been evenly or fairly distributed, and the potential for peace is yet to be realized in far too many places. | true | false | injustice | true | false | injustice | injustice |
| USA_63_2008_15 | Multilateral organizations have responsibilities. | true | false | justice | true | false | neither | conflict |
| CAN_49_1994_35 | As the leader of the multilateral Working Group on Refugees, we are tackling with determination that task entrusted to us by the international community. | true | false | justice | true | false | neither | conflict |
| CAN_40_1985_93 | Moreover we shall work towards the conclusion of a multilateral agreement prohibiting the possession and use of radiological weapons. | true | false | justice | true | false | neither | conflict |
| CAN_40_1985_31 | The challenges facing the multilateral system reach beyond this institution. | true | false | neither | true | false | neither | neither |
| CAN_56_2001_19 | We in Canada, as long- standing proponents of multilateralism and the United Nations system, have strongly welcomed the close collaboration evident between the United States Administration and the United Nations over these last two months. | true | false | neither | true | false | neither | neither |
| USA_20_1965_212 | We would be prepared to increase the amount of capital flowing through multilateral channels. | true | false | neither | true | false | neither | neither |
| CAN_38_1983_194 | They have not lost faith in the multilateral institutions we have so painstakingly constructed. | true | false | neither | true | false | justice | conflict |
| CAN_59_2004_32 | It is always preferable to have multilateral authority for intervention in the affairs of a sovereign State. | true | false | justice | true | false | neither | conflict |
| CAN_40_1985_125 | Bilateral relations between countries on the one hand, and the multilateral system on the other, ought to be mutually complementary and reinforcing. | true | false | neither | true | false | neither | neither |
| USA_33_1978_43 | The facilities of IMF have been expanded, and discussions are under way to expand the facilities of the multilateral development banks. | true | false | neither | true | false | neither | neither |
| CAN_49_1994_102 | Canada is ready to proceed with a comprehensive re- evaluation of its multilateral system as it applies to economic and social issues. | true | false | justice | true | false | neither | conflict |
| PAK_60_2005_108 | It is in the same spirit of constructive engagement that Pakistan wishes to pursue cooperative multilateralism and to strengthen the United Nations — this unique forum that is indispensable, in our interdependent world, for all States, large or small, powerful or weak. | true | false | justice | true | false | neither | conflict |
| AFG_41_1986_34 | The imperialist policy of using economic assistance as a means of exerting political pressure on the developing countries is no longer confined within the limits of bilateral relations; it has also been extensively employed to undermine the capabilities of multilateral institutions, with a view to curtailing or completely stopping the flow of international development assistance to developing countries, particularly those that dare refuse to submit to imperialist diktat. | true | false | injustice | true | false | injustice | injustice |
| PAK_21_1966_212 | The purpose of the conference, among other things, would be to consider, first, how the security of States without nuclear weapons can best be assured, preferably through multilateral guarantees; secondly, how they may co-operate among themselves in preventing further proliferation of nuclear weapons; and thirdly, the development and use of nuclear energy for exclusively peaceful purposes, through mutual co-operation and for mutual benefit, including the carrying out of explosions of nuclear devices for peaceful purposes under appropriate international supervision. | true | false | justice | true | false | neither | conflict |
| LKA_32_1977_110 | Existing national insurance agencies in developed countries could pool their risks through a reinsurance agency that might be set up on a multilateral basis. | true | false | justice | true | false | neither | conflict |
| BGD_49_1994_168 | Bangladesh looks forward to consolidating our relations with South Africa in more concrete ways, bilaterally and in all multilateral forums. | true | false | neither | true | false | neither | neither |
| PAK_27_1972_102 | Pakistan attaches the highest priority to both bilateral and multilateral efforts for disarmament. | true | false | neither | true | false | neither | neither |
| PAK_66_2011_23 | It is the United Nations and multilateralism that will safeguard that future. | true | false | justice | true | false | neither | conflict |
| BTN_67_2012_56 | As a small State, we have always attached the highest importance to multilateralism and the primacy of an effective United Nations that serves the interests of all its Member States. | true | false | justice | true | false | neither | conflict |
| AFG_64_2009_9 | In our increasingly interdependent world and the multilaterally oriented international system, the United Nations must assume greater responsibility for finding collective solutions to our challenges. | true | false | justice | true | false | neither | conflict |
| LKA_47_1992_81 | The successful conclusion of the convention on chemical weapons demonstrates the competence of the United Nations in multilateral negotiations. | true | false | justice | true | false | neither | conflict |
| LKA_77_2022_12 | In multilateralism, we talk to each other, we develop a relationship of trust and confidence and, if something were to come up, you have the base to work from. | true | false | neither | true | false | neither | neither |
| NPL_32_1977_77 | Similarly, there has been little progress in the multilateral trade negotiations in GATT on the reduction of trade barriers of developed countries, which restrict the exports of primary and processed commodities from developing countries. | true | false | injustice | true | false | injustice | injustice |
| IND_36_1981_154 | Total bilateralism at the expense of multilateral cooperation would run into alliances that may not be in the interest of harmonious relations and relaxation of tensions. | true | false | justice | true | false | justice | justice |
| PAK_71_2016_4 | Today, three decades after the end of the Cold War, our multipolar world is more free and vibrant, yet still chaotic and turbulent; more interdependent, but more unequal; more prosperous, yet still afflicted by poverty. | false | true | injustice | false | true | injustice | injustice |
| PAK_66_2011_13 | Each September we return to this great city and this grand stage so that we can restate and reaffirm the principles and values of multilateralism. | true | false | justice | true | false | neither | conflict |
| PAK_59_2004_94 | Today, there is welcome resurgence of support for multilateralism. | true | false | neither | true | false | neither | neither |
| MDV_68_2013_63 | SIDS need to be given full recognition within the global governance regimes and multilateral and financial institutions, and adequately integrated and institutionalized within the United Nations system. | true | false | justice | true | false | justice | justice |
| NPL_37_1982_17 | International economic co-operation and the system of multilateral co-operation continue to erode at an alarming rate. | true | false | injustice | true | false | injustice | injustice |
| BTN_38_1983_94 | Recently there has been much concern expressed at the deterioration of the resource position of some multilateral development organizations, UNDP in particular. | true | false | neither | true | false | neither | neither |
| PAK_53_1998_146 | This suggests the need for concerted action by the international community, which should include, first, the 14 strengthening of the capacity of multilateral institutions to address the issues of trade, finance and development in an integrated and coherent manner, thereby ensuring the effective governance of globalization. | true | false | justice | true | false | justice | justice |
| CMR_74_2019_81 | This occasion gives me the opportunity to warmly acknowledge the contribution of bilateral and multilateral partners that have always reaffirmed Cameroon’s unity and territorial integrity. | true | false | neither | true | false | neither | neither |
| CMR_39_1984_157 | As a corollary, multilateralism is dangerously declining; there are clear signs of this, mainly the current decrease in the resources made available to institutions such as UNDP, the fact that many States, particularly the more powerful ones, do not hasten to resort to the machinery offered by the United Nations to resolve issues of world-wide interest and, to some extent, the difficulties experienced now by UNESCO. | true | false | injustice | true | false | injustice | injustice |
| ZWE_45_1990_61 | Zimbabwe therefore wishes to appeal to the international community and to the various donors and multilateral economic institutions for support of our national efforts. | true | false | justice | true | false | justice | justice |
| ZWE_52_1997_7 | Africa has consistently underlined the need for reform of the United Nations and other multilateral bodies, to promote the democratization and effectiveness of the international decision-making process. | true | false | justice | true | false | justice | justice |
| ZWE_73_2018_40 | My country remains committed to strengthening multilateralism and the peaceful resolution of differences. | true | false | justice | true | false | neither | conflict |
| SWZ_63_2008_61 | The Kingdom of eSwatini reaffirms its commitment to the purpose and preservation of the central role of the United Nations in multilateral affairs. | true | false | justice | true | false | neither | conflict |
| SOM_74_2019_6 | In this globalized world, no nation, no matter how wealthy, strong or prepared, can individually stand alone against the tide of global challenges, which requires common action and coordinated multilateral responses. | true | false | justice | true | false | neither | conflict |
| ZMB_72_2017_5 | The theme for this session, “Focusing on people: Striving for peace and a decent life for all on a sustainable planet”, provides us with an opportunity to evaluate the existing multilateral approaches to address the challenges affecting our peoples. | true | false | justice | true | false | neither | conflict |
| CPV_71_2016_10 | As a small island State, Cabo Verde bases its foreign policy on the principles enshrined in the Charter of the United Nations and is certain that multilateralism is the most appropriate way to approach issues on the international agenda. | true | false | justice | true | false | neither | conflict |
| SEN_75_2020_44 | Furthermore, in implementing the Declaration on the commemoration of the seventy-fifth anniversary of the United Nations, which Senegal fully endorses, we must renew our commitment to the agenda for UN reform in order to restore multilateralism to its rightful place with the United Nations at its centre. | true | false | justice | true | false | justice | justice |
| LSO_59_2004_75 | That must be done against the background of multilateralism, upon which international peace and security is premised. | true | false | justice | true | false | justice | justice |
| GNQ_77_2022_7 | Equatorial Guinea is today launching an appeal on the need and importance of enhancing multilateralism and international cooperation, which are so necessary to address those global challenges and to revive development and sustainable economic growth in our countries, especially developing countries. | true | false | justice | true | false | justice | justice |
| BFA_76_2021_7 | The strategic vision you presented for your second term confirms your commitment and determination to work towards strengthening multilateralism and the United Nations in solving the multiple problems facing our world, especially in these difficult times caused by the coronavirus disease (COVID-19) pandemic. | true | false | justice | true | false | neither | conflict |
| KEN_77_2022_104 | It is time for multilateralism to reflect the voices of the farmers, represent the hopes of villagers, champion the aspirations of pastoralists, defend the rights of fisherfolk, express the dreams of traders, respect the wishes of workers and indeed protect the welfare of all the peoples of the global South. | true | false | justice | true | false | justice | justice |
| GIN_50_1995_90 | The United Nations and its specialized agencies, as the paramount instrument for cooperation and multilateral agreement, see their responsibilities increased today more than ever before. | true | false | justice | true | false | neither | conflict |
| GIN_28_1973_13 | The problem of international peace and security, the defence of national sovereignties, decolonialization and the struggle against imperialism continue to be the focus of both bilateral and multilateral meetings. | true | false | justice | true | false | justice | justice |
| GNB_75_2020_44 | We cannot talk about multilateralism while allowing the economic and financial embargo to continue to be imposed on one of the most supportive members of our Organization — Cuba. | true | false | injustice | true | false | injustice | injustice |
| SLE_57_2002_57 | Never since the end of the Second World War has multilateral cooperation become such a necessary means for resolving international disputes and for ensuring the well-being of people everywhere. | true | false | justice | true | false | justice | justice |
| SLE_67_2012_70 | We are confident that with sustained support from our bilateral and multilateral partners, we shall promote socioeconomic progress and provide a better standard of living for our people in an atmosphere of peace and security. | true | false | justice | true | false | neither | conflict |
| ZMB_44_1989_9 | We meet today against a background of great hopes and expectations for the success of multilateralism, which is so vital for the well-being of mankind. | true | false | justice | true | false | justice | justice |
Visualize
The plot below summarizes the outputs of both models and their overlap. I show the number of sentences each model assigned to the categories “justice”, “injustice”, and “neither”, with a separate bar for instances of agreement (“Overlap”). This side-by-side layout reveals the inconsistency across models, especially for ambiguous or borderline cases.
Important!
⚠️ The crucial insight for researchers considering LLMs for large-scale text analysis is the following: model choice matters, and the same prompt may yield divergent results across providers.
References
Baturo, Alexander, Niheer Dasandi, and Slava J. Mikhaylov. 2017. “Understanding State Preferences with Text as Data: Introducing the UN General Debate Corpus.” Research & Politics 4 (2): 19. https://doi.org/10.1177/2053168017712821.
Laurer, Moritz, Wouter van Atteveldt, Andreu Casas, and Kasper Welbers. 2024. “Less Annotating, More Classifying: Addressing the Data Scarcity Issue of Supervised Machine Learning with Deep Transfer Learning and BERT-NLI.” Political Analysis 32 (1): 116. https://doi.org/10.1017/pan.2022.15.