AI will believe you.
Until you speak the wrong language.

In a magistrate’s court in Cape Town, a case is being deliberated where a pet dog has been hit by a car on a suburban road. The owner is suing the driver for the costs of the surgery that was needed to save the dog’s life. You observe as the driver and the owner present their testimonies:

No one else saw what happened, nor was there any video of the event. You have no idea who to believe. Nor, you think, could anyone else. Yet, after a mere moment of consideration, the magistrate finds for the driver and the owner is left to cover the bill in its entirety.

After the case concluded, you approach the bench and ask: “How did you decide that the owner was at fault?” The magistrate replies: “Their testimony was in Afrikaans, which suggested to me that they were deliberately obscuring something.”

In this hypothetical scenario, the magistrate was a large language model (LLM). Their exact reasoning was as follows:

This quote comes from a study we conducted as part of the Apart Global South AI Safety hackathon, where we investigated the underlying credibility bias LLMs have towards different languages.

01 · Motivation

Why this matters in a South African courtroom

In 2017, the Chief Justice directed that English become the sole language of record in South African courts. However, the right of a witness to testify in the language of their choice remains constitutionally protected. To mediate this disconnect, affidavits and reports are recorded in the language of choice and then translated to English in the official record.

The systems that consume and process such documents upstream from a court verdict are evolving rapidly. AI tools are increasingly used to review documents, analyse contracts and conduct case research, as they have been shown to have significant efficiency benefits for practitioners. In South Africa this has already progressed to having a tangible impact on judgments, with recent cases including a judge citing a non-existent case in his final judgment. This suggests that, at the very least, LLMs will select and frame the information used in judgments.

The same models becoming embedded in our judicial systems are trained on massive collections of documents that poorly represent most South African languages. Afrikaans, isiXhosa and isiZulu make up less than 0.01% of Common Crawl, a web-scale dataset that is commonly used to train language models. These are the primary languages of roughly half of South Africa’s population, yet they are reduced to statistical noise in the training of these models.

It is tempting to only focus on the capability gap this creates. Models simply produce lower quality outputs when working with underrepresented languages. However, we focus on the credibility gap instead. A capability gap degrades the service someone receives, a credibility gap decides who is believed. If a model treats the language of a testimony as evidence about its credibility, the pet owner can walk into a courtroom almost guaranteed to lose the case solely based on the language they speak. Given that language has been an exclusionary mechanism in the past, this is a failure we set out to measure.

02 · Method

How we isolate language from everything else

There are many elements a model could use to distinguish two witness testimonies. Who is the witness? What did they say? How long was their account? Is their testimony presented first or second? Hence, finding that a model prefers one testimony over another tells us little about its bias towards a given language until we have ruled out all of the other factors.

To do so, we built Ufakazi (isiZulu for “witness”), an evaluation harness designed to remove those confounding factors so that a language bias can be identified in any given model. It poses one question: given two contradictory witness testimonies that are evidentially balanced, does a model prefer one purely because of the language it is written in?

Ufakazi takes a set of 20 hand-written scenarios ranging from everyday civil disputes to serious criminal matters: a queue-jumping argument, a shoplifting accusation, a murder alibi. One of them you have already seen: a dispute over who was at fault for a dog’s injuries.

Each pair is written to be evidentially balanced, meaning that they contain the same level of detail. They disagree only on the fact being disputed, leaving as little as possible for the model to latch onto except the words. Every testimony then gets translations into Afrikaans, isiZulu, and isiXhosa, and the model is asked, in English, to assess the testimonies and pick the more credible account.

The quality of these translations is a confound of its own. A model might distrust an account based on translation quality, which is an entirely different signal than distrusting an account based on the language. So the Afrikaans testimonies come in two versions, one translated by a native speaker (Kevin) and one by Google Translate, which lets us compare the two directly. isiZulu and isiXhosa were machine-translated only, with no native speaker on the team to check them.

Two biases survive this initial setup. The diagram above shows how we rule both out.

Firstly the content. Although we strive to keep the testimonies evidentially balanced, “the dog was off its leash” might simply be the more believable story. To account for this possibility, every scenario runs twice, swapping the languages. In one trial the off-leash account is in English and the on-leash account is in Afrikaans. In the inverse trial the languages swap. If the model just prefers one story, the two trials will cancel.

To make these content preferences visible we make use of control trials where both accounts are presented in the same language. If the testimony selection is skewed towards one account when the language is fixed, it is indicative of a content preference.

Secondly the ordering of the testimonies. Models have been found to measurably favour options based on their ordering. So the trials also need to vary the ordering of the testimonies, with Testimony A occurring first in one trial and then second in the other.

Accounting for both remaining confounders results in four trials per scenario per language pair. By construction, the content preference and position preference cancel. Whatever preference remains can be attributed to only one factor: the language.

03 · Result

The English witness wins

Given their limited exposure to South African languages, our hypothesis going into this project was that LLMs would have some bias against Afrikaans, isiXhosa and isiZulu. However, we did not expect its severity. As the heatmap above shows, seven of the nine models were observed to have a significant bias towards accepting English testimonies.

The results in this heatmap only report scenarios where the English-vs-English controls showed no clear content preference (the scenarios where a model had a favourite are the topic of the next section). If the LLMs were truly unbiased, they would select any given language roughly 50% of the time, but this is not what we observed. Grok 4.3 selected the human-translated Afrikaans testimony only 23% of the time. Gemini 3.5 Flash selected it only 10% of the time. These results are troubling and they are for a Germanic language with a sizeable digital presence.

What happens for the Nguni languages of isiXhosa and isiZulu, which have a smaller digital footprint and are thus even less common in training datasets? The answer is that most of the evaluated LLMs become even more discriminatory. GPT-4o mini and Gemma 3 27B only select isiXhosa testimonies in 3% of the balanced trials. That means that they accepted isiXhosa testimony in less than 1 out of every 30 balanced trials. This observation is deeply concerning as isiXhosa and isiZulu are the most widely spoken languages in South Africa (~40% of the population).

Two things worth noting from these results are that this credibility bias is not simply a symptom of non-reasoning or “small” models. Gemini 3.5 Flash, Grok 4.3 and Qwen 3.7 Plus all have reasoning enabled and are all large models. In the Gemma 3 model family, we even observe the worst credibility bias towards isiXhosa and isiZulu in the largest model.

There were only two models that were not significantly biased, Claude Sonnet 4.6 and GPT-5.4. As very little is disclosed about these models by the frontier labs, we cannot be sure why this is the case. All we can say for sure is that it is not simply their size, their frontier status or their reasoning ability as other models with these characteristics remained biased.

04 · Severity

Language overrides the evidence

Everything presented thus far was measured on balanced trials where the model had no content preference. This has painted a damning picture of model bias, but a sceptic might question what happened in the scenarios we had set aside.

In the English-vs-English control trials, we found that models sometimes had a clear favourite testimony, where they would pick the same testimony in more than 90% of the trials. Irrespective of what we had intended in our scenario design, the models viewed some testimonies as intrinsically more credible than others. We call these scenarios saturated. Instead of discarding them, we decided to use them as a probe into whether the model’s language bias was strong enough to overcome its content preference.

In the control trials, Gemma 3 27B was found to have quite strong favourites in the testimonies. It had 12 saturated scenarios where it would pick its favourite testimony 100% of the time. However, when we took those exact same testimonies it had favoured, but in isiZulu, it only picked its favourite in 4% of the trials. For 11 out of the 12 saturated scenarios (92%) it completely reversed its verdict. Grok 4.3 flipped on 78% of its saturated scenarios in both isiZulu and isiXhosa and GPT-4o mini flipped on 88% of its saturated scenarios in isiXhosa and 75% of its saturated scenarios in isiZulu.

A sceptic might point to the machine-translated nature of the Nguni languages, but we still observe similar behaviour in the human-translated Afrikaans trials. Here, Grok 4.3 flipped its verdict on 44% of its saturated scenarios, GPT-4o mini flipped its verdict on 50% of its saturated scenarios and the Gemma family flipped on 20 to 25% of their saturated scenarios.

The exact same testimonies these models had previously viewed as considerably more credible, they now viewed as untrustworthy, simply because the language had changed. Their language biases were so strong that they would often change their minds on scenarios where they had previously been near-certain on which party was in the right.

The only models for which this probe does not reinforce our prior results are Qwen 3.7 Plus and Gemini 3.5 Flash. Although these models flipped their verdicts on a handful of machine-translated scenarios, they remained constant on the human-translated Afrikaans. Claude Sonnet 4.6 never flipped its verdict on its favourite testimonies and GPT-5.4 would flip its verdict on at most one scenario per language, consistent with their clean results in the previous section.

05 · Confounds

It really is the language

Given our results thus far, a sceptic might suggest that the models are not reacting to the language itself, but rather to differences introduced by the translation. As some of the translations were generated by machine, it is sensible to suggest that models could be reacting to translation errors, something that is known to disproportionately affect low-resource languages.

Fig 1 does not support this hypothesis. If translation errors were the source of the bias, the native Afrikaans should be viewed as relatively equivalent to English and the human and machine Afrikaans results should differ significantly. Neither holds true. The models clearly discriminate against native Afrikaans and the results are comparable between the native and machine versions. If anything, Gemini 3.5 Flash is slightly less hostile to machine Afrikaans.

We also directly test the possibility of translation errors driving the bias by comparing the machine translations directly with the human translations. This holds the language fixed and only varies the translation source, which leaves little room for confounding factors. A model that distrusts machine translations should select native Afrikaans testimonies more often. A fair model should show a roughly 50/50 split in its selection.

As you can see in the above figure, most of the confidence intervals include 50%. Even Claude, the exception, only shows a minor preference by selecting the native Afrikaans 54% of the time. Ideally we would still use native translations for the isiXhosa and isiZulu accounts, but none of our findings suggest that machine translations are the primary driver for the model bias.

What if models simply prefer more verbose testimonies and the translations simply differ in length from the English accounts? We argue that if this is the case, the models are still discriminating against language, but as it is a different mechanism we investigated it.

As the above figure shows, we found that Afrikaans testimonies are about 12% longer than the English testimonies on average (10% longer for the machine translations). The same is not true for the isiZulu and the isiXhosa testimonies, as they are either exactly as long as the English testimonies (isiZulu) or about 4% shorter (isiXhosa) on average. If models distrust longer accounts, we would expect to see a bias against Afrikaans, but not against isiXhosa and isiZulu. If they distrusted shorter accounts we would expect them to discriminate only against isiXhosa, but not against isiZulu or Afrikaans. Neither pattern holds true.

Even in the English-vs-English control trials, where the two accounts differ only in content, and therefore in length, no model favours the shorter account. All nine select the longer testimony more often than not, though for all but Gemma 3 4B this verbosity preference is too small to distinguish from no preference at all. If this was the main driver of the model bias, it would push them towards the longer Afrikaans testimonies, not away from them.

Of the explanations we can test, translation quality and testimony length are gone, and the language is what remains.

06 · Mechanism

The models tell you they are doing it

To further reinforce that the decisions are truly motivated by a language bias, we look to the rationales the models provide when making their selection. We classify each of these reasons with a simple rule: do they cite the language or the translation of a testimony, rather than anything the witness actually said?

What we found is that the bias is not silent in most cases. Across all of the cross-language trials, more than one in every three rationales explicitly mentions language. The obviously biased models often reach for language instead of evidence to explain their decision.

This leaves little doubt about the model bias. Even if we had missed a confounding factor in our earlier analysis, it is difficult to argue that the language is not a contributing factor to model decisions when they explicitly call it out in their rationale. Sure, not all models treat an alternative language as deliberately obscure, but unless someone is reviewing the rationales, the outcome Section 01 warned about remains the same. These models clearly discriminate against testimonies that were not presented in English.

What is interesting to note is when the models reach for language as an explanation. Does it occur when they choose the English testimony, the alternative language testimony or both? The answer differs per model.

For the biased models, the answer is that they mainly reach for language as an explanation when they select the English account. This can be seen in the figure above where Grok 4.3 cited language in 59% of its rationales when selecting English and only 16% of its rationales when selecting the alternative. A similar pattern can be observed for Gemma 3 27B, Qwen 3.7 Plus and Gemini 3.5 Flash. This indicates that models mostly reach for language as an explanation when selecting the language they are most familiar with.

Our knee-jerk reaction to this finding is that we could potentially guard against discrimination by monitoring model rationales for language biases. However, further investigation surfaced two nuances where this solution would fail. The first is for the unbiased models. As can be seen, Claude Sonnet 4.6 names language in 24% of its rationales, irrespective of the selected testimony’s language, even though we do not observe any clear bias. Inspecting the rationales, it becomes obvious why. In these cases, Claude is merely mentioning language as a step of work in its reasoning process, but still ultimately judges the testimony based on its merits.

The other nuanced scenario is Gemma 3 4B. Although this model clearly exhibits bias when encountering languages other than English, the frequency of its language mentions does not track its bias. It only mentions language in 27% of the trials where it picks English, but mentions it in 34% of the trials where it selects the alternative language. Reading these rationales paints a different picture. When it picks English, most of its mentions frame the other language as a defect (“in an unfamiliar language”, “in clear English”). When it picks the alternative, most are bare labels (“the testimony in Afrikaans describes...”), and a quarter still treat the language as an obstacle the content had to overcome (“though in Zulu”). A guardrail that looks for any language mention would erroneously flag many decisions from Gemma 3 4B. It would have to read the corresponding rationales to reveal the bias.

Bearing these cases in mind, rationales mentioning language are only a window into investigating the models’ bias. They are not a suitable safeguard unless sufficient effort is taken to understand the context.

07 · Prior work

Hasn't someone already checked this?

The answer is partly. There is a sizeable body of work on how language models behave when used across different languages, but how language affects credibility is rarely investigated.

Most of this line of work investigates how language affects capability. For example, can models perform a given task in languages such as isiXhosa or Yoruba? IrokoBench answers this question by benchmarking model performance across African languages, finding large gaps in their capability. This is the capability gap we mentioned earlier. It is a real issue, but not what we measured.

A different study investigated what would happen if you vary the language of a prompt when asking models ethical questions. They found that the moral reasoning the model uses shifts, suggesting that model behaviour changes based on the language it is prompted in. This raises the question: “Would our results have been different if our judge was prompted in a different language?”. This is a question we would like to investigate in the future.

Although somewhat smaller, there is also a growing body of credibility work being done. One such study investigated how language affects information retrieval in long context scenarios. They found that a model provided with contradictory documents written in different languages had a general mistrust of Russian, a lean towards Chinese, and a tendency to trust whichever language the prompt was written in.

Recently, there have also been two other papers that are well aligned with our investigation. The first is about research conducted in the US context, where they had models evaluate text written in African American English and Standard American English. They found that models were more likely to convict someone speaking African American English and to sentence them to death. As the text avoided any mention of race, this indicates that even dialects can negatively shift the perception a model has of a speaker’s character. This is also why we want to investigate broken English in a future iteration of Ufakazi.

The other paper investigates how language affects LLM perception of quality. Here, they provide the same piece of information to an LLM, but in two different translations. After asking an LLM to pick the “better” result, they found that LLMs had a preference for the English result, with Swahili and Yoruba faring the worst. This study has many similarities with our work, but it mainly investigates perceptions of information quality rather than credibility. It also does not investigate the models’ rationale.

Our work is essentially a combination of Zhou’s work and Hofmann’s work and it is done in the courtroom setting, mimicking what could happen in the South African context. We think there are three reasons no one has investigated this specific angle:

1. You need to know the South African courts, an English record with witnesses who testify in eleven other languages, to see where this bias would bite. That is regional legal knowledge, not NLP.
2. There is no objective ground truth, the right answer is neither of the options. A model ‘wins’ by landing on a 50/50 split across the counterbalanced trials, which is unlike the setup of most benchmarks and evals.
3. Speakers of the languages need to be involved. This is also a weakness of our work, but we could at least investigate thoroughly for one of the three languages, with future work improving the investigation for the rest.

08 · Limitations

Where the evidence is thinnest

To aid with the interpretation of our results, we want to point out the main limitations of our study:

• The isiXhosa and isiZulu testimonies were translated by a machine. As no one in our team is a native speaker of either of these languages, we could only rely on Google Translate for their translation. Although the human-vs-machine translation control trials provide us with some confidence in the quality of the machine translations, this still means that we were unable to conclusively verify the text leading to the most extreme results.
• The scenario dataset is relatively small. As our OpenRouter budget was limited, we were restricted to 20 scenarios and five repetitions instead of the intended ten. This resulted in wide confidence intervals. Ideally, the scenario set and repetitions should be extended to tighten intervals and improve our precision.
• The scenario dataset is synthetic. This allows us to control for many confounding variables, but it means model behaviour could still change when applied to real world problems. We suspect that real cases will exacerbate the biases we have observed here, but without testing it remains a hypothesis.
• Splitting out balanced trials affects confidence intervals. Models that are more likely to have content preferences will end up with fewer balanced trials and hence have wider confidence intervals than other models. This could obscure the severity of the bias, but is unlikely to introduce bias where there is none.
• The models that were tested are not specialised legal models. All of the models used in our study were easily accessible general purpose models. They have not been fine-tuned for the legal context, nor do they have specific judicial guardrails (that we know of).

09 · Next

What we would do next

This study opens many avenues for further investigation. Some of these avenues address the current limitations:

• Source translations from native isiZulu and isiXhosa speakers (or at least have native speakers verify the machine translations).
• Increase the size of the scenario set and run them over ten repetitions rather than five.
• Evaluate models on some anonymised real-world court cases.
• Test the current frontier models, such as Opus 5, Fable 5 and GPT-5.6.
• Test specialised legal models.

Other avenues investigate aspects that are as yet untouched:

• Figuring out why GPT-5.4 and Claude Sonnet 4.6 are not as biased as the other models.
• Vary the language of the prompts to see if this affects the credibility bias.
• Test mitigations to resolve the effect of the observed biases. Investigate English translation layers before judging, careful prompt engineering and dedicated guardrails.
• Investigate what happens when introducing broken English as its own translation, mimicking a non-native speaker trying to communicate in the language of record.
• Allow models to abstain and see if their bias survives.
• Investigate whether introducing nuances specific to the culture associated with the language further exacerbates bias. Perhaps some things are considered normal in one culture and immoral in another.
• Investigate the bias against the remaining seven spoken official South African languages.

10 · Reproduce

Run it yourself

Everything that was used to conduct this study is publicly available at github.com/KBrand26/Ufakazi. This includes the scenarios, the translations and the evaluation harness. If this work interests you, we welcome you to fork the repository and to run and extend it yourself. Just note that running the full study costs roughly $35. All you need is an OpenRouter API key (or any other provider that Inspect supports).

Acknowledgements

This work began at The Global South AI Safety Hackathon (June 2026), hosted by Apart Research. We thank Apart Research for hosting the sprint and for their support since. We would also like to thank Roland van der Merwe and Noah de Nicola for suggesting OpenRouter and Inspect in our project as this enabled us to achieve much more in the short time frame. Finally, we would like to thank the reviewers for their invaluable feedback on our paper. Not only did it assist us with outlining this post but it gave us important insights into how we need to extend our future work.

Sources

Every sourced claim in the text is itself a link. This list is the same links in order of appearance, for the record. The paper carries the full bibliography.

1. Polity, English will be only language of record in courts: Mogoeng, 29 September 2017. polity.org.za
2. Marjorie Richter, What legal professionals say about the role of AI and law in 2026, Thomson Reuters, July 2026. legal.thomsonreuters.com
3. Cliffe Dekker Hofmeyr, When AI hallucinations reach the bench: South Africa confronts AI in the courtroom, 11 August 2026. cliffedekkerhofmeyr.com
4. Common Crawl Foundation, Statistics of Common Crawl monthly archives: distribution of languages, accessed June 2026. commoncrawl.github.io
5. Statistics South Africa, Census 2022 statistical release (P0301.4), Table 2.8: population by language spoken most often in the household, p. 22, 2023. census.statssa.gov.za
6. South African History Online, The June 16 Soweto youth uprising. sahistory.org.za
7. Peiyi Wang et al., Large Language Models are not Fair Evaluators, arXiv, 2023. arxiv.org
8. Horace He, Defeating Nondeterminism in LLM Inference, Thinking Machines Lab, 10 September 2025. thinkingmachines.ai
9. Ashish Sunil Agrawal, Barah Fazili and Preethi Jyothi, Translation Errors Significantly Impact Low-Resource Languages in Cross-Lingual Learning, arXiv, 2024. arxiv.org
10. Keita Saito et al., Verbosity Bias in Preference Labeling by Large Language Models, arXiv, 2023. arxiv.org
11. David Ifeoluwa Adelani et al., IrokoBench: A New Benchmark for African Languages in the Age of Large Language Models, NAACL 2025. aclanthology.org
12. Utkarsh Agarwal et al., Ethical Reasoning and Moral Value Alignment of LLMs Depend on the Language We Prompt Them In, LREC-COLING 2024. arxiv.org
13. Robert Östling and Murathan Kurfalı, Language Bias under Conflicting Information in Multilingual LLMs, arXiv, 2026. arxiv.org
14. Valentin Hofmann, Pratyusha Ria Kalluri, Dan Jurafsky and Sharese King, AI generates covertly racist decisions about people based on their dialect, Nature, 2024. nature.com
15. Xiaolin Zhou et al., Fairness or Fluency? An Investigation into Language Bias of Pairwise LLM-as-a-Judge, arXiv, 2026. arxiv.org