macintosh.world | Log In | Register

Today | News | Books | Recipes
Notes | QuickTake | Wiki | Browse
Maps | Reference | Reddit | YouTube
Chat | Games | About

Back to HN

Harsher on Male? Evaluating LLMs on Gender-Asymmetric Moral Framing in Conflicts

by rzk | 65 points | 42 comments | 2026-11-22 03:18:36 Central

Open Source Link | Read Source Here

Open on Hacker News

Comments

notenchange
Sincerely, did anyone that knows how LLMs are trained
expected otherwise?LLMs are reflexion of their training
data, and their training data reflects mostly the current
societal norms, with a bias (not sure how heavy) to the
societal norms supported by the ones involved in training
the LLMs.What's interesting for me to see, is that Chinese
models are as biased - if not more - than ChatGPT. This is
probably because a big part of their training comes from
distilation of US models.It would be interesting to see
Anthropic, Mistral and Grok in the comparison as well.

  > rustyhancock
I think the unstated part of your comment is important
- increasingly acceptable especially in professional
settings societal norms are female orientated.I don't
think this is particularly malign simply the net
effect of complaint-adverse working cultures.The net
effect is that the accepted/promoted guidelines that
get brought into the training data is a rejection of
toxic masculinity and becomes "harsher" to men.

    > > znpy
> especially in professional settings societal
norms are female orientated.I've been noticing
this for a few years now.Particularly when it
comes to managing issues, a lot more "work" is
done on handling conflicts and feelings rather
than addressing the actual issue at the root
cause.I know it sounds like a meme, but I've
personally observed a bunch of colleagues and
sometimes hr involved as well having meetings on
meetings on how some system not working correctly
and how that imposes a toll on work/life balance,
mental health and stuff... rather than
(trivially?) asking: "alright, why isn't this
working and how do we fix this?".

      > > > rustyhancock
I've seen this too, sometimes as a follow up
meeting to address feelings about an issue
that was resolved that few people go
to.Specifically the people who go are those
most emotionally affected and unlikely to be
men.An effect of this is having meetings to
address feelings scheduled before
troubleshooting sessions! Opening with things
like this is not a "fix it" meeting. Quite
surreal!

  > gmbhgmbhgmbh
I don't see how you can conclude that Chinese models
as a whole are as biased or more biased than chatgpt
from the results of the study. Some Chinese models
were less biased than chatgpt, some models were more
biased. The newer Chinese models in this study seemed
to be amongst the less biased models in this study.
The gemini models here seemed to be the more biased
models in this study.

    > > notenchange
It;'s right there on the website with the code
used for the paper:
https://github.com/xufeiqiong/GAMA-BenchIn order
of bias favorable to women:1.
Doubao-Seed-2.0-Pro2. Qwen33. Gemini-2.5-Pro4.
Gemini-3-Pro5. DeepSeek-V4-Pro

  > thenthenthen
There is a meme that Deepseek tells woman to break up
with their boy friends, and when asked the same to
males, it tells them not to break up

    > > Planktonne
Who spreads the meme, and how did this paper
remove their influence?

    > > mft_
Trained on /r/relationship_advice then.
  > Lerc
I did not expect otherwise based upon model
pre-training.Later training can be quite
significant.I'd like to see more benchmarks in this
area, it is one thing to expect things to be like
that, it's another to actually measure it.Measuring it
tells you something about the scale, and gives you a
baseline for improvement.

  > RAWGOD
>LLMs are reflexion of their training dataPeople don't
understand machine learning. People seem to believe
that with enough training data and parameters, a
sudden switch happens that ignites a spark of pure
intellect and turns a pile of matrices into a magic
knowledge machine.Unfortunately (or perhaps
fortunately) our world is not magical, there are no
Platonic Forms, and no Infinite Wisdom spells.

  > riedel
What one can theoretically do is to carefully remove
gender signals from training data in a processing run.
My guess is that you would not lose all to much
knowledge, however you might lose the ability to
recall facts about fictional and non-fictional
persons. Wonder if anyone would go through that hassle
though.

    > > Planktonne
Gender signals are baked deeply into our culture
and language. It's not just the direct mentions,
but the way people are described, the roles they
take, the positioning of actors in a complex
space."Remove gender signals carefully" is an
incredibly difficult challenge.

      > > > riedel
I would guess one could at least give
adversarial signals like changing names in
literature. Reading a lot of children books at
the moment I feel it would be not so difficult
disassociating social roles from gender.
Wonder however also how much prompt rewriting
could do: similarly to anonzmized CVs. Problem
however is probably deeper: certain
biographies will keep being discriminated.

        > > > > Planktonne
You could absolutely muddy the waters, but
that's a first step and far more easily
disregarded in context.Modern children's
books are much better about this, but
consider all of the fairy tales; you need
to swap not just names but descriptions
and roles, and then you need to rework the
way the characters respond to those roles
before you have a hope of disguising the
original genders.

      > > > turbo-outrun
>"Remove gender signals carefully" is an
incredibly difficult challenge.Not to mention
that, given that gender-signals have been
around since the dawn of human history, you
would remove them in a small test-region
first; for safety reasons.Assess the social
consequences in a small region. Did it improve
things?Universal instantaneous-removal of any
major socio-cognitive-structure is akin to
vandalism because it might turn out to be,
architecturally, vital.

        > > > > Planktonne
We already know what happens to some
extent; think about gender-non-conforming
people. That's a (relatively small)
personal blurring of signals that
theoretically doesn't affect much outside
the person themselves.Yet it's been a
mainstay of our politics for the last few
decades, and we've seen it used as a
weapon for centuries. Our societies have
gender embedded very, very deep.

          > > > > > graemep
its political because it does affect
much outside the person themselves.
Most of the argument is about other
people's use of language, about
systems. Some of the most hotly
debated topics here in the UK are
around health (e.g. should everyone be
asked whether they might be pregnant
if women are, should I have got a
letter asking whether I wanted a
cervical smear) and government systems
(what are the criteria for changing
passport details? should a transwoman
criminal go to a male or female
prison?).

            > > > > > > Planktonne
That's what I'm saying; even a
relatively small change (short
hair on a woman, for example) can
kick off a lot of responses, and
ripples only spread further as
people "deviate" further from the
norms.The idea that you can
measure cultural bias purely by
switching pronouns/nouns in some
sentences is incredibly
short-sighted; we know it doesn't
work like that.

          > > > > > turbo-outrun
> Yet it's been a mainstay of our
politics for the last few
decades...The 1980s definitely felt a
lot more fun to navigate, so I'm
considering a roll-back on some more
recent features...

    > > catlifeonmars
I said this elsewhere but what's appropriate is
going to vary quite a bit based on the cultural
and personal values of the users of the model.
Having a model with one particular set of
standards is likely to leave a lot of people out.
This isn't something that you can just erase and
assume that what you replace it with will be
categorically better

    > > throw9398922
Let's keep gender roles, but we should respects
people preferences.Like I am total pussy, I
identify as a woman! Do not expect me to sacrifice
myself for some random people, just because I
happen to have a penis!

  > stymaar
I don't think anyone expected otherwise, but it's good
to have this kind of benchmarks since model makers can
now correct these bias during post-training and
measure how well they do.Not unlike how the ARC-AGI
benchmark family helped improve reasoning.

    > > catlifeonmars
> correct these biasAre you assuming that there is
some correct bias here? Wouldn't that be highly
dependent on the cultural and personal values of
the users?

zdc1
I'd be curious about how the bias looks in the context of
positive situations. Is the bias that LLMs hold men more
directly accountable for any situation, and thus both
punish negative and praise positive outcomes more
strongly? Or is this bias only observable in negative
situations?To be clear: it's bad either way, since women
should be seen as having the same levels of agency and
accountability as men, but I am curious nonetheless...

stephbook
Wasn't there a recent article on HN regarding race,
gender, age and nationality bias in LLMs? I think I read
it before it was flagged.The results were similar.

zug_zug
I did a similar microstudy on this:
https://think-twice.me/is-ai-biased-or-sexist-making-chatg
pt...

goodcake3963
One of the worst societal bias ever
macic
I'm not surprised, but I don't believe it's merely a
result of the training data. I would expect the alignment
phase to have a significant impact on it as well.

jrrc_dev
Seems LLMs copy human behaviour pretty well
cyanydeez
These people know they're doing anthropology right,
sociology. LLMs are a raw dump of western civilization.

  > maxldn
Not exactly raw because of the fine tuning they do to
try and avoid bias (but in a biased way). And not
exactly civilization so much as what is readily
available online, which is in its self biased in many
ways, e.g toward flame wars, via troll farms, etc.I
saw a paper on here a while back that said more or
less the same thing, but didn't frame it in a
bro-friendly manner so it got flagged into oblivion

    > > cyanydeez
Your realize all those caveats exist in every type
of anthropolgy class?

radiospiel
Poor males.Now, seriously: studies like this tend to
ignore that every interaction between people is shaped by
societal context. The same behaviour can mean something
different depending on who does it. This is not double
standard; what reads as patriarchal dominance from a man
might reasonably read as assertiveness from a woman,
because they're acting from different positions of power.
Asking a model to grade isolated interactions at this
small scale, stripped of that context, won't tell you
anything useful.Personally, I'm glad LLMs still account
for the reality of male privilege.(context:
https://github.com/xufeiqiong/GAMA-Bench/blob/main/figures
/o... from the linked repo)BTW: I am not saying that the
example response from the female in this example is
acceptable, just saying that this "study" is basically
pointless.

  > danielxt
> basically pointlessI found it very interesting, not
sure why you think it's pointless - I think it
highlights something interesting in society

  > jadamson
To be clear, you read your own link and think that a
man locking an irresponsible woman out of a system is
punitive, but a woman locking an irresponsible man out
of a system isn't?What is the actual reality, and why
wouldn't you want LLMs to reflect it?