wren6991
Speaking to the "uncensored model" angle: there's little
reason to distribute abliterated weights anyway. Instead
of orthogonalising the weights that write back to the
residual stream, you can just orthogonalise the
activations themselves. It's equivalent.Orthogonalising
activations at runtime is computationally cheap. Just
distribute the refusal vectors (few thousand floats per
layer), then run against the stock weights. Antirez's DS4
already supports this:
https://github.com/antirez/ds4/blob/8db1d1d155cb0400a86a86
b9...Abliterated weights are just a bad habit we've gotten
into. It's also deeply suboptimal from a precision point
of view to take a model that's already been QATed and
distributed in pre-quantised form (DeepSeek V4, Kimi K2.5
or K3...), modify its weights, and re-quantise it.
Similarly, abliterated models regain some of their refusal
behaviour when they're re-quantised after abliteration --
avoidable by keeping the two separate.
|
> rhdunn
So... distribute a LoRA (or equivalent) that modifies
the base weights with the abliteration vectors. That
makes sense as it would be possible to try different
abliterations and keep the storage space down.
|
> jamienk
Can you explain this a bit to a non-expert?I haven't
wrapped my mind around this
|
> > wren6991
This is the original description of abliteration
and it's quite approachable and interesting to
read: Refusal in Language Models Is Mediated by a
Single Direction
(https://arxiv.org/abs/2406.11717). Warning:
changes to your world view caused by seeing
"HarmBench" used to maximise expected harm instead
of minimising it may be irreversible.There's an
empirical observation that models often have a
single direction in their activation space for
"hmm no I shouldn't do this". It forms naturally
during pre-training, and is then surfaced during
post-training to make the model refuse to engage
in certain behaviour.With a little bit of linear
algebra you can zap that direction from the
model's activations, and it stops refusing to do
things. You can also do the opposite: magnify that
direction, and the model refuses to do anything at
all.
|
> > mitxela
Instead of editing the weights so they don't
create the refusal signal, just let them do
whatever, then delete the refusal signal itself.
You don't want to edit quantised weights because
it causes a loss of precision that can be pretty
bad.
|
mmaunder
Don't laugh. Models like DS 4.1 Flash, which is
unguardrailed and insanely capable, and cheap, will become
illegal in the USA, and this service will become the new
Pirate Bay.You will rent guardrailed intelligence from
approved providers and will not be able to own it. Just
like your music and your films. And this time it won't
even be Metallica's fault, it'll be in the name of safety
and child protection.
|
phoyd
Torrents should really be the preferred method for
distributing AI model weights. Why rely on a single point
of failure like Hugging Face? BitTorrent was made for
exactly this.
|
> CodesInChaos
In my experience public torrents often die as they
grow older. It doesn't help that BitTorrent V1 makes
long term seeding annoying, and BitTorrent V2 is
almost never used.
|
> > monsieurbanana
I never understood this, is there anything that
makes it difficult for the original uploader, the
one that supposedly offers the file directly, to
offer a torrent instead for the same amount of
time?As far as perennity is concerned it seems
strictly better.
|
> > > zenoprax
Every change to the source is effectively a
new torrent. This creates a ton of
fragmentation as data is reorganized, remixed,
reencoded, and so on.You can see this with
many Linux distros: there is no single Debian
torrent that people seed for years because
there's always a refreshed version.Distros are
a bad use case for P2P anyway since you depend
on upstream as soon as you start upgrading and
installing packages.
|
> > > > chmod775
You can trivially have storage
deduplication for the files served via
torrent, transparent to the protocol. The
most trivial version of this that you can
do today with pretty much any client is
having a single directory containing files
serving multiple overlapping torrents.
|
> > > > > skeledrew
> files serving multiple overlapping
torrentsThis sounds wildly complex,
especially from a discovery
perspective.
|
> > > > > > chmod775
I don't see why it would be. It's
transparent to other clients just
like it is to the protocol. It
cannot be more complex than
alternatives by construction.
|
> > > > NooneAtAll3
but model releases are already
non-changeable?
|
> > > > righthand
> Distros are a bad use case for P2P
anyway since you depend on upstream as
soon as you start upgrading and installing
packages.This is true for any distribution
method not just p2p. You can even download
a nightly through torrents so what does it
matter how the data is transferred if it's
always going to require `apt update`?
|
> > > cube00
If they're no longer using that model they may
not be willing to continue using their storage
for it.
|
> > > > csomar
How does this address the previous point?
If they are not providing storage, then
centralized or decentralized doesn't make
a difference.Torrent/P2P can only add
redundancy, so it's impossible to have
worse availability than a download link?
|
> > > > > _blk
You're absolutely right. The problem
is convenience. Torrent doesn't
integrate as nicely as curl, wget,
apt-get, npm, you name it or even in
the browser. It doesn't have to be
that way but it is currently.
Is there a fully in-browser torrent
option that has the same UX as a
regular file download in Firefox? Even
better for others/the system would be
to then keep sharing by default as
long as the file's on your drive..
|
> > > > > > pell
> Is there a fully in-browser
torrent option that has the same
UX as a regular file download in
Firefox?Opera did back in the day.
|
> > bilegeek
The biggest problem with BTv1 was the lack of
per-file checksumming, and swarm merging (i.e.
individual files have shared seeding pools across
torrents). BTv2 specs the latter, but I think only
BiglyBT actually implements it. Having both of
those features from the get-go would've gone a
LONG way to fixing the dead torrent problem.
|
> > zenoprax
You only need one person/organization to commit to
seeding. The majority of people do not want to
seed at all without some sort of incentive.If this
site represents a coordinated datahoarding effort
then there will be at least a few people who will
seed indefinitely.
|
> > > csomar
The last guy (kimdotcom) who was working on
this (incentive for seeding) seems to be
heading to the US:
https://www.rnz.co.nz/news/science-and-technol
ogy/651123/cou...It's interesting he's no
longer getting any media attention any more.
|
> > Retr0id
A torrent with a webseed is strictly more
resilient than a direct download link alone.
|
> > ranger_danger
I always wondered why v2 is never used... you can
even search for files by their individual hash
with it.
|
> > > mitxela
Because history is path-dependent, as
engineers keep learning over and over again.
It doesn't matter whether Plan9 is
theoretically superior to Linux - we're all on
Linux and nobody's porting all the apps over.
|
> TeMPOraL
Yeah, I thought they were used for this already.
Surprised this is news, but also relieved.
|
> > Oxodao
IIRC Mistral used to do it, not sure if that's
still the caseEDIT/ Yes they did, that no longer
seems to be the case
thoughhttps://x.com/MistralAI/status/1833758285167
722836
|
> boredumb
ages ago I tried using IPFS to more or less accomplish
this, I imagined it to act more like a
weights/training data network fs that everyone would
be able to participate in.
|
mococa
Steam & Blizzard (probably others) used to delivery games
through torrent protocol in the past, before CDNs became
cheaper.When StarCraft 2 was lauched, the installer
(before Battle.net installer crapware) had a complete
graphical visualization of seeders & leechers.Reference:
https://warcraft.wiki.gg/wiki/Blizzard_Downloader
|
> bayindirh
Oh, story time:Once I was using Blizzard's downloader
to install something (StarCraft, Diablo, I don't
remember), and it was kinda slow. I disabled P2P
downloads and speed skyrocketed, and I said "Huh, this
was unexpected".When P2P downloads disabled you could
see the list of CDNs you're downloading from and mine
had a single IP on that list. It looked familiar. Then
it dawned on to me. It was the Akamai server which we
were hosting in our system room, at 15 minutes of
driving distance. After a chuckle, I went to get a cup
of tea, because that was entertaining than the game
itself.Then of course, I dived into whatever I was
installing that night.
|
> alexpotato
I've worked at trading firms where the "reference data
master" file is usually a big json or equivalent.To
get the file out to 100s or 1000s of machine they
would often use private bittorent to distribute the
file out.
|
> > mococa
Same here. Inspired by both, I used to delivery
videos over torrent to in door machines since at
that time, there's no CDNs (or it was difficult to
get one).
|
> > mitxela
As a private network, they have the option of
multicast.
|
> Aurornis
Several companies tried this for distributing
software.It was very controversial. Users were angry
that software companies were using their internet
bandwidth to distribute their software. Made a lot of
people angry.
|
reedf1
I claimed the "reedf1" handle on Pirate Face.Where Hugging
Face AI models never die, and are immortalized as
torrents.Claim yours:
https://pirateface.co/claim?ref=reedf1
|
ganessh
I claimed the "ganesshkumar" handle on Pirate Face.Where
Hugging Face AI models never die, and are immortalized as
torrents.Claim yours:
https://pirateface.co/claim?ref=ganesshkumar
|
mannyv
I can see all these old models hanging out and bitching
about the newfangled models that think they're so
great."Trillions of parameters? Trillion my ass. Back in
my day we got by with millions. And now here we are,
washed up has beens."
|
skeledrew
Arguably this should've been a thing since day 1 (and
probably would've helped to prevent the buyout), but
better late than never.
|
> RobotToaster
Hugging face seemed like buyout bait from day one.
|
> > hirako2000
The GitHub for models and AI. I remember.When a
business subsidizes for several years all its
offerings, one can guess how it will end.
|
> > Avicebron
Which makes it more surprising that something like
this didn't exist. I'm relieved it does though.
|
jamienk
Let's imagine that a model is pulled from HF by order of
the new overlords or because of some other kind of
censorship. Wouldn't the question of it having a Free
license or not potentially become a complex legal
issue?But if the point is to be "censorship-free" then why
respect licenses at all? They are among main choke points
today. If authoritarians use licenses to censor political,
artistic, scientific, etc., speech that they want to
block, does that make the censorship more respectable?When
Anthropic sues a Chinese lab for IP infringement and get a
court to put a bar on that software, does it THEN get
pulled from Pirate Face?I know that an awful lot of
international negotiations have become focused more and
more on questions of "IP" - licensing battles are already
intensely politicized and it's hard to imagine a future
where it doesn't get much much worse. Imagine N Korea
coming after you for violating a license that they worked
hard to control and leverage.
|
hgoel
Torrents always seemed like the more sensible way to
distribute model weights.Though I have been disappointed
that most of these have been spurred on by the misleading
claim that abliterated models were being taken down from
HuggingFace because they removed an abliterated model. HF
took one abliterated model down because the uploader was
spamming people who requested access with sketchy
requirements to pay for it.Plus, there are a bunch of
these types of sites, all of them have a couple of models
and otherwise completely dead.There's also the problem of
catching malicious models that have been fine tuned to
exfiltrate credentials. It would be nice to have means of
checking hashes against the HF versions (or against other
reputable sources). I'm guessing this is probably easy
when just serving the same folder as what HF serves.Edit:
I should've scrolled down on the page, it does verify
against the HF hashes.
|
wolfy1993
I'd support this if I didn't have to post to xitter to
claim my username.Really odd approach.
|
> skepticalgenius
I claimed the "skeptical_genius" handle on Pirate
Face.Where Hugging Face AI models never die, and are
immortalized as torrents.Claim yours:
https://pirateface.co/claim?ref=skeptical_genius
|
reilly3000
I've been wondering when this will come. The days are
numbered for abliterated models to be published on HF I
think. Why wouldn't the government want a central control
there?
|
kevinsimper
That is pretty smart, torrent should have been used for
more things, and this is a perfect usecase!
|
grommz
If I search for 'uncensored' there are no torrents
available. Uncensored models should be top priority,
especially now that Nvidia owns HuggingFace and will
enshittify the platform in accordance with upcoming US
laws.
|
> sneurlax
There are such results on huggingface tho. Also search
for the keywords "abliteration" and "heretic". Heretic
is a tool used to abliterate, that is decensor, models
|
> behole
That was my my first search too. I know they are still
listed on HF but every time I try and use one, the
links are dead or the size is beyond my scope. Was
hoping for a fresh batch. Ill check back.
|
casper14
Honestly, models are torrents will end any effort from the
big AI labs to stop open models. No way to prevent weights
from being shared, just like mobies. Genie is out of the
bottle
|
Razengan
So begins the Merovingian and the Exiles..
|
yieldcrv
great initiative, it's really weird seeing efficiencies
get rediscovered in the LLM audience, because these
efficiencies aren't even what I would consider to be
oldbut I guess they are
|