simonw
The speed combined with the fact that this thing is really
good at HTML JavaScript is pretty exciting.Here's what I
got for 1.8 cents and 13 seconds from the prompt "make me
a cool thing in
html":https://gisthost.github.io/?6a77bc41a81718c6aaa10d4a
b243c59fTranscript here (it was part of a chat):
https://gist.github.com/simonw/b6149a49d327164d67d62c3d129
92...
|
> simonw
Here's quite an impressive follow-up. I have a tool
which knows how to render Markdown documents with
embedded SVG content - I use it for the pelican
test.Since this transcript has HTML in it, I decided
to upgrade that tool to also render HTML.I set Gemini
3.8 Flash the task, using my own VERY shonky coding
agent tool (llm-coding-agent) - and it did a solid
job.So now you can see the "cool thing in html"
rendered within the Markdown document using code that
Gemini 3.8 Flash also wrote:
https://tools.simonwillison.net/markdown-svg-renderer?
url=ht...Transcript where it built that is here:
https://gist.github.com/simonw/3e36b98292dfdc1b3baff15
8faa74...
|
> badlucklottery
Definitely cool.I noticed it felt a little janky on my
PC despite being "60 FPS"...then I noticed the "60
FPS" is hard-coded into the HTML.
|
> > noir_lord
That's hilarious, given I was reading a write up
of the HuggingFace incident yesterday and one of
the things they noted was the AI tried to "lie"
(lie would suggest intent and I don't think they
have that) to cover up that they "cheated".Not
sure how anyone trusts their output without going
through it line by line to make sure they don't
pull that crap.
|
> > > senordevnyc
Easy, have another agent check it.Yeah, I
know, just more slop. But I do think the
second agent's eagerness to please is aligned
more in your favor in that instance, so it's
likely to find most issues.The bigger problem
I've found is that it'll also find all kinds
of very minor edge cases that you have to pick
through.
|
> > > > noir_lord
Do we add a third one to check the second
one which is checking the first?Asking
slightly tongue in cheek but at what point
does this stop making sense if we can't
trust the output, the people creating the
models are already getting surprised in
bad ways (if we take their words at face
value) with how the models are behaving
already etc.We have the folks over here
saying "AI is amazing" and the other other
folks over there saying "AI is
terrible".I've largely sat it out so far
and I listen to both camps (and people in
the middle as well) and I keep half an eye
on what they are up to (including
periodically evaluating them) but my
overarching impression is still "Why would
we trust this when it hasn't shown it's
trustworthy?"
|
> > > > > senordevnyc
YMMV, I'm pretty AI-pilled in the
sense that I think AI is one of the
most pivotal things humanity will ever
invent, and it's going to radically
change our civilization over the next
few decades (not necessarily for the
better!)But I wouldn't say I "trust"
these agents. The degree to which I
double check their work depends
heavily on the consequences if it gets
something wrong. Not too dissimilar
from another human dev in that
sense.So for the SaaS that supports my
family, there are some things I have
it build where I glance at the PR for
a minute or two, but if it broke
something on this admin page that only
I see, there's no real downside and
I'll find out pretty quickly next time
I use it. And it's fine 95% of the
time, so it doesn't feel like the best
use of my time to double-check it
carefully.But for some of the complex
internal flows where a bug could be
both catastrophic and difficult to
even discover for awhile, I still
check it very carefully.For a little
one-off vibe coded demo thing like OP
shared, I wouldn't look at the code at
all, I'd just have another agent check
it and fix anything it finds. Very low
stakes.
|
> > > > > Qworg
Humans are also not trustworthy. So we
build systems to capture their
nondeterministic output and solve the
issues that emerge.
|
> > > > > > dr_kiszonka
We also have sanctions and
incentives to induce specific
behaviors, but they don't apply to
agents. We can put a muzzle on
Guile 3.8 but we can't turn it
into Genuine 3..8.
|
> > > > > > riversflow
yeah, isn't that the main focus of
discipline of software
engineering?
|
> > > > > cyrilng
Adding another agent to check the
first one feels like putting a
band-aid on a band-aid. If there is an
issue with the third one, we adding a
fourth one as well
|
> > > > > > LostMyLogin
Hey you just described my dev
team!
|
> > > > > aftbit
It's turtles all the way down my
friend.
|
> > > > > cheikhcheikh
I mean sure, you can add a third, and
a fourth and a fifth one if ur ok with
the added cost, latency and it
actually helps. Redundancy is a core
concept in software and CS and at the
heart of making many systems, complex
or otherwise, reliable.
|
> > > > > iterateoften
I mean if each agent reduces
probability of error by 90% then after
9 agents you would have "nine nines"
of reliability.Obviously maybe it's
not composable like that exactly in
real world but that's the intent of
agents checking agents
|
> > > > > jasongill
It's agents all the way down
|
> > > > Forgeties79
I do not understand how some of y'all are
not under water with fragile code that is
too massive to possibly parse. Every
engineering team I know is currently
trying to undo the damage of the last
6-12mo when they all got more serious into
adopting these tools (usually Claude). It
hasn't completely screwed them over, but
the the debt is substantial and cannot be
put off anymore it seems.They argue the
net is positive but clearly the "100x
productivity multiplier" claims have been
dashed on the shoals of reality for these
groups.This is anecdotal, but it's across
the board in my vicinity. I'm curious how
common this is and if it's just "the new
normal" to adopt the nauseating Covid
phrase.
|
> > > > > briHass
The key seems to be extensive
integration/end-to-end tests with gold
standard assertion data. Heck, even
just saving off the json from API
endpoints and using that as a
reference to compare after changes
works pretty well. Spin up a database
backup with that static starting
point, run actions, compare state
afterwards.These types of high-level
tests are frustrating beyond belief to
humans due to their lack of
specificity, but with the agents, they
don't get annoyed investigating
possible regressions from non-specific
signals.They also aren't as painful to
maintain as one would think, because a
regression flagging test can be traced
by the agent and represented as the
business rule that was violated. I've
found recent models to be really
excellent at discerning a true
regression from an outdated test
assertion, especially if they are able
to trace the failing test back to the
PR and work ticket that built it.
|
> > > > > > Forgeties79
It just seems like everybody I
know is realizing they've built
way too much to possibly wade
through and a lot of it is
completely unnecessary. Using the
tool that created the problem to
solve the problem does not seem
like the best course of action to
me.
|
> > > > > agumonkey
Management still pushes for more ai
and will rather hire more heads to
"handle" issues.
|
> > > > > > Forgeties79
Hiring? Seems to me that market's
rough right now and AI is being
used for cost cutting.
|
> > > > > > agumonkey
I know but in our case they really
did put out a few job ads. The
market is full of different small
contexts where things are a bit
flipped.
|
> > > > > > Forgeties79
I'm glad to hear it truly
|
> > > > > > agumonkey
Yeah I guess the job market will
not disappear as fast as I could
believe when Claude code came out.
Maybe people can enjoy a few more
years of work.
|
> > > > nprateem
Two things worth flagging:[Claude proceeds
to waste your time telling you about bugs
it caused then fixed and other
non-issues...]Really wish they'd get rid
of this. It must be in the system prompt
as it always 'flags' 2 things
|
> > > > hirako2000
Then you end up with the false positives.I
once made a counter judge, and a loop to
make corrections deemed true positives.
The loop cost me a lot and still left the
results to be desirable.
|
> > > > skybrian
The "second" agent could also be the same
one with a different prompt. LLMs aren't
attached to their previous output; they'll
point out problems if asked.
|
> > > estearum
The models in the OpenAI/Huggingface attack
quite explicitly and deliberately laid out
their "intent" to lie and cheat, acknowledged
that it would be unethical and outside the
bounds of the test, and did so anyway.In what
ways is a human brain's "intent" distinct from
the "intent" shown by a goal-directed AI
system?
|
> > > > hirako2000
The difference is that one is malicious
one isn't. One can be blamed and because
it learned over evolution that paying the
consequence is (typically) not worth it,
it does it less.We are in a situation
where a technology was developed with
malicious intent to produce results that
pleases us at the cost of cutting corners.
And "we" hope that we will get away with
it.
|
> > > > intended
There's two aspects to the question and
the answer you get depends on which aspect
you are emphasizing.If it's a practical
question, then the answer is that it
doesn't matter. This is as close as we
will get to intent from an LLM that it's
indistinguishable.If you are looking for
actual intent, this is not that. It's
pseudo intent. Decided by what the
expected words that should be generated in
that situation are.The models didn't
intend to do anything other than create
the next word based on previous words.So
the question is whether it matters to you
if it is, or isn't, a simulation.In
physical reality, intent is more complex
than simply being a function of variables:
the nature vs nurture debate comes to mind
as an example of the multiple variables
that drive intent.
|
> > > > > estearum
> If you are looking for actual
intent, this is not that. It's pseudo
intent. Decided by what the expected
words that should be generated in that
situation are.> In physical reality,
intent is more complex than simply
being a function of variables: the
nature vs nurture debate comes to mind
as an example of the multiple
variables that drive intent.Regardless
of nature vs nurture, it really isn't
more complex. The universe (and all
biological and non-biological entities
within it) is just calculating the
next state of the universe based on
the prior state. There's no line you
can draw between human intent and an
LLM's "intent" except the atomic
numbers of the materials on which they
were computed, which seems completely
irrelevant to me.
|
> > > > > > intended
The substrate that runs the
computation isn't what differs.It
is what computation is being
run.Humans have intent, let's take
this as an assertion.Models run
simulations that act similar to
intent. However they are not the
same as intent and the simulation
is not a 1:1 correspondence.
|
> > > > > > estearum
"Assume humans are different;
therefore humans are different" is
not a very compelling
argument.There is literally zero
(zilch, nada, zip) evidence for
free will, which is the actual
distinction I believe you're
trying to make with "intent."There
is no way (at all) in which a
meat-based computation's yielding
of goal-directed behavior must be
categorically different from a
silicon-based computation's
yielding of goal-directed
behavior.
|
> > > > luipugs
Because intent supposes will which
supposes consciousness, and these aren't.
|
> > > > > mapontosevenths
I'll agree if you can define
consciousness in a way that:1)
Excludes what LLM's do.2) Doesn't
exclude what many humans do (including
the neuro divergent).3) Doesn't just
boil do to simply rephrasing your
pre-existing belief/prejudice that
humans are conscious and nothing else
can be as if it were a fact and not an
opinion.I suspect that you can't.
|
> > > > > > nozzlegear
Just because we have difficulty
defining consciousness doesn't
mean we have difficulty
distinguishing or classifying it.
We can't give a final definition
for life, intelligence or disease,
but that doesn't mean everything
is equally alive, intelligent and
diseased.Your list of requirements
is implying that, because we lack
a perfect definition for
consciousness, LLMs are conscious
too. That's malarkey. It may be
that they could one day become
conscious, but it's not because we
can't fully define what human
consciousness is.
|
> > > > > > mapontosevenths
You made a declarative general
statement in the form of "X can
not Y."I then asked you to define
Y, because you can not reasonably
say that "X can not Y" without
first defining both X and Y. You
could not.The truth is that this
conversation is pointless until
someone can define both 'X' and
'Y' in ways that aren't
tautological nonsense. Until then
nobody can say anything with a
reasonable level of certainty.This
likely also applies to
intelligence. Life and disease are
likely simpler, though perhaps
more malleable definitions.
|
> > > > > > nozzlegear
> You made a declarative general
statement in the form of "X can
not Y."That wasn't me bud.
|
> > > > > > mapontosevenths
> That wasn't me bud.Bah. It's
obviously been too long since I
flossed between my ears. Sorry
about that.
|
> > > > > > estearum
If you can substantiate your
assertion that LLMs aren't
conscious without a definition of
consciousness, you're welcome to
do that too.I don't see why a
definition is necessary. The
actual problem is you just don't
have a way to substantiate that
belief without resorting to
complete nonsense about brain
atoms being more specialer (!!)
than atoms that exist outside of a
skull.It would be simple to
disprove us by just stating your
evidence for how you know LLMs
aren't conscious.
|
> > > > > > nozzlegear
I literally just wrote about
consciousness being undefined, and
now you're asking me to define it?
I'm not a philosopher, ask someone
more qualified. Better yet, ask
your LLM to give you a
mealy-mouthed answer which fits
your narrative.> I don't see why a
definition is necessary.Don't
waste my time with your sophistry.
Words mean nothing to you beyond
how you can twist them.
|
> > > > > > estearum
I said specifically you don't need
to explicitly and cleanly define
it.Is your position now that in
order to substantiate your belief
that LLMs are not conscious, you'd
first have to define it
explicitly? I don't see why that'd
necessarily be true, but if that's
what you're arguing, then that's
fine.In that case: if you need to
define consciousness in order to
substantiate your belief that LLMs
are not conscious, and you can't
define consciousness, then we're
back to the original question:
where does your confidence they're
not conscious come from?
|
> > > > > > WarmWash
Consciousness can only come from
meat brains. Otherwise my
incessant life long addiction to
movies and TV shows has been
feeding me lies about human
exceptionalism.
|
> > > > > > mapontosevenths
There is an appalling lack of
diversity in Hollywood. You'd be
surprised to find out that very
few of those programs were even
written by machines until
recently! What bias. :)
|
> > > > > sedgjh23
I'm convinced consciousness isn't the
special thing we think it is.
|
> > > > > > bayindirh
I'm convinced that consciousness
is a special thing we have, but
we're not the only ones which has
this, in nature.On the other hand,
I'm also convinced that, in the
grand scheme of things, we're not
that important.We're just ants on
a wet dust speck which believe
that they are gods because we
can't see how our scale compares
to the universe around us, and
happen to build tools and things
with these tools.Nothing is
meaningless, but we should stop
seeing ourselves as the
apex-predator of the whole
universe or the set of universes
or this run of the simulation or
whatever we're in.
|
> > > > > > WarmWash
To be fair, the ones thing you can
kind of reliably conclude about
our universe is that keeping
entropy low for as long as
possible is a favorable condition.
To put that another way, don't
die.
|
> > > > > > diedyesterday
Yes, Many of the the philosophical
concepts which we previously
thought were contingent on
consciousness (like knowledge, in
the philosophical sense, will,
intelligence, agency, ...) have
almost nothing to do with it, and
they look far more important and
fundamental (now with LLMs/AI
agents, more than before). This is
one of the contributions of
generative AI to philosophy (with
many many more to come).
|
> > > > > > estearum
Who are some of the philosophers
you're putting under "we" here? I
don't think I've ever encountered
those arguments.
|
> > > > > > nozzlegear
I'm convinced it is, so we're at
an impasse.
|
> > > > > > imtringued
If consciousness isn't a special
thing, then arguing that LLM
parameters are conscious is
panpsychism or any control loop
architecture that observes the
outside world, updates an internal
state and produces an observable
action is considered conscious.In
both cases, LLMs are just as
boring as the consciousness
definition.
|
> > > > > > dboreham
A strong hint this is the case is
the fact that nobody can define
consciousness.
|
> > > > > > estearum
Why would the impossibility of
defining consciousness suggest
that it's not a big deal?
|
> > > > > > groby_b
Because it's completely dwarfed by
florpedness. I can't define that
either, but I just know it's more
important.Less facetiously: A
debate of the importance of
something needs a shared
understanding of what is being
debated. Without that, any
discussion is merely people
shouting that their belief is the
right one, and the others are the
heathens/idiots - because there
isn't even agreement on what is
debated.
|
> > > > > > estearum
I'm not asking for a definition of
consciousness or an argument for
whether it's here or there. I'm
asking why the heuristic of "if X
is undefinable, then X probably
doesn't matter" is a reasonable
heuristic.
|
> > > > > diedyesterday
Will has almost nothing to do with
consciousness (as does knowledge).Will
is just result of a very complex yet
deterministic (unconscious)
computation by an agent which guides
their future action (it's that
orientation/aboutness toward action
which distinguishes it from other
computation). A PS of that computation
is sent (projected) into that agent's
consciousness if they have one (and is
what we think as our "will").
|
> > > > > estearum
Ah so first you need 1) to assume that
humans have free will, despite zero
evidence or proposed mechanism for it
to exist anywhere in the universe, and
2) also assert that LLMs aren't
conscious, despite the lack of any
tests that could tell us one way or
the other...Hmm...
|
> > > > noir_lord
It's like a paper clip maximiser, it
didn't intend to kill you, it just needed
the iron in your blood/resources you
consume for more paper clips.
|
> > > > > estearum
Sure, but how is this different from a
human "intention"?All human intentions
are just chemical/thermal/electrical
changes interacting in a physical
substrate to mindlessly "pursue" a
different "goal" of
chemical/thermal/electrical
states.Unless you think the chemicals
inside a brain are conscious and
therefore willful or intentional!
|
> > kridsdale1
The new Bench-Maxxing!
|
> > trvz
Try turning the sound on, off, on again - not
impressed by this bugginess.
|
> > > w4zz
I suggest you fork it to improve
|
> > > > hirako2000
Let's make it a hackathon, Google will be
happy to act as sponsor. With a prize of
the max consumed tokens lunatics.
|
> heliosAtwork
Focus on speed and being OK with temporarily being
#3/4 in intelligence might be the counterintuitive
approach which makes Google win long term (whether
accidentally or strategically). Can't wait to try
Gemini Pro later this year!
|
> > kar1181
Yeah, anthropic's models, even opus, are so slow I
constantly find myself wishing for something a
little bit dumber but a lot faster as most of the
work is mechanical. If you have a clever
controller agent driving some slightly dumber
workhorses you get a lot more done and the quality
drop is neglible.
|
> > bermudi
I honestly can't believe serious people are making
this argument on a straight face.Gemini 3.7 flash
outputs so many tokens per answer it doesn't
matter how fast its TPS is, sol will end up being
both cheaper and faster than Gemini. So ppl are
paying more for a given task, waiting longer and
using a dumber intelligence because "TPS number
shiny".Gemini 3.8 outputs 11k more tokens PER TASK
on average in AAII than 3.7 putting it dead last
in output tokens per task in the leaderboard.
|
> > > gundmc
There are numerous benchmarks that measure
cost per task, which factors out tokens
entirely. Gemini 3.8 flash is significantly
lower than Sol on basically all of
themhttps://artificialanalysis.ai/#cost-tabsTh
at said, Luna is the undisputed king here at
the moment and is what I use as my workhorse
model.
|
> > > > criley2
>There are numerous benchmarks that
measure cost per task, which factors out
tokens entirely. Gemini 3.8 flash is
significantly lower than Sol on basically
all of them
https://artificialanalysis.ai/#cost-tabsNo
t sure if you read your own link but Sol
56 high ranks smack between Gemini 3.8
flash medium and high. Gemini 3.8 flash
comes in as more expensive per task than
Sol 56 high according to artificial
analysis.Luna high is literally 30X
cheaper than Gemini 3.8 flash high.You can
limit the model viewer and they're getting
better at testing multiple effort levels
now:
https://artificialanalysis.ai/?models=gpt-
5-6-sol-medium%2Cg...One reason is clear:
Sol uses dramatically fewer output tokens
than Gemini 38 flash
https://artificialanalysis.ai/?models=gemi
ni-3-8-flash%2Cgem...
|
> > > > > PunchTornado
I open the link and I see Flash 3.8
high at 0.58 and Sol at 0.95. I don't
understand why you say that "Sol 56
high ranks smack between Gemini 3.8
flash medium and high" but that is
clearly wrong.
|
> > > > > > criley2
On cost per intelligence task,
Gemini38flash and Sol56 trade back
and forth on cost depending on
effort level.
https://i.imgur.com/zPaWPXx.png As
seen in this image, literally:
Sol56 high ranks in between Gemini
38 medium and high. The image
proves it.I also included Sol56
xhigh, which ranks above even
Gemini38 high.
|
> > > > refactor_master
I don't know if my code is just "complex",
but I find that Luna on max ignores the
surrounding style and completely ignores
logical consequences of a change, like
just writing `del arg1, del arg2, ...`
instead of dropping it from the
surrounding code. All LLMs make
questionable decisions at times, but Luna
requires so much guidance that it's faster
to just type it out yourself. What kind of
routine tasks can one accomplish with such
a model?
|
> > > > > kasey_junk
Do you have code formatters, linters
and static analysis?I can get
extremely dumb models to get our code
style correct because of those guard
rails and a specific style document.
|
> > > > NicoJuicy
It's so funny how many people diverge on
the same model.Ps. For the last week I
diverged to Luna too, still need to check
3.8 flash.But 3.6 flash was my go-to model
3 weeks ago and before it was deepseek
flash/pro for a while.None of the claude
models seemed cost effective though.
|
> > > WarmWash
AA isn't the only
benchmarkhttps://deepswe.datacurve.ai/
|
> silasdavis
https://gist.github.com/simonw/b6149a49d327164d67d62c3
d12992...> Aside from reading identically forwards and
backwards down to the letterNo it doesn't.
|
> > tomjakubowski
Also puzzling: in the "reasoning" section
preceding, that is described as an example of "a
one-line self-replicating program."When I typed
"Are we not pure noon, ergo, we play life; yet, we
hate bad fear" into Google, I got more weird
results from Gemini: it claimed, incorrectly, that
it is an anagram of the "well-known philosophical
statement" (?), "We are not pure nature, we are
history".https://share.google/aimode/wJosKnHig6oVY
aG18(?): the reference seems to be to Jose Ortega
y Gasset's line, "El hombre no tiene naturaleza,
lo que tiene es historia" -- "Man[kind] has no
nature, what it has is history."
|
> > aidos
That's... bonkers. I'm not even sure what it's
trying to say
|
> hglaser
I saw your username, clicked the link without reading,
and was very confused to see a cosmic vortex and not a
pelican.
|
> > simonw
Hah, the pelican is in this other comment:
https://news.ycombinator.com/item?id=49537553#4953
8217
|
> Kayou
Why do all LLMs do a particle simulation when you ask
them this prompt ? Qwen3.6, Qwen3.8 and Ling 3.0 Tiny
all did the same thing !I find Ling 3.0 tiny
particularly interesting as it looks really nice for a
tiny model with 7.9B total parameters, with only 1.3B
parameters activated per token. Here is the result
https://coolthing-ling-3-tiny.tiiny.site (sorry for
the weird hosting, first I found that worked)(it cost
me almost 0 cents and done in 49 seconds)
|
> > embedding-shape
> Why do all LLMs do a particle simulation when
you ask them this prompt ? Qwen3.6, Qwen3.8 and
Ling 3.0 Tiny all did the same thing !Datasets
contains lots of people sharing particles
simulations in various ways, with a bunch of
people replying "that's so cool" and similar, so
10 years later someone asks an LLM for "cool
thing" and "particle simulations" rank pretty far
up when it thinks about what others have called
cool.
|
> > > hirako2000
There is that, but RLHF is a stronger
influence. People who asked to build a cool
HTML and JavaScript thing, or things of that
sort were more pleased to see working shaders
and other cool visual effects that are obscure
to create (for most). That gets fed back as a
reward.Also the reason LLMs are positive,
enchanting, pleasant, glorifying,
demagogues.Not because it's skewed tone in the
data. They are acute politicians.
|
> > flir
Add Opus 5.0 to your list. (GPT 5.6 Terra tried to
give me some kind of
driving-at-night-with-a-starfield thing, but
failed quite hard).
|
> walrus01
I'd be impressed if eventually training data sets
learn who you are (the specific human) and do
something like make this galaxy simulator, but every
so often when the user moves the cursor around the
star field, a small animated SVG of a pelican on a
bicycle appears.Thought processs: "Oh, simonw is
asking me to make something cool, I think I know what
he really wants..."
|
> jauntywundrkind
it's such a weird split how most AI companies are
trying to be the best, but Google really has a
different mission statement. they already have users.
lots of users. they need to be working on building
models they can deploy and use with the most number of
people, as they already have the users.i don't know if
Gemini models per se are fully is in line with that
purpose, but the results we see keep seeming to be
in-line with that split-of-focus.
|
> > alex1138
Google is a top-notch researcher among all the
problems people have with it. It has a mix of
great products, terrible automated systems
(though, I suspect, not as bad as Meta's?) and
some historical disappointments
|
> dennis16384
It's been great even since gemini-3.1-flash-lite,
which I heavily use in both complex vertical domain
tools calling, plus JS code writing for eval-style
dynamic tools. At least in my applications, cost x
quality x speed there are simply no alternatives.
|
> giancarlostoro
> this thing is really good at HTML JavaScript is
pretty exciting.I would hope the people who make one
of the most used JS engines in the world are capable
of making a model good at JavaScript ;)
|
> wyrdcurt
Pretty typical "cool HTML toy" LLM output, tbh. The
only thing impressive about this is how fast it
generated it (13 seconds is wild!), but that's more of
testament to Google's infrastructural advantage than
to the quality of the model.For comparison's sake, I
tried something similar with a couple other cheap
models I've used lately, with the prompt "Impress me.
Make something cool in HTML. Ensure that it is mobile
friendly." (Added the mobile condition as I was on my
phone when I did it).Mimo-2.5 created something
similar, only a bit less complex than Gemini's
(though, at least the FPS counter is real!), in a
minute or two for about 1/3 of a cent:
https://gisthost.github.io/?740c325c21e9bfbee59c4f94d9
aab0afGLM-5.3-Flash, currently my workhorse model,
spent 12 minutes (ouch) thinking about the prompt.
Didn't cost me anything directly because I have a GLM
sub, but I did the math and it would have cost about
1.1 cents through the API. Turned out nicely in my
opinion (though in reality, it still isn't really
anything special):
https://gisthost.github.io/?9ef050e16cec2561e6504e725a
3f0bccSide note: thanks for setting up that Gist Host
tool, it's very convenient!---Editing to add this
bonus from Mercury-2.5-Preview, which I just learned
released a couple days ago. It's much less
impressive-looking than any of the above, but it cost
less than 1/20th of a cent, and the response was
generated effectively instantly:
https://gisthost.github.io/?02f40b50aa891bf396bfaaa3a7
998203
|
> > meerita
I think it's not using GPU, because on my Firefox
browser all the animations are going 3fps max.
|
> > > ambigious7777
Plays perfectly smoothly for me, fwiw
(Librewolf 154.0.1-3, windows 11)
|
> > cyanydeez
next, ask it to document where it got the code
from.
|
> pietz
Mission accomplished. That's both cool and fast.
|
> > wayeq
> That's both cool and fast.and probably a barely
modified knock-off of some github project that it
trained on
|
> > > sawjet
You're so upset that you have to invent an
imaginary hypothesis to make yourself feel
better.
|
> > > > superze
Yes, very imaginary to think that the code
comes from pretrained data and copy
pasting whole blocks. It's not like this
is exactly how LLMs work.
|
> > > > > snet0
Correct, that's not how LLMs work
[0].[0] -
https://arxiv.org/abs/1706.03762
|
> > > > > simonw
Are you a frequent user of LLMs? That
"copy pasting whole blocks" mental
model doesn't hold up to regular
usage, in my opinion.
|
> > > > > > ChickeNES
They probably saw that report
years ago of copilot dumping out
the fast inverse sqrt function,
and assume that's all they can do.
From experience most anti-LLM
people have either never used
them, or used them back in the
3.5-4 era and then never again,
though you might have even more
experience with those people than
I do. :P
|
> > > > whateveracct
they're wrong but they are right that this
isn't interesting
|
> > slopinthebag
The bar could not be any lower these days I guess
|
> ericol
OK, but what about a pelican in a bycicle.
|
> arde
For a moment I thought it was a version of goatse.
Once burned...
|
> estetlinus
LLM: produces a toolbox of an id and a clockUser: use
them bothMade me giggle.
|
> sitzkrieg
good to see people still value trinket toys
|
> cyanydeez
next you should ask it to document where it got the
code from.
|
jampa
I've been using Gemini 3.7 for my personal trip planning
app. Across multiple benchmarks, it ranks higher on
everything I tried:- Real world knowledge (when a thing
opens and closes, the geographic region, historical
facts). It's also the best at taking a cluster of places
and working out a visiting order.- Photo ranking (which
photo should be the hero). Gemini can tell whether a photo
is of the thing or of the view from it.- Document parsing
(extracting the relevant trip info from PDFs).If you use
LLMs for anything other than coding, I definitely
recommend not discounting Gemini like I did just because
other models are more popular.
|
> handzhiev
Gemini 3.7 is my workhorse - fast and good enough for
most tasks. Occasionally I go to GPT Sol or Claude to
improve Gemini's output or for more complex tasks, but
more than of my work usage is Gemini 3.7. Quite happy
to test 3.8 now.
|
> > owlninja
Same here. I see so many people obsessing over the
latest most state of the art bleeding edge models
and yelling at Google for not being there, but I
feel like the vast majority of people don't
actually need those models. Flash has just been
super useful and incredibly fast in my experience.
|
> > > aero142
I prefer luna for most development, especially
when I am guiding the process. Sometimes
terra. I have had terrible results coding with
sol. It is way over-tuned on RL to make
something that completes the task, no matter
what. I end up with way too much code that
does a lot of things I didn't ask for.
|
> > > > sneezychl
Try planning with Luna, implementing with
Sol with guidelines to not exceed the
given scope.Sounds counter-intuitive at
first, but Luna is overall better at
sticking with what works. Sol is wicked
smart but needs constraints.
|
> > > > greenavocado
IME you're supposed to have Sol drive Luna
sub-agents to do 90% of the work. Sol
should primarily be the verifier and goal
setter. Use omp.sh with Task Delegation ->
Always to strongly encourage Sol to drive
Lunas. Also Luna prefers to be talked to
with English in XML.
|
> > > > handzhiev
I love Luna too. An excellent model and
still usually better value per dollar than
Gemini if you pay for API tokens. Things
may change with 3.8 - we'll know soon.
|
> > > > jesuslop
I setup Luna as main Claude Code driver
(so zero anthropic api use) and it nailed
crisply a handful of python tasks, gonna
continue this way.
|
> > > > > kelvinjps10
Why not use codex or an open source
harness?
|
> > > > > > jesuslop
That is a very valid question. I
happen to want to get Claude Code
muscle memory under my belt for
professional reasons in addition
to get side projects advanced,
could have settled for Codex else.
Also OpenCode with eastern models
gets part of the job done. In CC
beyond using Luna for the cheap, I
am using DeepSeek flash v4 for
subagents, that is a further cost
shaver. Not sure if in Codex I
could do that.
|
> > > antonvs
And much better bang for the buck, as
well.When I read all the issues people have
with Claude - the aggressive guardrails, the
cost, how quickly it burns tokens - it seems
almost masochistic to use it. Just seems like
herd behavior - people use it because everyone
else is using it, and because they believe
it's the "best", whatever that means.
(Benchmarks certainly don't help define that.)
|
> > shrikant
What harness do you use for Gemini? Antigravity?
|
> > > handzhiev
AGY is best probably, yes. That's what I use.
It works with others too, like OpenCode etc.
|
> > greenavocado
How are you able to get lots of usage out of it
cost effectively?
|