Today | News | Books | Recipes
Notes | QuickTake | Wiki | Browse
Maps | Reference | Reddit | YouTube
Chat | Games | About

GitHub - kyle-pena-nlp/jevchat: Turns Jev into a chatbot · GitHub

Open Original Page

GitHub - kyle-pena-nlp/jevchat: Turns Jev into a chatbot · GitHub

Sign in Appearance settings

Platform AI CODE CREATION GitHub CopilotWrite better code with AI

GitHub Copilot appDirect agents from issue to merge

MCP RegistryIntegrate external tools

DEVELOPER WORKFLOWS ActionsAutomate any workflow

CodespacesInstant dev environments

IssuesPlan and track work

Code ReviewManage code changes

Code QualityEnforce quality at merge

APPLICATION SECURITY GitHub Advanced SecurityFind and fix vulnerabilities

Code securitySecure your code as you build

Secret protectionStop leaks before they start

Solutions BY COMPANY SIZEEnterprises

Small and medium teams

BY USE CASEApp Modernization

BY INDUSTRYHealthcare

Resources EXPLORE BY TOPICAI

Software Development

EXPLORE BY TYPECustomer stories

SUPPORT & SERVICESDocumentation

Open Source COMMUNITY GitHub SponsorsFund open source developers

PROGRAMSSecurity Lab

Maintainer Community

Enterprise ENTERPRISE SOLUTIONS Enterprise platformAI-powered developer platform

AVAILABLE ADD-ONS GitHub Advanced SecurityEnterprise-grade security features

Copilot for BusinessEnterprise-grade AI features

Premium SupportEnterprise-grade 24/7 support

Sign up Appearance settings

You signed in with another tab or window. Reload to refresh your session.
You signed out in another tab or window. Reload to refresh your session.
You switched accounts on another tab or window. Reload to refresh your session.

kyle-pena-nlp

/

jevchat

Notifications
You must be signed in to change notification settings

Security and quality
0

Additional navigation options

Security and quality

Code Open more actions menu

NameNameLast commit message

Repository files navigation

jevchat turns that into a chat model. At every step it asks Jev one question:

Given the user's question and the reply written so far, which symbol comes next?

The options are an alphabet plus an option to stop emitting. Jev returns a probability for each
one, and the sampler draws the next symbol from that normalised distribution.
Append, repeat, and stop when STOP is drawn.

There are several alphabets (including truncated token lists) and sampling strategies available.

The idea is for fun, the cost is somewhat impractical, and the results are hilarious.

This was a Claude accelerated experiment. I described the sampling algorithms, strategies, and so on, and it implemented them.

Put your Jev key in .env next to pyproject.toml (git-ignored):

JEV_API_KEY and TYPESAFE_API_KEY are also accepted. Values in .env win over
exported ones, so editing the file is enough to switch keys.

poetry run jevchat # interactive chat
poetry run jevchat ask "do people need water?"
poetry run jevchat alphabets # what you can sample from
poetry run jevchat bench # compare every mode (table below)

The reply appears as it is sampled, in a panel with a live readout of the
generation rate - symbols/s, characters/s, milliseconds per API call, elapsed
time - and the top few symbols Jev scored at the last step, so you can watch the
distribution the sampler is drawing from.

Ctrl-C cancels. The first press stops generation once the in-flight request
returns and keeps the partial reply; a second press aborts immediately. In chat,
the partial reply stays in the conversation history. ask exits 130 when cancelled.

Chat commands: /help, /alphabet [name], /temp <v>, /stop-bias <v>,
/reset, /stats, /exit.

Two things are swappable: how the distribution over the next symbol is
obtained (-s/--strategy), and what it is over (-a/--alphabet). Every
combination below is a runnable command.

choice asks one question over the whole alphabet. bisect sorts the alphabet
and asks earlier/later yes-no questions until the group is small, then asks one
choice question inside it.

# choice - one question over the whole alphabet (the default)
poetry run jevchat -s choice ask "how many eyes do people have?"

# ...without the re-ordering that cancels Jev's position bias (worst mode)
poetry run jevchat -s choice --no-shuffle-criteria ask "how many eyes do people have?"

# ...averaging 4 re-orderings, sent as 4 parallel questions in one request
poetry run jevchat -s choice --ensemble 4 ask "how many eyes do people have?"

# bisect - earlier/later down to groups of 20, each split asked both ways
poetry run jevchat -s bisect ask "how many eyes do people have?"

# ...cheaper: bigger groups, each split asked once
poetry run jevchat -s bisect --bisect-cutoff 32 --no-bisect-swap ask "how many eyes do people have?"

# buckets - the alphabet split across many questions, each with an OTHER escape.
# The only strategy that can hold more than 255 symbols.
poetry run jevchat -a words1k -s buckets ask "what colour is snow?"
poetry run jevchat -a bpe5k -s buckets --bucket-size 127 ask "what is the capital of france?"

# refine - buckets, then a question over the winners, then a rescored nucleus.
# Twice the probability on the right symbol and ~19x the vocabulary resolved.
poetry run jevchat -a words1k -s refine ask "where do fish live?"
poetry run jevchat -a words1k -s refine --refine-nucleus 6 --refine-rounds 2 ask "..."

# hypothesis - options are the resulting texts (the default)
poetry run jevchat -p hypothesis --window 40 ask "what colour is snow?"

Links

Browse another page: