Cowry Tech News
AI Cloud

ChatJimmy is Llama 3.1 8B welded into a chip. It is very fast

chatjimmy.ai is the public demo for Taalas's HC1, a chip with Meta's Llama 3.1 8B built into the silicon. We tested it. The speed is real, by the service's own metadata. The trade is one model per chip, an input cap near 6,000 tokens that fails silently, and a 2024-era model. AMD agreed to buy Taalas in August. As of Oct. 21 it had not announced that the deal closed.

By Cole Harlan · 22 Oct 2026 · 12 min

Close-up of a silicon wafer patterned with chip dies, lit in rainbow colours
A generic silicon wafer, not Taalas hardware. Image: Rob Bulmahn, Wikimedia Commons, CC BY 2.0. Cropped.

Type a question into chatjimmy.ai and the answer can land before your finger is off the Enter key. The model answering is not new. It is Meta’s Llama 3.1 8B.

ChatJimmy is the public chatbot for Taalas, a Toronto chip startup. The site’s own model list, at /api/models, returns exactly one entry: llama3.1-8B, “owned_by” “Taalas Inc.” That is Meta’s Llama 3.1 8B, an open-weights model released in mid-2024. The new part is the hardware. Taalas built the model’s weights into the logic of a custom chip it calls HC1.

Taalas unveiled HC1 and the chatbot on Feb. 19 and 20, 2026, in a post by co-founder and CEO Ljubisa Bajic titled “The path to ubiquitous AI”. On Aug. 6, AMD announced a definitive agreement to acquire Taalas for an undisclosed price. AMD said the deal is “subject to customary closing conditions and regulatory approvals.” As of Oct. 21, AMD’s investor relations press-release page listed no announcement that the deal has closed; its most recent release there was dated Oct. 12.

This is a review. It asks one question for anyone building on AI: what do you get when you trade a general-purpose GPU for a chip that can only ever run one model, and when is that trade worth it?

We read Taalas’s technical material, its API documentation and the independent coverage. We also ran a small number of test prompts against the public demo on Oct. 21.

What HC1 is

Taalas’s product page lists the HC1 Technology Demonstrator as a TSMC 6nm chip with an 815 mm² die and 53 billion transistors, shipped in a 2.5 kW server. It runs one model: Llama 3.1 8B.

In a normal inference setup, model weights sit in memory, usually HBM stacked beside a GPU. The chip spends much of its time and power moving those weights back and forth. Bajic’s post says Taalas “eliminates this boundary” by “unifying storage and compute on a single chip, at DRAM-level density.” The company calls the results “Hardcore Models.”

Taalas says the design drops the parts that make modern AI servers expensive. Its post lists them: “no HBM, advanced packaging, 3D stacking, liquid cooling, high speed IO.”

The cost is flexibility. The weights are fixed, and Taalas says so. heise, the German tech outlet, put it directly: HC1 can run only Llama 3.1 8B, not arbitrary other models.

Taalas says two things stay adjustable. The post says the model “retains flexibility through configurable context window size and support for fine-tuning via low-rank adapters (LoRAs).” LoRAs are small add-on weight sets that adjust a model’s behavior without retraining it. We could not test LoRA support. The public demo exposes no such option.

The speed claim, and what we measured

Taalas’s headline number is 17,000 tokens per second per user on Llama 3.1 8B. That is a Taalas claim. The product page footnote says the test used 1,000 input and 1,000 output tokens. It lists an “Nvidia Baseline (H200)” and a “B200 measured by Taalas,” says Groq, SambaNova and Cerebras figures come from Artificial Analysis, and says Taalas’s own number was “run by Taalas labs.”

So it is a company benchmark. heise noted at launch that independent third-party measurements did not yet exist. We have not found any since.

Our test is not a benchmark either. We sent a small number of requests to the public demo on Oct. 21. Every figure below is what ChatJimmy’s own response metadata reported back.

  • A 225-token answer (a Python function) decoded at about 15,700 tokens per second. The service reported 14 milliseconds of device compute time.
  • A 105-token answer decoded at about 15,690 tokens per second.
  • Time to first token on short prompts was about 1.2 milliseconds.
  • On a long prompt of about 6,000 tokens, the service reported reading the input at roughly 25,500 tokens per second, with about 236 milliseconds to the first output token.

Those self-reported numbers sit in the same range as Taalas’s claim. The service generates them about itself. They say nothing about behavior under real concurrent load, cost per useful answer, or how the same prompts would run on a GPU cluster. Taalas has not published those figures.

heise also tried the chatbot at launch and reported reaching just under 16,000 tokens per second, close to what we saw. For comparison, heise cited about 230 tokens per second for an Nvidia H200 on the same model, from Nvidia’s own baseline data, and about 1,936 for Cerebras, from Artificial Analysis benchmarks.

The quality trade: 3-bit and 6-bit weights

The other half of the story is what the model loses on the way into silicon.

Taalas addresses this in its own post. Bajic writes that when the company started its first design, “low-precision parameter formats were not standardized,” so HC1 uses “a custom 3-bit base data type.” It continues: “The Silicon Llama is aggressively quantized, combining 3-bit and 6-bit parameters, which introduces some quality degradations relative to GPU benchmarks.”

Quantization means storing each weight in fewer bits. Going to 3 bits for much of the network saves area on the chip, which matters when the weights are the chip. It also adds rounding error that compounds through the layers.

Taalas has not published a benchmark table showing how much quality HC1 loses against the full-precision model. It is the biggest gap in its public material. If you are evaluating HC1, ask for standard evaluation scores for the HC1 model against stock Llama 3.1 8B at 16-bit, on the tasks you care about.

In our short session, simple tasks were fine. It got 17 × 23 right (391). It solved the bat-and-ball puzzle correctly ($0.05). It counted three r’s in “strawberry.” It wrote a correct palindrome function.

Two results showed the limits. Neither is unique to Taalas. Asked who won the 2024 U.S. presidential election, it said the election “has not yet occurred.” That is Llama 3.1’s training cutoff, not a hardware fault. It is also what users of this chip get until the model changes. When we pasted text repeating one word 500 times, then 600 times, and asked it to count that word, it answered 1,442 and 100. Both were wrong. Small models are poor at counting across long inputs at any precision, so we can’t pin that on the quantization. The answers still came back fast, and they were still wrong.

Taalas says its second-generation silicon “adopts standard 4-bit floating-point formats, addressing these limitations.” We have not seen HC2 tested anywhere.

The input limit

The developer of an unofficial wrapper around ChatJimmy reports that it accepts about 6,064 input tokens, and that longer requests return an empty response with no error. The chatjimmy-api project on GitHub documents it. That wrapper is a community project, not Taalas documentation.

We checked. A prompt of 6,025 input tokens worked. A prompt of roughly 6,225 tokens came back as an empty body with no error message and no metadata. The reported ceiling matches what we saw, within a couple of hundred tokens.

For operators, the silent failure is the bigger problem. An empty 200 response with no error will pass most health checks. If this sits behind a retrieval system that sometimes stuffs in long documents, count tokens yourself before the call and alarm on empty completions.

This is the public demo. Taalas says the context window is configurable, and its API may behave differently. Its API documentation does not state a context limit.

One model per chip: what it means for updates

On a GPU, swapping Llama 3.1 for a newer model is a download and a restart. On HC1, the model is the chip. A new model means new silicon.

Taalas’s answer is fast fabrication. Bajic’s post says the company has “a platform for transforming any AI model into custom silicon,” and that “from the moment a previously unseen model is received, it can be realized in hardware in only two months.”

Two months from model to a design ready for hardware is Taalas’s figure. We have no independent check on it. It also leaves out the time to get wafers out of a foundry, package them, test them, build servers and deploy them in a data center. Taalas’s February roadmap said a second model, “a mid-sized reasoning LLM” on HC1, was “expected in our labs this spring” and would reach its inference service “shortly thereafter.” After that, “a frontier LLM” would be built on second-generation HC2 silicon. “Deployment is planned for winter.” As of Oct. 21, the public demo still serves only llama3.1-8B. We have not seen Taalas or AMD announce the reasoning model as live.

That sets the economics. A hardwired chip pays off when a customer will run one exact model, at high volume, long enough to recover the cost of masks, wafers and servers. Some workloads fit: a fixed classifier, a voice agent, a high-volume extraction pipeline where the model is good enough and will stay frozen for a year. Most frontier chat products ship model updates every few months. They don’t fit.

Where it fits in the inference stack

In cloud terms, HC1 is a fixed-function appliance for one model. It does not replace a GPU.

Taalas offers it two ways. The free demo at chatjimmy.ai needs no login. The Taalas API is OpenAI-compatible, with /v1/chat/completions and /v1/completions endpoints. It requires a Bearer token and applies per-minute rate limits that return HTTP 429 when exceeded. Its example model name is three_bit_numerics, which is a fairly honest label. Bajic’s post says developers have to apply for access.

Taalas’s API documentation, as we read it, describes plain text completions and does not document tool calling. Agent builders who depend on tool use should confirm support with Taalas before planning around it, whatever the speed.

The likely fit is a fast tier in a routing setup. Short, latency-critical calls go to the hardwired model. Anything long, complex or tool-heavy goes to a GPU-served model. That is an architecture decision, and it means running two stacks.

On security: the public ChatJimmy endpoint has no authentication. Fine for a demo, bad for anything real. Taalas’s privacy policy says it and its third-party providers “do not use your prompts and outputs for any purpose, including to create, train, fine tune, or otherwise improve an AI model.” Its terms of service still give Taalas a licence to “collect, use, process, store” user content to provide the service, reserve broad rights over usage data, and tell users: “Do not share information that you don’t want the Services to use, such as any sensitive personal information.” Neither document we read gives a specific retention period for prompts. Treat the demo as a public test bench. Put no customer data in it, and build no production systems on community wrappers around it. Taalas’s API, with keys and rate limits, is the sanctioned route.

Why AMD agreed to buy it

AMD’s Aug. 6 release says Taalas’s technology “optimizes inference dataflows, significantly reducing compute and memory bottlenecks associated with general-purpose architectures.” AMD said it plans “to integrate the technology into its accelerator roadmap and develop system-level solutions with AMD Instinct GPUs.”

Vamsi Boppana, senior vice president of AMD’s Artificial Intelligence Group, said in the release that Taalas’s technology and team “strengthen our AI portfolio by delivering differentiated inference performance and efficiency.” Bajic said joining AMD “will give us the scale, engineering resources and global reach to accelerate our innovation.”

CNBC reported that an AMD representative declined to give a price, and that Taalas had raised $219 million since its 2023 founding. Bajic’s February post said the first product was built by “a team of 24 team members, and a total of just $30M spent, of more than $200M raised.”

AMD has not published a product name, schedule or model list for Taalas-derived hardware. Until it does, the deal reads as AMD buying a team and a method. No product line is shipping under AMD’s name.

The verdict

ChatJimmy is a working demonstration. HC1 serves a real model far faster than the GPU figures Taalas and heise cite for the same model. Taalas’s measurements say so, heise’s hands-on test agrees, and so does the service’s own metadata. Putting the weights in the logic and deleting the memory wall is serious engineering, and Taalas states the costs openly.

It is also an 8B model from 2024, quantized to 3 and 6 bits, with an input cap near 6,000 tokens that fails silently, no documented tool calling and no independent benchmark. You can’t update the model without new silicon. Taalas’s promised second model has not appeared on the public demo.

For an AI/ML team, the open question is quality. Ask for evaluation scores against the full-precision model. For a cloud team, the open question is lifecycle. Ask how long the model stays frozen, what a model change costs, and what happens to the hardware when it does. For both, ask what AMD’s ownership would change about support for HC1, and when the deal is expected to close.

The speed is proven on the demo. Whether a frozen model is good enough for your workload is something only your own tests will show.

How we tested: Cole Harlan sent a small number of requests to the public chatjimmy.ai demo on Oct. 21, 2026, between about 5:20 and 5:40 PM ET. All speed and timing figures cited from our session are the service’s own self-reported metadata, not independent measurements. Cowry Tech News has no commercial relationship with Taalas or AMD.

Sources