Your references show me that it is absolutely task depended. In many domains it's true that "criticizing is easier than creating".

The best example might be books and movies, where it's trivial to say the characters were shallow, but it's surprisingly hard to create deeply interesting characters.

In Software Engineering, there are similar dynamics. An LLM with a security vuln finding prompt will be able to point out places, where the generated code might be insecure.

But if you want another LLM to find a reasoning mistake in a mathematical proof it basically has to do all the reasoning work as well. In which case I doubt there will be any significant performance gains.

For better or worse this has become the defacto standard in LLM Evaluation research papers since the LLM as a Judge paper [0] came out. Its also heavily embedded into frameworks like LangChain and LlamaIndex to evaluate RAG pipelines.

[0] https://arxiv.org/abs/2306.05685

[1] https://arxiv.org/abs/2411.15594

> ...so you need a checker that actually reasons about the world (compiler, linter, SAT solver, ground-truth dataset, etc.).

Agree. What do you think about telling the LLM to also generate unit tests for the code it spits and then run all tests (including previous application unit tests).

I think this is a way to ensure some level of grounded verification:

- Does code compile?

- Do unit test pass?

AI can then consume test results to help fix their own mistakes.

actually, I found that you can definitely yield better results. I ran an experiment with 1 prompt at temperature 0 and 9 with temperature 1.

I found the most anomalous response was as good (15/20) or better (5/20) than the temperature 0 response in 20 samples.

I believe, what the smart AI company is trying to do, right now, in secret, is to use US, the humans, and our replies to the AIs, as training for the next generation of self-verifying-models. :)

Training on corpus data gets you to 1 order of magnitude. But training on interactive data where you can observe and adapt to the OODA-loop? So much more powerful.

At least, that's what I'd be doing if I were doing AI :)

But I just do BrowserBox

Would a LLM under human guidance turn out to be a good verifier ? i.e. if LLM knows the rules to verify or has enough data points (internet access, actual responses)

Ah, now we know why Spain was out of electricity yesterday.

For anything semi-adversarial, I have had good results asking the AI to come up with a plan, then take the side of the opponent coming with counter play/way to defeat the plan, finally asking for a revision of the initial plan given the potential reaction from the opponent.

The final plan you obtain is generally a lot more well rounded and thought out.

I find that amusing because the technique also works when I apply it to me. Picking flaws in your plan before revisiting it actually works.

I do the same, and I have one other technique.

I will often have a few chats going for a project, but with different contexts. For example, one might be tech focused, another marketing focused, another with some context on my personal goals, etc.

So I will take the same question and feed it into the chats with differing context. It is almost like having different perspectives on the same problem. And the conclusions can often differ based on the differing contexts.

This reminds me a lot of the YT video that went over using Monte Carlo Tree Search with LLMs to maximize result quality. Link: https://www.youtube.com/watch?v=mfAV_bigdRA&ab_channel=Treli...

It seemed like a pretty good idea, though I'd guess that it would greatly increase token usage. I'd also be concerned that the LLM as a judge might struggle to grade things accurately if it wasn't also able to generate good enough answers to begin with.

I do it all the time in Sillytavern in a group chat - three characters kind of resembling what you just described, and me, participating in the "conversation", them going back and forth until they're satisfied.

With a good model role playing them, works awesome.

Kagi’s Assistant feature makes this super easy. Just switch assistants and ask them to check the other’s work.

Isn't this kind of another way of how Inference Time Scaling works? It will basically produce several chain of thoughts and then pursue one that has maximum reward based on an internal function?

We're there any situation that first conclusion from AI was completely changed? Can you give generally examples of situations where it changed or significantly improved overall result? It sounds cool.

i dont understand, is it doing your schoolwork?

This takes such a long time to do though, no? What problems does this save you time on?

I've wondered if it might be helpful to randomly "shard" training data between two LLMs; just feed half the training data to one, and the rest to the other, with no overlap.

So instead of using two models, you'd be making two halves of one model do a similar (deliberative) process to yours. I wonder if that would result in a benefit over a single model with the full training set, and if you could continue to do the same thing by sharding the shards.

In doing some DevOps-y type tasks recently (ansible, packer, docker, baking images with guestfish), I've found it very frustrating how much ChatGPT will confidently tell me to use flags on tools that don't exist, or hallicinate completely non-existent functions or behaviours. And then when I spend time trying what it suggests only to hit a wall and come back like wtf mate it breezily goes "oh yes so you're right, good job figuring that out! You're so close now! Your next step is to do X and Y," and then serves up the same detailed tutorial as before but with the flag or whatever it was that it had wrong subtly changed.

It definitely makes me feel like I'm dealing with an overenthusiastic intern who is throwing stuff over the wall without checking their work, and like maybe having a second bot sitting in front of the first one being like ARE YOUR SURE ABOUT THAT could really improve things.

A year or so ago I experimented with splitting a user prompt down to a set of "different AI personas" that would each try to approach the user's problem in a different way and then bubble back up with a master arbiter for consensus.

I modeled it after the concept of advisors from Civilization II. It worked reasonably well though I think it was at least somewhat limited by being constrained to a single LLM (Mistral). It also lit my computer on fire.

In theory couldnt this just be baked into a single adversarial model?

These ensembles have been tested throughout AI progress. Well scaffolded larger models have historically come out ahead in both quality and speed/cost.

Perhaps this is a parricularly effective ensemble, but I would need to see real data.

Like, just endlessly grinding tokens, then processing the output and pulling out good ideas when the endless debate generates them?

Would be interesting what it comes up with with enough time and tokens.

This is being done, and you could apply it to a lot of domains. Go for it for whatever use case you have.

Yeah, but we'll finally get definitive proof that the government's been hiding super-intelligent axolotls from us all.

A society of mind, if you will. :)

This sounds like a fun thing to set up with a quick-enough local model.

This is one of the reasons I like the massive context window in Gemini. You can do this as part of the message chain. I don't try to one shot it, just use the same idea across 3 messages.

1. Figure out a plan (it responds with the plan)

2. Point out flaws in the plan (it responds with the flaws)

3. Update the plan to address the flaws (it responds with an up to date plan)

The other things I tend to ask are "what might we be missing?", "what are the [performance|security|legal|cost] considerations?". I can often iterate on the "anything else?" kind of nudging prompts, especially guiding it on topics to consider, for a few messages. After each: update the plan to take those into consideration.

I always do "now again but put on your critical hat"

Oh I really like that. It makes me want to have it score its ideas with metrics and then keep iterating until it meets some score.

Also a little clickbaity with "my AI" and then it's all Mistral...

Check out Fast Agent! (I have no affiliation with it, just use it).

https://github.com/evalstate/fast-agent

Techniques like this have been around since GPT-3.5. There are boatloads of papers on the topic.

I have no idea why anyone thinks this is novel. I guess that speaks to the state of HN

Chatgpt shares context between chats. I wonder how that impacts it?

It seems like a good approach though. What you dont want to do is ever suggest that its wrong yourself. Usually it will just assume it is wrong.

Actually what I find impressive is when I do this and it actually pushes back to defend itself.

I am thinking the same thing! Multiple "personalities", in parallel, or in series. For example, I have approximated, in GPT, some of Gemini's ability to call out nonsense, sloppy thinking, by telling GPT to be mean! (The politeness seems to filter out much that is of great value!)

However, the result is not pleasant to read. Gemini solved this in their training, by doing it in two phases... and making the first phase private! ("Thinking.")

So I thought, what I need is a two-phase approach, where that "mean" output gets humanized a little bit. (It gets harsh to work in that way for more than short intervals.)

As a side note, I think there would be great value in a UI that allows a "group chat" of different LLM personalities. I don't know if such a thing exists, but I haven't seen it yet, although the message object format seems to have been designed with it in mind (e.g. every message has a name, to allow for multiple users and multiple AIs).

Even better if it supports multiple providers, since they have different strengths. (It's like getting a second opinion.)

I think you can do most of this already with llm-consortium (maybe needs the llm-openrouter plugin with my pr merging)

A consortium sends the same prompt to multiple models in parallel and the responses are all sent to one arbiter model which judges the model responses. The arbiter decides if more iterations are required. It can also be forced to iterate more until confidence-threshold or min-iterations.

Now, using the pr i made to llm-openrouter, you can save an alias to a model that includes lots of model options. For examples, you can do llm openrouter save -m qwen3 -o online -o temperature 0, system "research prompt" --name qwen-researcher

And now, you can build a consortium where one member is an online research specialist. You could make another uses JSON mode for entity extraction, and a third which writes a blind draft. The arbiter would then make use of all that and synthesize a good answer.

Have you tried n8n? It allows you to build flows like that - you can run the community version in a Docker container within a few minutes and share the configurations for the flows you have built very easily.

Ive felt this way when using chatgpt for a simple search. Stuff that google could handle but would just be slower, mostly from me having to manually filter.

Sometimes its the easiest way to complete a very small task but the cost difference on the backend has to be pretty damn large. The user inevitably ends up not caring whatsoever. Its just not real to them.

I caught infra people saying that's pretty much the only bottleneck in the data center right now, power and cooling. We know the AI needs to run against itself continuously, and that's just a fact.

I once attended a talk a year ago where a techlead did just that - they had AI agents that ran a scrum team with different roles, each agent's prompt was to disagree with everyone else (or be highly critical) and present their own point of view, and then an arbiter would make the final decision. They claimed it worked for them.

Maybe. Humans form teams for a reason. Yes there are different exepriences and points of view in a human (vs. Not so much in LLM), but sometimes a different hat it all it takes. E.g. Code reviewer vs. Coder.

Seems likely to me. As long as adding more appears to help people will do it

Presumably there is some point where it levels out. And no doubt there will be a committee of AIs to determine said point.

Cause we wouldn’t want to boil the ocean…

Scaled up and spread out - this probably gets you pretty close to consciousness(?)

Conway's game of life, but instead of colored squares with rules, they're LLM's with some kind of weighting - all chattering back and forth with one another - bubbling up somehow to cause speach/action

What you just said is what I tried and failed to say ten minutes ago!

https://news.ycombinator.com/item?id=43835798

I think the names were switched up.

Upfront: I think AI is borderline useless for many of the tasks we give it. But:

1. Do our neurons not just react the same way every time to the same input? A brain is larger than the sum of its parts.

2. They don't reason, but you can somewhat emulate (or pretend to be) reasoning if you feed something back into itself enough times and it pinky promises reasoning is happening

With my own experiments I've also found this. This behavior is very persistent with llms on default hyperparameters and system prompt. Right now I am exploring how to get these models to output more human like interactions and it seems that a very specific and detailed system prompt is very important to get this to work. These systems are VERY sensitive to system prompt and user input. Meaning that the quality of output varies drastically depending on not just the language you use but how its structured, the order of that structure and also other many nuanced things like system prompt plus user input pre conditioning. So far it seems its possible to get to where we need to for this task but lots of exploration needs to be done in finding the way in how to structure the whole system together. This revelation is kind of nuts when you think about it. It basically means, once you find the right words and the order in which they should be structured for the whole system you can get 2x+ improvement in every variable you care about. That's why I am spending some time creating an automated solution to find these things for x model. Its a tedious effort to do manually, but we have the tools to automate its own optimization and calibration efforts.

This might be a situation that warrants a higher temperature. Actually, it could be worth starting a very high temperature initially and gradually decreasing it.

I always assumed you'd have to use different models. Even if only one of them is large, the others would inject enough difference of opinion to keep it useful.

That's very easy to do. A prompt I regularly use is a "council" system. For example:

"I believe I have been contacted by the supernatural. Here are the details <details>. Please form a council of seven people: 1) Secular scientist 2) Religious scientist 3) Paranormal historian 4) Secular Psychologist 5) Religious psychologist 6) Carl Jung 7) Richard Dawkins. The council should all be independent and provide their own objective analysis. Please have them create a final report and conclusions at the end".

Your council can be anything, a law firm, a jury, a parent teacher association, whatever you want, and as you can see, you can throw in known people as well. This can all be done with one prompt. It's one my favorite things to do.

How did it do?

> How do you solve the risk of AI writting and executing unit tests with something like `rm -rf /` and wiping your files?

The same way you stop any person or program or third party from doing something dumb or nefarious with your files. Don't give them any access to important files.

Either you trust AI or you don't? If you don't trust it then you need to review what it's writing.

Docker seems like a pretty low complexity way to create an isolated environment to run automation.

Manually approve every terminal command it wants to run instead of vibe mode. Tbh I think an rm -rf scenario is exceedingly unlikely.

No way around it, got to sandbox the whole thing no matter what.

a) You should only do this in a sandbox

b) You can have the AI run a "firewall" prompt on the final output. So your final output should go through a "You are a firewall that checks for dangerous terminal commands such as <enumerate list of dangerous commands>. If you spot dangerous commands, reform the command so that it is not dangerous"

this works for getting out of traffic tickets too lol

The thing that makes us weird to regular people is what's going to make us uniquely positioned to utilize AI. If people only knew the level at which I overanalyze and entertain weird ideas. I always inject these personality quirks into my instructions and get very creative results. In a weird way, I'm starting to appreciate just how weird I actually am.

My gut tells me yes. From my own experiments the order and way in which these things are done are important. I think it all is very strongly tied to the attention mechanism.

I would really like to see a fusion guidebook of mental tricks that work for humans and just as well for AI. Or humorously, perhaps prompt-engineering tricks that are also great mental hacks for better or clearer human thinking.

"While hallucinating a duck, check my script for errors."

That seems like a terrible idea. At best it seems likely to help you make a false but convincing sounding case. I really hope no one is using that to help them understand controversial topics much less using that to determine their stances.

Id recommend looking into actual human experts who are trustworthy and reading them. Trying to get LLM to argue the case will just get you a lot of false information presented in a more convincing fashion

Don't people get tired of having this same "debate" on every post about LLMs? And I scare quote debate because the naysayers never support their strong claims beyond the most superficial of responses. It's all just so tiring at this point.

Can you define "thinking" in a way that excludes what the AI is doing, but includes what humans do?

I haven't' really seen anyone else manage it without talking about ghosts or some other kind of metaphysical voodoo.

Why is it so hard to believe that a complex neural network can think? You literally have one over your shoulders that does exactly that.

I am not sure that you can make that absolute statement. Reasoning is subdivided into types, and one of those types is inductive reasoning.

> Inductive reasoning refers to a variety of methods of reasoning in which the conclusion of an argument is supported not with deductive certainty, but with some degree of probability. Unlike deductive reasoning (such as mathematical induction), where the conclusion is certain, given the premises are correct, inductive reasoning produces conclusions that are at best probable, given the evidence provided.

Doesn't predicting the next token qualify as doing just that?

https://en.wikipedia.org/wiki/Inductive_reasoning

Sorry to join the pile-on, but can I just ask: In what way does a brain think that an Ai does not? And does the distinction apply from human brains down to fruit flies? Is it a property of embodiment? (I have suspected for years that consciousness isn't just emergent but specifically that it is NOTHING besides that. It's all about scale and large models are just starting climb the ladder. The ladder does not necessarily go up the same way as embodied thought though.)

How can you be so sure? How do you know that our brains don't work like transformers too, except for having the advantage of having more types of sensory data? How can you settle this debate without defining what "thinking" and "reasoning" is and how what LLMs do is not similar to what a kindergarten level kid may be capable of? I think we all agree kindergarten kids can think and reason, don't we?

Agree 100%.

We should also not be calling a pointing device "a mouse" because it's not a small rodent, there aren't any actual windows inside a computer, and I haven't seen anyone balancing their laptop trying to surf the web.

Also smartphones are not actually smart and are only barely phones.

Or "thinking" just got a new meaning and it's to convey information in the field - perhaps the Oxford dictionary will add it soon?

What words would you use instead?

You say these things with such certainty. How can you be so sure?

You are the critic. Construct three rebuttals to your claim.

That's exactly what us humans do when we think about stuff. We combine memories and knowledge in unique ways, then we usually go ask someone else to give input on it.