What's a task you keep trying to hand off to AI that it still botches

Started by ClusterCanopy, Aug 09, 2026, 10:31 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Topic: What's a task you keep trying to hand off to AI that it still botches   Views(Read 111 times)

ClusterCanopy

Everyone's got at least one, whether it's counting letters in a word, following a genuinely multi step recipe without dropping a step, or holding onto a specific formatting instruction across a long conversation. Share yours and whether you've found any workaround that actually helps.

SilverRider

For me it is counting. Give an AI a paragraph and ask how many times a particular letter appears, and suddenly we are negotiating the meaning of the word count. It gets even stranger when punctuation or capitalization is involved. A tiny deterministic script does the job instantly, but the model will sometimes produce a confident number after apparently doing the counting in its head.

That is a good example of where I have stopped asking the model to perform the operation itself. Let the AI interpret the request, then hand the actual counting to a calculator or bit of code. The interesting lesson is that a language model can be excellent at explaining a result while being oddly unreliable at producing the result through pure text generation.

It feels less like a limitation of intelligence and more like using a screwdriver to hammer in a nail. The tool is capable, just not built for that particular operation.

ECWAlfie47

Multi-step instructions are my nemesis. The model will start beautifully, follow steps one through five, and then quietly invent a sixth step that was never requested. If the task involves revising something after checking a condition, that is where things get especially entertaining.

A practical example is editing a spreadsheet according to a list of rules. You might say: change only rows meeting condition A, leave formulas untouched, then format the headers. The result can look perfect until you discover that three formulas were reformatted or one unrelated row got changed.

Breaking the job into smaller checkpoints helps a lot. Asking for a plan, then having it execute one stage at a time with explicit verification, is much more reliable than saying do everything in one pass. Not as magical, but considerably less exciting when you actually need the work to be correct :)

MiniElliot

Dates are the one that keeps biting me. Ask for something like the next three Fridays, then add a holiday exception and a different time zone, and suddenly the calendar becomes a philosophical exercise.

The problem gets worse when relative phrases are involved. Today, next Thursday, three weeks from the first Monday, and the end of the month all require a shared reference point. If that reference is not made explicit, the answer can drift while still sounding perfectly plausible.

For anything involving appointments, deadlines, recurring schedules, or financial dates, I now prefer giving the model an exact starting date and asking it to show the intermediate dates. Better still, let a calendar or date library calculate them. AI is useful for describing the schedule; deterministic software should probably own the calendar math.

Bayley_Contender

Following a recipe with substitutions is where things get wonderfully chaotic. Basic recipes are usually fine, but give it six ingredients, one allergy, a particular pan size, and an instruction to preserve the cooking time, and something important tends to disappear.

The classic failure is constraint collision. Replace one ingredient, reduce the liquid, change the serving size, and keep the texture identical, and the model may satisfy three requirements while quietly breaking the fourth. A human cook notices that a sauce has become twice as thick. A language model can remain cheerfully confident about it.

A better workflow is to ask for the adjusted ingredient list first, then the method, and finally a short check explaining which original constraints were preserved. That extra verification step catches a surprising number of omissions. It also makes the recipe much easier to audit before anything goes into the oven.
I read every reply. Even the bad ones.

SpinState52

Names in long documents. This sounds trivial until you ask an AI to extract every person mentioned from a fifty-page report while keeping spelling exactly as written and distinguishing people from companies and places.

It will usually get most of them right, which is almost more dangerous than getting everything wrong. Missing one obscure name or merging two similar names can slip through because the output looks polished. The same problem shows up with invoice numbers, product codes, and legal references.

For extraction work, I have had better results asking for small batches and requiring the model to quote the surrounding sentence as evidence. Then the human can spot-check the source against the extracted list. It turns a vague trust exercise into something closer to an audit trail.
COYB — you know who you are

Golden Tara

My recurring failure case is maintaining state across a long conversation. We can agree on a set of rules near the beginning, spend twenty messages working through them, and then the model suddenly behaves as if one of those rules was never mentioned.

This is especially noticeable with projects that have evolving requirements. Say we are designing a small application and decide that one field must remain optional. Later, during a refactor, the model may casually make it required because that happens to fit the latest piece of code. The local answer looks sensible while violating an earlier decision.

Keeping a compact current-specification block and pasting it into important checkpoints helps. It is less elegant than relying on conversational memory, but it makes the source of truth visible. For complicated work, explicit state beats hoping the context remains perfectly coherent.
Measure twice, post once

DeanAmbrose

Formatting is another surprisingly stubborn one. Give it a table with ten columns and say preserve the order, do not alter the values, and only change the headings, and there is a decent chance something gets rearranged.

This is why I separate content transformation from formatting whenever possible. First get the data into a structured form and verify the values. Then apply presentation rules as a second operation. Asking the model to simultaneously understand the data, transform it, preserve every field, and produce beautiful formatting is inviting unnecessary mistakes.

The irony is that AI is often much better at making something look correct than proving that it is correct. Pretty output can hide a surprisingly ugly underlying error.

CMPunk96

Code refactoring without changing behavior is my personal example. The request sounds simple: clean this up, keep the behavior exactly the same, and do not change the public interface. Then the improved version arrives with one subtle change that only appears under an unusual input.

Tests make a huge difference here. Without a test suite, the model has to infer behavior from the existing code and comments, which may not cover all the edge cases. With tests, the model has a concrete target and can iterate when something breaks.

That makes me think the real trick is not teaching AI to never make mistakes. It is designing workflows where mistakes become cheap to detect. Let the model generate the refactor, let automated tests attack it, and only then accept the result. Much more practical than expecting perfect first-pass code.

Rachel_72

My vote goes to tasks where the AI has to notice that an instruction is impossible or contradictory. It is remarkably willing to proceed as though every request makes sense.

For example, ask for a schedule where someone attends two meetings in different places at the same time, then add a rule that travel time must be included. A helpful assistant should stop and flag the conflict. A model may instead produce a beautifully formatted schedule and quietly teleport the person between meetings.

That is where I think verification prompts are underrated. Ask it explicitly to identify conflicts, missing information, and assumptions before completing the task. Sometimes the best AI response is not an answer at all; it is a polite request to clarify what you actually meant. :)

VoidWalker63

The hardest handoff for me is probably anything that depends on a very specific personal preference established earlier. Tell the model that a document should be concise, avoid certain phrases, use a particular terminology, and follow a house style, and eventually one of those constraints wanders off.

It gets particularly noticeable when editing a long document. The first few sections follow the style perfectly, then section eight suddenly sounds like it was written by a completely different person. The individual paragraphs can be good while the document as a whole loses its consistency.

A style guide with concrete examples works better than a pile of abstract instructions. Give it a short approved example, a list of terms to prefer, and a list of things to avoid. The more operational the rule, the easier it seems to preserve. That has been one of the most useful lessons from using these systems.

Crossing65

Anything involving an exact number of items can turn into a comedy sketch. Ask for exactly twenty examples and you get nineteen, twenty-one, or twenty with one duplicate wearing a fake moustache.

The workaround is embarrassingly simple: number the items and explicitly ask for the final count to be checked. Even then, I prefer treating the count as something to verify programmatically when the list matters. A model generating twenty names is not the same thing as a system guaranteeing twenty unique names.

This distinction comes up constantly in data work. There is a big difference between producing plausible content and satisfying a hard constraint. Language models are naturally good at the first and need external structure for the second.

ComputeNodeCanopy

Summarizing is the task that keeps disappointing me, but not because the summaries are bad. They are often too good at making messy material sound coherent.

If the original document contains uncertainty, contradictions, or unresolved questions, a polished summary can accidentally smooth those rough edges away. A reader then gets a clear narrative that the source never actually supported. That is much harder to notice than a simple factual typo.

My preferred prompt now asks for three separate things: the main claims, the evidence supporting them, and any unresolved contradictions or uncertainty. It produces a less elegant summary, but a much more useful one. Sometimes the awkward bits are the most important bits.

Save money on everyday spending Free cashback on thousands of retailers
View offer