Most automation gets by with image search and text recognition. Sometimes both fail — the picture looks different every time, or there is no text at all. That is where the AI modules come in: they judge an image instead of comparing it.
You list the possible states — fight, menu, loading screen — and the flow branches to whichever the AI sees. And if it cannot tell for sure, it takes the 'unclear' exit instead of guessing. The answer is always checked against your list, so it can never invent a state of its own.
Gold, level, enemy name: one call, one output per value. Useful wherever text recognition gives up — small type, a busy background, numbers drawn into an image.
This one runs outside the app: the control API lets an assistant like Claude Code or Codex read the module catalogue, write a bot, have it checked, start it, read the log — and even look at a screenshot to fix its own mistakes.
For recognising things on screen, a local model on your own machine is enough — we tested it that way. The app finds a running model server by itself and preselects a model that can read images.
| Local model | Provider (Claude, GPT …) | |
|---|---|---|
| Cost per question | none | by image area and text length |
| Speed | 0.2–0.6 s once the model is loaded; the first call takes about 15 s | usually 1–3 s, depending on the connection |
| Data | the image stays on your computer | the selected area is sent to the provider |
| Setup | install Ollama, pull one model, press „Find local models“ | enter the provider key |
One measurement worth knowing: a tight area beats the whole window. On the same screenshot and the same question, the shrunken full window was read wrongly, while a 750×385 cut-out of the right spot was read correctly — at 54 KB instead of 4.4 MB.
Not for every click. An image search takes milliseconds and always gives the same answer; the AI takes a few tenths of a second and can be wrong. So use it where understanding is needed, and leave the rest to template and text recognition.
And it does not remove the responsibility: a bot that clicks on its own can click wrongly. The safety net — panic key, dry run, failsafe — matters just as much with AI as without.