Matrix workshop: reading a button bar and a game board as a matrix

A matrix is a grid of cells the bot reads in one go: every cell gets a state such as "yellow", "orange", "empty" or "x100". That lets you evaluate button bars, inventory slots, skill bars and game boards without building a node for every single field. Since version 1.4.0 you set everything up in the Matrix workshop and see immediately what the bot will read.

Download example bot: buy amount Download example bot: Connect Four

The idea

1. Read matrix lays a grid over an area and recognises every cell.
2. The data nodes Find cell, For each cell, Find line, Count matrix and Matrix cell evaluate the matrix — each delivers the screen centre of the cell (x, y), which you connect straight to Click.
3. Wait for cell and Matrix change react when something happens.

Part A — button bar: which buy amount is selected?

Clicker Heroes has the buttons x1, x10, x25, x100, MAX. The selected one is orange, the others yellow, unaffordable ones grey.

Step 1 — add Read matrix

1. Capture window as usual, then attach Read matrix from Perception.
2. The properties panel shows the Matrix block with a summary — click Open matrix workshop….

Step 2 — draw the grid

1. The workshop shows the window. Drag a rectangle exactly around the five buttons.
2. Set Rows to 1 and Columns to 5. If the buttons differ in width, move the cyan dividing lines.
3. Every cell now shows a colour dot (the sample point) — still without a name.

Step 3 — move the sample point next to the label

At first the sample point sits in the cell centre — on buttons that is exactly the label, which is why the colours scatter.

• Drag the ring (with no cell selected) to the left, e.g. to 15 % — that moves the sample point for all cells next to the label.
• If one cell needs a different point: click the cell, then drag its ring. That deviation applies only to this cell and is listed under "Sample point".

Step 4 — name the colours

1. In Recognition keep Colour (palette).
2. Click ＋ Colour from cell, then a yellow button → "Colour 1" with that colour appears. Rename it to yellow.
3. Likewise: orange button → orange, grey button → grey.
4. Watch the badges on the cells: they now read yellow, orange, grey. If a red ? remains somewhere, raise the tolerance a little or move the sample point.

Step 5 — apply and use

Click Apply. The panel now says "1 × 5 cells · colours: 3 (yellow, orange, grey)". During a run the log shows ↳ Row 0: yellow | yellow | orange | yellow | grey.

• Find cell with label orange → output found, zeile/spalte tell you which amount is selected.
• To click MAX: Find cell with label MAX works in text mode (see part C) — in colour mode use Matrix cell with column 4 and connect x/y to a Click node.
• Count matrix with value orange → anzahl = 1 means "one amount is selected".

Alternative: image mode (template per state)

If colours are not enough (e.g. buttons with icons), set Recognition to Image (template per state): click ＋ Record from cell, then a cell — the crop is saved as a template and called "State 1". Repeat once per state and assign names. The bot compares every cell with all templates; below the similarity it shows ?. The templates live in your profile's template folder and are embedded when saving.

Part B — game board: Connect Four

A 6 × 7 board with red and yellow pieces on a dark background.

1. Read matrix → workshop: rectangle around the board, Rows 6, Columns 7.
2. Palette: Colour from cell on a red piece → red, on a yellow one → yellow, on an empty field → empty.
3. Find line after the matrix (data cable matrix → matrix):
   • Label red, Length 4, Directions all, Gap 0 → output found means "red has four in a row" (horizontal, vertical or diagonal). richtung says which.
   • Where do I win? The same node with Gap 1, Label of empty cells empty, Gravity on: it finds a line of three with one gap a piece can drop into — x/y point to exactly that cell → Click.
   • Where must I block? Same settings, but Label = the opponent's colour.
4. For each cell with filter empty delivers all free fields one after another — handy for "play anywhere free" (order random).

Part C — numbers and text per cell (OCR)

Set Recognition to Text / number (OCR): the bot reads the text of every cell (Sudoku, tables, quantities in an inventory, the labels of the buy-amount buttons). Tips: Magnification 3–4 for small digits, Digits only when only values matter, Invert for light text on a dark background. The result table shows the texts read; an empty cell reads as ·.

Part D — waiting and detecting changes

• Wait for cell (Perception): choose a matrix node, a cell (or "any") and a label and wait with a time limit until it appears — e.g. "wait until cell [0,4] is grey" (cooldown over).
• Matrix change (Conditions): compares with the previous pass and takes geaendert on differences — zeile/spalte name the first changed cell.

When cells show "?"