Aller au contenu principal

Detection Strategies

Once your macro goes beyond simple taps, the biggest factor in reliability is not the click itself, but how you detect the target. Advanced macros work best when they choose the right detection strategy for the UI they are automating.

Start With the Simplest Reliable Signal​

Before tuning settings, first choose the most stable signal:

  1. Use fixed coordinates only when the target position never changes
  2. Use image detection when the visual shape is stable and distinctive
  3. Use text detection when labels are easier to recognize than icons
  4. Use a smaller Region() when you already know roughly where the target should appear

If multiple strategies work, prefer the one with the smallest search area and the least ambiguity.

Screen vs Region​

The most common performance mistake is searching the full screen for everything.

Use Screen when:

  • the target can appear almost anywhere
  • you are building the first version of a macro
  • you are exploring a new app flow

Use Region when:

  • the target always appears in a known section of the UI
  • the screen contains repeated icons or repeated text
  • you want faster and more stable matching
local actionArea = Region(900, 1600, 1080, 2200)
actionArea:click("continue-button.jpg")

Fit Region Technique​

One of the most useful advanced techniques is what you described as fit region.

The idea is:

  1. Find the object once using Screen:find()
  2. Convert that match into a fitted Region using result:getRegion()
  3. Reuse that fitted region for the next searches if the object size stays stable

This works well when:

  • the object always has the same width and height
  • the object contents may change, but its container area stays consistent
  • the full-screen search is expensive and repeated often
local result = Screen:find("inventory-panel.jpg")
if result then
    local fitRegion = result:getRegion()
    local collectButton = fitRegion:find("collect-button.jpg")
    if collectButton then
        fitRegion:click("collect-button.jpg")
    end
end

Why this is powerful:

  • the first full-screen search is the expensive step
  • once the object is found, all later searches are limited to the fitted bounds
  • this can reduce OCR and image matching cost significantly in repeated flows

Use fit region when:

  • you have a stable panel, card, popup, slot, or container
  • the macro repeatedly interacts inside the same matched object
  • the same object appears in different positions but keeps the same dimensions

Avoid using fit region when:

  • the object is heavily resized between screens or devices
  • the target area expands or collapses dynamically
  • the first match is too generic and may bind to the wrong container

In practice, fit region is one of the best ways to convert a slow Screen macro into a fast Region macro without hardcoding coordinates.

Image Detection vs Text Detection​

Prefer image detection when:​

  • the target is mostly graphical
  • the UI language may change
  • the text is stylized or rendered inside a button
  • OCR struggles because the text is too small or low contrast

Prefer text detection when:​

  • the label changes position but keeps the same wording
  • the same visual component appears in many themes or states
  • the button shape changes but the label stays readable
  • you need to match text generated dynamically by the app

Word-by-Word vs Line-by-Line OCR​

When using text detection, the detection method matters:

  • Use setDetectionMethod(1) for a single word or very short label
  • Use setDetectionMethod(2) when the target is a phrase or full line
Screen:click("Login", ClickParams():setDetectionType("TEXT"):setDetectionMethod(1))
Screen:wait("Your session has expired", 5000, FinderParams():setDetectionType("TEXT"):setDetectionMethod(2))

Tune Match Score Gradually​

A match score that is too high causes misses. A score that is too low causes false positives.

A practical tuning process:

  1. Start with the default score
  2. If the target is missed, lower the score slightly
  3. If the wrong thing is matched, increase the score slightly
  4. If both happen, reduce the search area before changing the score further

In most cases, reducing ambiguity with Region() is safer than pushing match score too aggressively.

Use Grayscale Only When It Helps​

Grayscale can improve speed and sometimes improve consistency, but it is not automatically better.

Use grayscale when:

  • the target is defined more by shape than color
  • lighting or color shifts make matching unstable
  • you want faster matching on lower-end devices

Avoid relying on grayscale when:

  • color is the main difference between similar targets
  • you are trying to distinguish active and inactive states by color

Dynamic Scaling​

If the same macro runs across devices with different resolutions, dynamic scaling can help image detection remain usable.

Use dynamic scaling when:

  • the template was captured on one device but executed on another
  • the same app layout is resized across phones or tablets

But if the UI is already highly consistent on a single device, a well-cropped template plus a good region is usually more stable than over-tuning scaling.

Crop Templates Aggressively​

A good template should contain only what makes the target unique.

Good template:

  • the icon itself
  • the unique label area
  • the minimum stable visual pattern

Bad template:

  • large surrounding whitespace
  • backgrounds that animate
  • content that changes every run
  • nearby elements that are not part of the target

Use Two-Step Detection for Hard Screens​

When a screen is noisy, use a staged approach:

  1. Wait for a stable anchor element
  2. Narrow the search area
  3. Detect the final target inside that area
Screen:wait("settings-title.jpg", 5000)

local footerArea = Region(0, 1700, 1080, 2400)
footerArea:click("save-button.jpg")

For advanced macros, this is usually the safest order:

  1. Wait for a stable state
  2. Restrict the search area
  3. Choose image or text detection intentionally
  4. Tune score only after reducing ambiguity
  5. Verify the result before continuing