Cookie Clicker Automation with AskUI's Vision Agent

Cookie Clicker, as the name suggests, is a very click-heavy game that requires a lot of attention and can not be played on the side.

Why not try automating it with AskUI's Vision Agent to gain an edge while you go to the bathroom or get another cup of coffee?

1. Step: Install AskUI

Follow the installation guide in the README of our GitHub project.

After the installation, create a cookieclicker.py inside your project with the following skeletton code:

from askui import VisionAgent
import sys

with VisionAgent(display=3, log_level=logging.DEBUG) as agent:
    sys.exit(0)

2. Step: Make Sure to Have Focus

When you start the Vision Agent from your IDE or terminal the browser window on a second display where the cookie clicker resides in does NOT have focus and may fail to automate because of it. So we move the mouse cursor there and trigger a left-click:

from askui import VisionAgent
import sys

with VisionAgent(display=3, log_level=logging.DEBUG) as agent:
    agent.client.mouse(1000, 200)
    agent.client.click("left")
    sys.exit(0)

3. Step: Start with the Big Cookie

We are going for a very simple automation with a single prompt here, just to get some cookies when we are getting a new cup of coffee. Therefore we use the agent.act(<prompt>) method to specify a prompt with our automation needs:

agent.act(
         """
         The game cookie clicker is already in the current browser tab.
         Play Cookie Clicker like this:
         1. Left-Click on the really big cookie in the browser window 20 times.
         Repeat the step above 3 times.
         """)

This clicks the big cookie on the left 20 times and repeats the step 20 times. Run it by running in your terminal:

python cookieclicker.py

4. Step: Expand the Prompt

For the sake of demonstration - and more cookies - let us add two more steps to the prompt which increase our cookie output by buying grandmas or cursors if possible:

agent.act(
         """
         The game cookie clicker is already in the current browser tab.
         Play Cookie Clicker like this:
         1. Left-Click on the really big cookie in the browser window 20 times.
         2. Check if you can buy a Grandma. If you can buy one, buy it.
         3. Check if you can buy a Cursor. If you can buy one, buy it.
         Repeat the three steps above 3 times.
         """)

5. Step: Expand with your Ideas

This is just the beginning. You can add even more steps and gain more cookies without risking RSI. Start building your own Cookie Clicker Vision Agent with AskUI.