How to write your own detection AI model?

[sirius02]

Want to automate resource extraction in games? Aimbot on pixels? Make a recognition system? Or do you need to recognize objects on the screen for something else? In any case, in this guide I will try to explain how to do it in a short and simple way.​

​

First, let's go over the requirements:
- Python
- Account on roboflow
- Video card (I used a 3060 for training)
- Patience

First, you need to install python - You can do it from here
Follow the installing instructions and go to roboflow website, it totally free!
We need this site to make an annotation for our screenshots.
After logging in you need to create a new project

YOU MUST CHOOSE OBJECT DETECTION
Tip: if you do not want to public share your annotations just use random project name
After choosing the name we will have the menu like this


This is where we can upload our screenshots, videos, youtube videos, gifs, or anything that well represents the item we need to look for on the screen

After uploading you will have menu like this - I recommend to try Auto Label, but it mostly ruins the detection, so we press Manual Labeling and then Start Annotating

Here is the main menu, here is everything simple, just make a box on the item that you need to teach your AI model

In the end it must be like that, do not extend it too much, so recognition in games would work more correctly. To make it more precise just add more photos from any fov and angle that is possible.
Then add images to dataset and go to Generate tab

You can look all options, but its not necessary and press continue till it generate training dataset.

Press export Dataset and save it in YOLOv8 format. After downloading you can close the tab with roboflow.
Congratulations, you just created the dataset where your future model will be trained!

Training part:
Open cmd and write

pip install ultralytics

If everything correct than you can unzip the folder with our dataset in any place and create a new file with name "main.py" in the same file directory
After that just open that file with any code/txt editor and write the next code:

from ultralytics import YOLO

# Loading model
model = YOLO('yolov8n.pt')  # 'n' for small model, you can choose 's', 'm', 'l', 'x'

# Model training
results = model.train(data='path/to/your/data.yaml', epochs=400, imgsz=640)

The model is started training!
If you want to train it with GPU - you need to download driver CUDA for NVIDIA and idk what for AMD
After that you will have the results with file named best.pt, it should be tested on images before involve it in code, so let's make a quick test!

from ultralytics import YOLO

# Loading model
model = YOLO('path/to/your/best.pt')

# Path to your image for testing
results = model('path/to/test/image.jpg')

# Visualisation
results[0].plot()

If everything is correct - good job, now you can use it any game you would look like!
If something is bad at recognation - make sure you created good database and have enough epochs for training this model, maybe it have too many parameters and tags, you can look how good your dataset on roboflow Live check tab!

 

[sirius02]

Yes, I'm tired of writing this towards the end of the post, but I can answer your questions below the post if anything is unclear!