We talked about how to use Python Realize page automation in the browser , No matter which way you implement , The corresponding operations are performed by locating the elements in the page .
Today, let's talk about how to realize automatic operation on the desktop . Similar to browser page Automation , Desktop automation also requires positioning the mouse on the desktop , Then perform corresponding operations according to the positioning position .
Our hero today is pyautogui,pyautogui It's pure. Python Of GUI Automation tools , Through it, the program can automatically control a series of mouse and keyboard operations to achieve the purpose of automated testing .
The installation of this module is the same ：
pip3 install pyautogui
After installation, it can be used directly .
The most basic desktop operation is mouse operation , We can control the movement of the mouse ：
# Move the mouse
pyautogui.moveTo(200,400,duration=2)
pyautogui.moveRel(200,500,duration=2)
The whole desktop is the origin with the upper left corner as the coordinate axis , All operations are based on this origin , To determine the operating position .
The first line is to move the mouse to the specified pixel (200,400) Location , The second line of code is to move the mouse to the right according to the current point 200px, Move down the 400px Move in this direction .
Both lines of code have a common parameter duration, This parameter represents the movement time , That is, the mobile operation is completed within the specified time , The unit is seconds .
Run these two lines of code , Observe the changes of the screen mouse , Isn't that amazing ？
We can also get the mouse position ：
print(pyautogui.position())
It's easy to understand , Is to get the coordinate position of the mouse in the current screen , Run this line of code , We will get information such as the following ：
Point(x=400, y=900)
Usually , Our mouse has left and right buttons , There is a button in the middle of the advanced mouse .
My mouse has only two buttons , There is no key in the middle , alas ~
pyautogui There are corresponding processes for these three key operations ：
# Mouse click , Default left button
pyautogui.click(100,100)
# Left click
pyautogui.click(100,100,button='left')
# Right click
pyautogui.click(100,300,button='right')
# Click in the middle
pyautogui.click(100,300,button='middle')
Mouse click , If you don't specify button Parameters , The default is to click the left button , The first two parameters are the position of the click coordinate .
Run this code , See what happens to your desktop ？
In addition to the mouse click operation , There are also double-click operations ：
# Double left click
pyautogui.doubleClick(10,10)
# Right click
pyautogui.rightClick(10,10)
# Double click the middle button
pyautogui.middleClick(10,10)
The operation function is also very simple , I believe you can see it at a glance , If you can't see at a glance , Please look more ！
Friends who are familiar with the front end may immediately associate with , Mouse operation has the process of pressing and releasing , Our screen operation also has corresponding control ：
# The mouse click
pyautogui.mouseDown()
# Mouse release
pyautogui.mouseUp()
We can control the mouse to drag to the specified coordinate position , And set the operation time ：
pyautogui.dragTo(100,300,duration=1)
This operation is similar to the previous movement .
According to the experience of the previous movement , We also drag the mouse in the direction ：
pyautogui.dragRel(100,300,duration=4)
In desktop operation , Sometimes we need to scroll the mouse up or down , At this time, we can use scroll This function controls ：
pyautogui.scroll(30000)
The parameter is an integer , Indicates how many units to scroll up or down , This unit may vary depending on the operating system . If you scroll up , Pass in a positive integer , Scroll down to pass in negative integers .
Let's start with a scenario ： Now I want to find a red dot on the screen , What would you do ？ The usual way is to get the color value of the red dot , Then compare the points on the screen one by one , Until we find it .
pyautogui It provides support for our operation scenario , There are three functions that do these three things .
im = pyautogui.screenshot()
im.save('screenshot.png')
rgb = im.getpixel((100, 500))
print(rgb)
match = pyautogui.pixelMatchesColor(500,500,(12,120,400))
print(match)
The first one is the function to get screen shots , It can return a Pillow Of image object ; The second is to get the color of the coordinate points specified in the screenshot , return rgb Color value ; The third is to compare the color of the specified coordinate point with the color of the target , Returns a Boolean value .
Let's upgrade the requirements ：
I'm going to find it on the screen now edge Browser Icon , What would you do ？
The usual practice is to know first edge What does the browser icon look like , Is it green or blue , Is it fat or thin , Right ？ Then match the icons on the screen , Until we find an icon that is the same as our target icon , And you get the result .
therefore , Our code is as follows ：
# Image recognition （ One ）
oneLocation = pyautogui.locateOnScreen('1.png')
print(oneLocation)
# Image recognition （ Multiple ）
allLocation = pyautogui.locateAllOnScreen('1.png')
print(list(allLocation))
You can capture the icon of an application on the desktop , Save as picture , Then use the above lines of code to identify , Recognition success , You will return results similar to the following ：
Box(left=20, top=89, width=33, height=34)
[Box(left=20, top=89, width=33, height=34)]
This is where the picture is on the desktop , If you can't find the picture , It will return None.
Keyboard input has the following common functions ：
for instance , You usually input an exclamation point （！） How to operate the keyboard ？
Hold down shift Key , And then press and hold 1 Key , That's all right. . use pyautogui Control is ：
pyautogui.keyDown('shift')
pyautogui.press('1')
pyautogui.keyUp('shift')
Run the above code , If your mouse is in the edit box , You will get an exclamation point ！
We can also output content directly ：
pyautogui.typewrite('python', 1)
The first parameter is the output , The second parameter is the interval , The unit is seconds .
Run the above code , In your editor, there will be every 1 Seconds in sequence python Of 6 Letters .
Sometimes we need to enter some special symbol keys on the keyboard , such as Line break 、 Direction keys, etc , These have corresponding keyboard strings to represent ：
pyautogui.typewrite(['p','y','t','h','o','n','enter'])
Run the above code , The editor will output python After that, change the line .
For the strings corresponding to other special keys, please refer to the official instructions .
If I want to copy a content , In most cases, the fast key will be used ctrl + c, According to the above , This is how we should achieve ：
pyautogui.keyDown('ctrl')
pyautogui.keyDown('c')
pyautogui.keyUp('c')
pyautogui.keyUp('ctrl')
It is troublesome to write like this , And you need to control the sequence of pressing and releasing keys .
pyautogui It provides us with a quick function ：
pyautogui.hotkey('ctrl','c')
The effect achieved is the same as the above 4 The line code is the same .
When you are simulating a desktop operation , If there is a branch operation, it needs to be judged according to the actual situation , Do you need a place where you can choose which branch to take ？
pyautogui I have considered this situation with great consideration , You can interrupt the current operation by popping up a selection box , Select the action Branch .
way = pyautogui.confirm(' Leader , Which way should I go ？', buttons=[' Rural road ', ' waterway ', ' land route '])
print(way)
This is us HTML Page confirm Selection box , After selecting the option , We can get the selected options , Then make a judgment based on this option , Enter the corresponding operation branch .
In addition to selecting the confirmation box , There are other message boxes ：
# Warning box
alert = pyautogui.alert(text=' Warning ！ Enemy Strike ！', title=' Warning box ')
print(alert)
# Password box
password = pyautogui.password(' Please input a password ')
print(password)
# Normal input box
input = pyautogui.prompt(' Please enter the command ：')
print(input)
pyautogui The basic knowledge of is introduced here , This python The function of the module is very powerful , The functions are very simple , Yes python Beginners are friendly . After learning these basic knowledge , You can use a combination of these basics , To implement some interesting desktop Automation , Go and try it ！
【python Study 】
learn Python The partners , Welcome to join the new exchange 【 Junyang 】：1020465983
Discuss programming knowledge together , Become a great God , There are also software installation packages in the group , Practical cases 、 Learning materials
