程序師世界是廣大編程愛好者互助、分享、學習的平台,程序師世界有你更精彩!
首頁
編程語言
C語言|JAVA編程
Python編程
網頁編程
ASP編程|PHP編程
JSP編程
數據庫知識
MYSQL數據庫|SqlServer數據庫
Oracle數據庫|DB2數據庫
您现在的位置: 程式師世界 >> 編程語言 >  >> 更多編程語言 >> Python

[Do it 1] Python monitors database changes

編輯:Python

前言

用的yolov5,作者自己寫的loadStreamfunction is based on streams.txt裡面的rtspStream address list to create a new thread,Then realize multi-channel monitoring.


That's basically what the picture says,I'm already for a whole business,Changed this a littleloadStreams方法,those availablertspStream address is saved to a newlist裡面,Then create a new thread.

How to check availability,就是用opencv VideoCapture.isOpened() try it slowly,If it can't connect, it will be removed

上面這個流程,是蠢(Thinking of timed tasks,Restart the server every now and then,In fact, subconsciously,want to use the most direct(不動腦子)way to fool this problem).talk to the teacher,The teacher also said,Change it to a trigger.It's been ten days,today while there is nothing else to do,Brain can turn,來學一下.

技能要求

監聽數據庫,然後 Read and write the latest database information to a file,Then restart the server with the file.

准備工作

  1. pycharm新建一個python文件
  2. nacicat 打開自己的數據庫
  3. 找一篇 pythonA blog that monitors database table changes

When looking for a blog,(I actually found some),But mostly polling the database,That is, it is almost the same as my scheduled task above..But what I want is a trigger,only when the database changes,才會執行,Is that what triggers it?,一直盯著,That's called monitoring.

Then saw this

確實,The idea is to monitor the database from 轉變為 用mysqlThe trigger then generates the log file(只生成rtspThe change log of the address column),然後再用python的watchdog(this is the wheelPython Monitoring file increase、修改、刪除等變化)monitor this log file. This way you don't have to monitor the database,becomes a monitor file.(Mainly because the database of colleagues is very slow to connect,So if you can't monitor, you can't monitor.)
另外一個想法,I wonder if I can passmysql的觸發器,直接調用pythonservice restart script.這就牽扯到,Should this script also be placed on the machine where the colleague's database is located?.If it's my idea,Then the log file I need to monitor above,It must also be generated on the local address of the database.,But my server is deployed inautodl的docker裡面的,How do these two communicate with each other??
好麻煩,要不就 poll the database.

Later, if this thing is really going to be online,The database must be on the server..mysql部署在dockerThat means the log file is in the same place as my project's code.
If it is a place,Then you must visit yourself very quickly.,polling is ok;or then set the trigger,It is also possible to monitor files generated by triggers;or the trigger fires and then callspythonIt is also possible to restart the script.

I don't know what will happen in the future,The feasible way is to look at.

輪詢

寫腳本 輪詢 寫的挺好,Two types of monitoring are listed——Added and changed
In my business, it is mainly to process the newly added data.,how to add them tostreams.txt,So I can write in the callback method of polling“將數據寫入streams.txt的文件操作”.
After I typeset his code,就是如下:

import os
import sys
import time
import threading
import pickle
import pymysql
class BaseListener(object):
# Use a thread to start listening
def __init__(self):
self.checkpoint = 0
self.stop_flag = True
self.listen_thread = threading.Thread(name="Listener", target=self.do_listen)
self.listen_thread.start()
def start(self):
self.stop_flag = False
def stop(self):
self.stop_flag = True
def set_checkpoint(self, v):
# Set breakpoints for monitoring,Persistent on disk if needed
self.checkpoint = v
def get_checkpoint(self):
return self.checkpoint
def do_listen(self):
filename = 'utils/streams.txt'
with open(filename, 'w') as f:
f.write('')
cnt=0
while True:
if not self.stop_flag:
conn = pymysql.connect(host="localhost", user="root", password="123456", charset='utf8')
# 監聽用sql語句,應當以id倒排,需要使用 WHERE id > {CHECK_POINT}進行篩選,如
# sql = "SELECT * FROM a WHERE id>{CHECK_POINT} ORDER BY id DESC"
cursor = conn.cursor()
sql = "SELECT id,monitor_name FROM `fall`.`monitor` WHERE id>{CHECK_POINT}"
checkpoint = self.get_checkpoint()
sql_listen = sql.replace("{CHECK_POINT}", str(checkpoint))
# fetchallstatement to read all records
cursor.execute(sql_listen)
# 獲取所有記錄列表
results = cursor.fetchall()
#record the last cursor
rec_id =""
if(len(results)!=0):
cnt += len(results)
rec_id = self.callback(results,cnt)
self.set_checkpoint(rec_id)
# 關閉數據庫連接
cursor.close()
conn.close()
# Set the polling time according to the situation
time.sleep(1)
def callback(self,data):
filename = 'utils/streams.txt'
# 這是do_listena callback function called,Pass the data over for processing,在子類中實現
with open(filename, 'a+') as f:
f.write(data)
class BaseMonitor(object):
""" Base class for monitoring data changes """
def __init__(self):
self.prev_data = None
self.stop_flag = True
self.monitor_thread = threading.Thread(name="Monitor", target=self.do_monitor)
self.monitor_thread.start()
def start(self):
self.stop_flag = False
def stop(self):
self.stop_flag = True
def do_monitor(self):
while True:
if not self.stop_flag:
self.execute(self.extra_sql)
data = self.fetchall(self.base_sql)
if data:
str_data = pickle.dumps(data)
if str_data != self.prev_data:
self.callback(data)
self.prev_data = str_data
def callback(self, dictdata):
# 這是do_monitora callback function called,Pass the data over for processing,在子類中實現
print
"Should be implemented in subclasses!"
class Listener(BaseListener):
def callback(self, data,cnt):
filename = 'utils/streams.txt'
if(cnt!=len(data)):
p = sys.executable
os.execl(p, p, *sys.argv)
rec_id=""
# 這是do_listena callback function called,Pass the data over for processing,在子類中實現
with open(filename, 'a+') as f:
for i, row in enumerate(data):
ip = row[1]
if(i==0):
f.write(ip)
else:
f.write('\n'+ip)
rec_id = row[0]
return rec_id
class MonitorTest(BaseMonitor):
def callback(self, dictdata):
print
"Monitor:",dictdata
if __name__ == "__main__":
ad = Listener()
ad.start()

其余兩個 有機會再寫吧
監控日志 trigger call script
Then this restart function,在這篇博客mentioned in the header,所以在pycharmThe above cannot be successfully demonstrated,但在服務器linuxShould be onok了.

windows下任務管理器中運行在pycharm或者其他ide下的python應用程序,我們知道此時pycharm是進程,而運行的.py文件是線程功能,這樣如果監測進程實現起來比較繁瑣,因此可以將.py文件轉換為.exe文件使用pyinstaller將py文件轉換成.exe可執行文件,這樣在windows直接執行.exe文件*

So the task is like this,最後還是選擇了,輪詢的方法,Then every time I talkstreams.txtDocument empty,重新寫入,When the next time after the first batch is written,就重啟.(because i'm restarting,重啟之後,Those record marks,都沒有了,in the blog case,Also expect this tag to record,Then it is convenient for the next query)
So I just put The number of records was recorded,If the number increases,我就重啟.《——對 I have become this logic
okokok
煩死了,yourself too rubbish,such a small function,I babbled for four or five hours.


  1. 上一篇文章:
  2. 下一篇文章:
Copyright © 程式師世界 All Rights Reserved