Because the article is too long , Xiaobian also sorted the article into PDF file , If you want to watch and learn more conveniently, click here to download
Part 1 _ Take elder martial brother Ali and arrange it carefully 170 Avenue Python Interview questions , I have landed successfully 【 Comprehensive part ： Network programming 】
Part I _ It's from senior brother Ali 170 Avenue Python Interview questions , I have landed successfully 【Python The basic chapter 】
Comprehensive part ： Database and framework
121. List common databases
122. Three paradigms of database design
123. What is a database transaction
124. MySQL Index type
125. One to many and many to many application scenarios in database design
126. Briefly describe the trigger 、 function 、 View 、 stored procedure
127. Commonly used SQL sentence
128. The difference between primary key and foreign key
129. How to open MySQL Slow log query
130. MySQL Database backup command
131. char and varchar The difference between
132. Leftmost prefix principle
133. Failure to hit the index
134. Database read write separation
135. Database sub database sub table
136. redis and memcached Compare
137. redis How many databases are there by default db And function
138. redis What are some persistence strategies
139. redis Supported expiration policies
140. How to ensure redis The data in is hot data
141. Python operation redis
142. be based on redis Implement publish and subscribe
143. How to efficiently find redis One of the KEY
144. be based on redis Achieve first in first out 、 LIFO and priority queues
145. redis How to realize master-slave replication
146. Cycle to get redis A very large list of data in
147. redis Medium watch The role of the command of
148. redis Distributed lock
149. http agreement
150. uwsgi,uWSGI and WSGI The difference between
151. HTTP Status code
152. HTTP Common request methods
153. Responsive layout
154. Implement a simple one AJAX request
155. The same-origin policy
156. What is? CORS
157. What is? CSRF
158. The front end implements polling 、 Long polling
159. sketch MVC and MTV
160. Idempotence of interface
161. Flask Advantages of frameworks
162. What is? ORM
163. PV、UV The meaning of
164. supervisor The role of
165. Use ORM And native SQL Advantages and disadvantages
166. List some django Built in components of
167. list Django Perform native sql Methods
168. cookie and session The difference between
169. beautifulsoup Role of module
170. Selenium Module description
Relational database ：MySQL,Oracle,SQLServer,SQLite,DB2
Non relational database ：MongoDB,Redis,HBase,Neo4j
Establish a scientific , A standardized database needs to meet some specifications , In order to optimize the way of data storage , In a relational database, these specifications can be called paradigms
First normal form ： When the relationship pattern R All properties of cannot be decomposed into more basic data units , call R Is to satisfy the first paradigm , Short for 1NF
Relationship model R All attributes of can no longer be decomposed
Second normal form ： If the relationship pattern R Satisfy the first paradigm , also R All non primary properties of are completely dependent on R Each candidate key attribute of , call R Satisfying the second paradigm , Short for 2NF
Non primary attributes depend on each key attribute
Three paradigms ： set up R It's a relational model that satisfies the first normal form condition ,X yes R Any property set of , If X Non delivery depends on R Any of the candidate keywords , call R Meet the third paradigm , Short for 3NF
Data cannot be transferred , That is, each attribute has a direct relationship with the primary key, not an indirect relationship
Business （Transaction） Is the basic unit of concurrency control . The so-called business , It's a sequence of operations , These operations are either performed , Either not , It is an indivisible unit of work
In a relational database , A transaction can be a SQL sentence 、 A group of SQL Statement or whole program . Four properties ： Atomicity , Uniformity , Isolation and persistence
MySQL At present, there are mainly the following index types ：
General index
unique index
primary key
Composite index
Full-text index
An example of a one-to-one relationship ： One student corresponds to one student file , Or everyone has a unique ID number
One to many relationship example ： A student belongs to only one class , But there are many students in a class
Examples of many to many relationships ： A student can choose more than one course , There are many students in a course
trigger ： A trigger is a special stored procedure , It is the database in insert、update、delete Code blocks that are automatically executed when
function ： Many built-in functions are provided in the database , You can also customize functions , Realization sql Logic
View ： A view is a virtual table formed by query results , Is a projection of a table obtained by some operation
stored procedure ： Encapsulate a piece of code , When it comes to executing this code , This can be achieved by calling the stored procedure （ Call again after the first compilation without recompilation , Than one by one sql Sentence efficiency is high ）
DML（ Data operation language ）
SELECT - Getting data from a database table
UPDATE - Update the data in the database table
DELETE - Delete data from the database table
INSERT INTO - Insert data into a database table
DDL（ Data definition language ）
CREATE DATABASE - Create a new database
ALTER DATABASE - modify the database
CREATE TABLE - Create new table
ALTER TABLE - change （ change ） Database table
DROP TABLE - Delete table
CREATE INDEX - Create index （ The search button ）
DROP INDEX - Delete index
The primary key and foreign key are defined to maintain the integrity of the relational database The primary key is the unique identifier that can determine a record . Can't repeat , Not allowed to be empty
Foreign keys are used to associate with another table . It's a field that can determine another table record , Used to keep data consistent
The primary key foreign key index definition uniquely identifies a record , Can't repeat , It is not allowed to be empty. The foreign key of a table is the primary key of another table , Foreign keys can be repeated , It can be a null value. There is no duplicate value in this field , However, there can be null values to ensure data integrity and establish connections with other tables to improve the speed of query sorting. There can only be one or more
Modify the configuration file , Then restart the service to take effect
stay linux Next ,vim /etc/my.cnf, stay [mysqld] Add... Under the content item ：slow_query_log = ON long_query_time = 2 # Query over 2 Seconds will be recorded
Command line , But it will fail after restarting the service SET GLOBAL slow_query_log = 'ON'; SET GLOBAL long_query_time = 2;
mysqldump -u user name -p Database name > Exported file name
char： It's very convenient to store fixed length data ,CHAR The index on the field is of high efficiency , The length must be defined in brackets , There can be default values , Like defining char(10)
varchar： Store variable length data , But storage efficiency is not CHAR high , The length must be defined in brackets , There can be default values
mysql Create multi column indexes （ Joint index ） The principle of having the leftmost prefix , That is, the leftmost priority , Such as ：
If there is a 2 Column index (col1,col2), It's already true (col1)、(col1,col2) Indexed on
If there is a 3 Column index (col1,col2,col3), It's already true (col1)、(col1,col2)、(col1,col2,col3) Indexed on
Use or Keywords can result in failure to hit the index
Left leading query will result in failure to hit the index , Such as like '%a' perhaps like '%a%' The index column of a single column index is null Full value matching invalidates the index , The composite index is all null The index is invalid when
A column whose composite index does not conform to the left prefix rule cannot hit the index , If we have 4 Columns a、b、c、d, We create a composite index INDEX(a,b,c,d), Then the query that can hit the index is a,ab,abc,abcd, Other than that, the index couldn't be hit
Forcing a type conversion causes the index to fail
Negative query criteria will make the index unusable , such as NOT IN,NOT LIKE,!= etc. If mysql It is estimated that full table scanning is faster than indexing , Index is not used
Read / write separation , Is to divide the database into master and slave databases , A main library for writing data , Multiple slave libraries complete the operation of reading data , The master and slave libraries synchronize data through some mechanism , Is a common database architecture
Database horizontal segmentation , Is a common database architecture , It's an algorithm , The architecture that divides the database . Each database in a horizontal segmentation cluster , It's usually called a “ Fragmentation ”. The data in each slice does not coincide , The data in all slices are combined to form all data .
The horizontal segmentation is divided into two parts: in database table and in database table , It is based on the internal logical relationship of the data in the table , Distribute the same table to multiple databases or tables according to different conditions , Each table contains only a part of data , Thus, the data quantity of a single table is reduced , To achieve a distributed effect
redis and memcached It's all about storing data in memory , It's all memory databases . however memcached It can also be used to cache other things , For example, pictures 、 Video and so on
redis It's not just about supporting simple k/v Data of type , It also provides list,set,hash Such as data structure storage
Distributed settings , Can be one master and many slaves or one master and one slave
Storage data security ,memcached After hanging up , The data is completely lost ;redis Can be saved to disk on a regular basis （ Persistence ）
disaster recovery ,memcached After hanging up , Data is not recoverable ; redis Data can be lost through aof recovery
redis The default is 16 A database , The data in each database is isolated , such , When storing data , You can specify to store different data in different databases . And only a single machine has , If it's a cluster, there's no concept of a database
RDB Persistence ： Yes, it will Reids Database record timing in memory dump Persistence to disk AOF（append only file） Persistence ： take Reids The operation log of is written to the file as an append
Three common expiration strategies
Delete regularly Set up key At the same time , For the sake of key Create a timer , Let the timer be in key When the expiration time of , Yes key To delete
Lazy deletion key Do not delete when expired , Get... From the database every time key Check if it's overdue , If overdue , Delete , return null
Delete periodically Delete expiration every once in a while key operation
redis Use lazy delete + Delete policies on a regular basis
limit Redis Memory footprint ,Redis Will eliminate strategies based on their own data , Load hot data into memory . therefore , Calculate the approximate memory occupied by all hotspot data , Then set it up Redis Memory limit
Use redis Third party libraries to operate
import redis
# Create a redis Connection pool
def redis_conn_pool():
pool = redis.ConnectionPool(host='redis-host', port=redis-port,
decode_responses=True, password='redis-pwd')
r = redis.Redis(connection_pool=pool)
return r
subscriber
if __name__ == "__main__":
conn = redis.Redis(host='',
port=12143, password='')
ps = conn.pubsub()
ps.subscribe('chat') # from chat Subscribe to news
for item in ps.listen(): # Listening state ： If you have any news, bring it here
if item['type'] == 'message':
print(item)
print(item['channel'])
print(item['data'])
Publisher
if __name__ == "__main__":
number_list = ['300033', '300032', '300031', '300030']
signal = ['1', '-1', '1', '-1']
pool = redis.ConnectionPool(host='redis-12143.c8.us-east-1-3.ec2.cloud.redislabs.com', port=12143,
decode_responses=True, password='pkAWNdYWfbLLfNOfxTJinm9SO16eSJFx')
r = redis.Redis(connection_pool=pool)
for i in range(len(number_list)):
value_new = str(number_list[i]) + ' ' + str(signal[i])
print(value_new)
r.publish("chat", value_new)
import redis
con = redis.Redis()
con.keys(pattern='key*') # * Represent wildcard
class Zhan:
def __init__(self,conn):
self.conn = conn
def push(self,val):
self.conn.rpush('aaa',val)
def pop(self):
return self.conn.rpop('aaa')
class Dui:
def __init__(self,conn):
self.conn = conn
def push(self,val):
self.conn.rpush('bbb',val)
def get(self):
return self.conn.lpop('bbb')
class Xu:
def __init__(self,conn):
self.conn = conn
def push(self,val,count):
self.conn.zadd('ccc',val,count)
def get(self):
a = self.conn.zrange('ccc', 0, 0)[0]
self.conn.zrem('ccc', a)
return a
Configure from the server SLAVEOF 127.0.0.1 6380 # master server IP, port
def list_iter(name):
"""
Customize redis List incremental iteration
:param name: redis Medium name, namely ： iteration name Corresponding list
:return: yield return List elements
"""
list_count = r.llen(name)
for index in xrange(list_count):
yield r.lindex(name, index)
watch Used in the last step of transaction operation, that is, executing exec For someone before key To monitor , If this is monitored key Changed , Then the transaction is cancelled , Otherwise, the transaction will execute normally
by redis Cluster design lock , Prevent multiple tasks from modifying the database at the same time , Its essence is to set a timeout string for each host in the cluster , When more than half of the machines in the cluster are set successfully, it is considered that locking is successful , Until the lock expires or is unlocked, the second task will not lock successfully
Hypertext transfer protocol （HTTP,HyperText Transfer Protocol) It is the most widely used network protocol on the Internet .HTTP Is a client and server-side request and response standard . Client is end user , Server side is website . Generally by HTTP Client initiates a request , Establish a specified port to the server （ The default is 80 port ） Of TCP Connect ,HTTP On that port, the server listens for requests from clients , And respond
WSGI： The full name is Web Server Gateway Interface, It's a description web server How to communicate with web application Specification of communication .django,flask And so on
uwsgi： It is a protocol between server and server application , Specifies how to forward requests to applications and return ; uwsgi It's a line protocol, not a communication protocol , This is often used in uWSGI Data communication between server and other network servers
uWSGI： It's a Web The server , It has achieved WSGI agreement 、uwsgi、http Such agreement .Nginx in HttpUwsgiModule Its function is to communicate with uWSGI The server exchanges
1xx: Information
2xx： success
3xx： Redirect
4xx： Client error
5xx： Server error
GET,POST,PUT,DELETE,PATCH etc.
The reactive layout is Ethan Marcotte stay 2010 year 5 A concept put forward in May , In short , A website can be compatible with multiple terminals —— Instead of making a specific version for each terminal
AJAX It's a way to do this without reloading the entire web page , Technology to update some web pages .
AJAX = asynchronous JavaScript and XML
$(function(){
$('#send').click(function(){
$.ajax({
type: "GET",
url: "test.json",
data: {username:$("#username").val(), content:$("#content").val()},
dataType: "json",
success: function(data){
$('#resText').empty(); // Empty resText Everything in it
var html = '';
$.each(data, function(commentIndex, comment){
html += '<div class="comment"><h6>' + comment['username']
+ ':</h6><p class="para"' + comment['content']
+ '</p></div>';
});
$('#resText').html(html);
}
});
});
});
The same origin policy restricts how documents or scripts loaded from the same source interact with resources from another source . This is an important security mechanism for isolating potentially malicious files
If two pages of the protocol , port （ If there is a designation ） Same as the host , The two pages have the same source . We can also call it “ agreement / host / port tuple”, Or simply called “tuple". ("tuple" ,“ element ”, It means that some things are combined to form a whole , such as （1,2） It's called binary ,（1,2,3） It's three yuan )
CORS The full name is cross domain resource sharing （Cross-Origin Resource Sharing）, It's a kind of AJAX How to request resources across domains , Support for modern browsers
CSRF（Cross-site request forgery）, Chinese name ： Cross-site request forgery , Also known as ：one click attack/session riding, Abbreviation for ：CSRF/XSRF
polling
var xhr = new XMLHttpRequest();
setInterval(function(){
xhr.open('GET','/user');
xhr.onreadystatechange = function(){
};
xhr.send();
},1000)
Long polling
function ajax(){
var xhr = new XMLHttpRequest();
xhr.open('GET','/user');
xhr.onreadystatechange = function(){
ajax();
};
xhr.send();
}
So-called MVC Is to put web Applications are divided into models (M), controller (C), View (V) Three layers , They are like plug-ins , It's loosely coupled . The model is responsible for business objects and database objects (ORM), The view is responsible for interacting with the user ( page ), controller (C) Accept the user's input and call the model and view to complete the user's request
Django Medium MTV Pattern ：
Model( Model )： Objects responsible for business objects and databases (ORM)
Template( Template )： Responsible for how to display the page to users
View( View )： Responsible for business logic , And call when appropriate Model and Template, Essentially with MVC identical
Interface idempotency means that the results of one request or multiple requests initiated by the user for the same operation are consistent , It won't have side effects due to multiple clicks
concise , light , Extensibility is strong , High degree of freedom
ORM The full name is Object Relational Mapping, Object relation mapping . Its implementation idea is to map the data of tables in the relational database to objects , In the form of objects , In this way, developers can turn the operation of the database into the operation of these objects
PV： yes （page view） Traffic volume , Page views or hits , Measure the number of pages visited by website users . Every time a user opens or refreshes a page in a certain statistical cycle, they will record 1 Time , If the same page is opened or refreshed multiple times, the total number of views will be accumulated
UV： yes （Unique Visitor） Independent visitor , Count the number of users who visit a site in a period of time ( With cookie On the basis of )
supervisor Management process , It's through fork/exec Think of these managed processes as supervisor To start , So we just need to add the path of the executable file of the process to be managed to supervisor In the configuration file
advantage ：
Easy to use object-oriented , Clear sentences
Effectively prevent SQL Inject
Facilitate the dynamic construction of statements , For the same operation of different tables, it is more elegant to adopt polymorphic implementation ;
To some extent, it is convenient to reconstruct the data layer
Easy to set hook function
shortcoming ：
It's not easy to deal with complex query statements
The performance is more direct SQL Bad
Admin Components ： It's right model Add, delete, modify and query the components provided by the corresponding data table in
model Components ： Responsible for operating database
form Components ： Generate HTML Code ; Data validation ; Check information return and display
ModelForm Components ： For database operations , Can also be used for user requested authentication
Use execute Perform custom SQL Direct execution SQL sentence （ Be similar to pymysql Usage of ）
from django.db import connection
cursor = connection.cursor()
cursor.execute("SELECT DATE_FORMAT(create_time, '%Y-%m') FROM blog_article;")
ret = cursor.fetchall()
print(ret)
Use extra Method :queryset.extra(select={"key": " Native SQL sentence "})
Use raw Method
Perform the original sql And return to the model
Depend on model Model , It is mostly used for query operation
cookie Is the key value pair saved on the browser side , Can be used for user authentication
sesseion Is to save the user's session information on the server ,key The value is a randomly generated string ,value The value is session The content of , Depend on cookie Save each user's random string to the user's browser
BeautifulSoup The library is parsing 、 Traverse 、 maintain “ Tag tree ” The library of
url = "http://www.baidu.com/"
request = requests.get(url)
html = request.content
soup = BeautifulSoup(html, "html.parser", from_encoding="utf-8")
Selenium It is a library that simulates the operation of the browser , According to our instructions , Let the browser load the page automatically , Get the data you need , Even a screenshot of the page , Or judge whether some actions on the website happen, etc
from selenium import webdriver
browser = webdriver.Chrome()
browser.get('https://www.taobao.com')
print(browser.page_source) # browser.page_source Is to get all the pages html
browser.close()
