This article mainly introduces how to use Pandas Filter values ¹.
Updated: 2022 / 6 / 21

Python | Pandas | Screening

• expression
• • Other functions
  • • isin
    • • 1.
      • 2.
    • isna/notna
    • • 1.
      • 2.
• query
• • Operator
  • expression
  • Null value
  • • isna / isnull
    • notna / notnull
• Reference link

expression

Other functions

isin

See here ²

1.

• Application scenarios :
  One list There are many values stored in the , You need to extract the values that appear in this list in the target Dataframe All rows in
• Method :

X = data[data['X'].isin(['A', 'B', 'C', 'D'])]
# Unamed:22 Unamed:23
# 561 NaN NaN NaN 
# ... ... ... ...
# 1071 7.7 70805 NaN
# 1072 8.0 80851 NaN
# 
# type(X)
# <class 'pandas.core.frame.DataFrame'>
#
# X.dtype
# AttributeError: 'DataFrame' object has no attribute 'dtype' 

X = data.loc[data['X'].isin(['A', 'B', 'C', 'D'])]

2.

• Application scenarios :
  One list There are many values stored in the , You need to determine whether the value appearing in this list is in the target Dataframe All rows in
• Method :

X = data['X'].isin(['A', 'B', 'C', 'D'])
# 0 True
# 1 False
# ...
# 1076 True
# 1077 False
# 
# type(X)
# <class 'pandas.core.series.Series'>
# 
# X.dtype
# bool

isna/notna

See here ³

1.

• Application scenarios :
  Select column value as null/None/nan The line of
• Method :

X = df[df[2].isna()]
# 0 NaN
# 1 NaN
# ... ...
# 105 NaN
# 106 NaN
#
# type(X)
# <class 'pandas.core.series.Series'>
# 
# X[2].dtype
# object

2.

• Application scenarios :
  The selected column value is not null/None/nan The line of
• Method :

df[df[2].notna()]
# 1869 ABCD
# 10712 EFGH
# ... ...
# 10723 IJK
# 10766 LMN
#
# type(X)
# <class 'pandas.core.series.Series'>
# 
# X[2].dtype
# object

query

More about , You can see df.query The method of data query is introduced in detail ⁴

Operator

df.query('Q1 > Q2 > 90') # Type of direct writing sql where sentence 
df.query('Q1 + Q2 > 180')
df.query('Q1 == Q2')
df.query('(Q1<50) & (Q2>40) and (Q3>90)')
df.query('Q1 > Q2 > Q3 > Q4')

expression

df.query('team != "C"')
df.query('team in ["A","B"]')
df.query('team not in ("E","A","B")')
df.query('team == ["A","B"]')
df.query('team != ["A","B"]')
df.query('name.str.contains("am")') # contain am character 

Null value

See here ⁵

isna / isnull

df.query('col1.isnull()', engine='python')
# col1
# 21 NaN
# 42 NaN
# ...
# 60 NaN

notna / notnull

df.query('col1.notnull()', engine='python')
# col1
# 31 ABCD
# 32 EFGH
# ...
# 65 HIJK

# For columns with spaces in their names , You can use backquotes to enclose 
df.query('B == `team name`')
# Support for incoming variables , Such as ： Greater than average 40 Points of 
a = df.Q1.mean()
df.query('Q1 > @a+40')
df.query('Q1 > `Q2`[email protected]')
# df.eval() Usage and df.query similar 
df[df.eval("Q1 > 90 > Q3 > 10")]
df[df.eval("Q1 > `Q2`[email protected]")]

Reference link