目錄
1. Pandas的基本概念
1.1 Series方法：
1.2 DataFrame類似於二維數組，有行列之分
2.選擇：從Series和DataFrame實例中選擇部分數據
2.1 Series:索引或索引位置
2.2 Series屬性:iloc，loc（按“行”來索引）
3. DataFrame的索引方式
3.1 按 行 或 列 進行索引
3.2 讀取多行多列：loc方法
3.3 二維選擇
4. 缺失值與數據自動對齊
4.1 Series方法
4.2 DataFrame方法
4.3 填充NaN方法：
6.數據合並與分組
6.1 合並兩個DataFrame兩種方法：
6.1.1 簡單拼接----concat
6.1.2 根據列名查詢逐一合並---merge
6.2 Pandas也支持類似於數據庫查詢語句GROUP BY,可完成分組按照某列
7. 時間序列處理
7.1 時間差的運算
7.2 pandas與datetime
7.3 pandas可生成日期范圍通過方法.date_range函數
Pandas： 數據分析，在Numpy基礎上增加了高級功能：數據自動對齊，時間序列支持、缺失數據靈活處理等等 Series、DataFrame核心數據結構，大部分Pandas功能都圍繞這兩種數據結構進行 Series是一個值得序列，可以理解成一維數組，有一個列和一個索引，索引可以定制
import pandas as pd
s1 = pd.Series([1,2,3,4,5])
print(s1)
"""
D:\Anaconda3\python.exe D:/Python_file_forAnconda3_python/數據分析/自定義學習/Pandas01.py
0 1
1 2
2 3
3 4
4 5
dtype: int64
Process finished with exit code 0
"""import pandas as pd
s2 = pd.Series([1,2,3,4,5],index=['a','b','c','d','e'])
print(s2)
"""
D:\Anaconda3\python.exe D:/Python_file_forAnconda3_python/數據分析/自定義學習/Pandas01.py
a 1
b 2
c 3
d 4
e 5
dtype: int64
"""import pandas as pd
import numpy as np
df = pd.DataFrame(np.random.randn(4,4),index=['a','b','c','d'],columns=['A','B','C','D'])
print(df)
"""
D:\Anaconda3\python.exe D:/Python_file_forAnconda3_python/數據分析/自定義學習/Pandas01.py
A B C D
a 0.341299 -1.501784 1.069910 0.879989
b 0.416756 1.066293 0.569988 2.745966
c 0.711972 -0.336308 -0.006444 1.322002
d 2.217314 -0.281477 -0.706486 0.117150
Process finished with exit code 0
"""通過指定索引-index和標簽-columns創建DataFrame對象，可以通過df.index和df.columns訪問索引和標簽:
df.index
Out[12]: Index(['a', 'b', 'c', 'd'], dtype='object')
df.columns
Out[13]: Index(['A', 'B', 'C', 'D'], dtype='object')import pandas as pd
import numpy as np
s2 = pd.Series([1,2,3,4,5],index=['a','b','c','d','e'])
print(s2[0])
print('_______')
print(s2[0:3])
print(s2['a'])
print("________")
print(s2['a':'c'])
"""
D:\Anaconda3\python.exe D:/Python_file_forAnconda3_python/數據分析/自定義學習/Pandas01.py
1
_______
a 1
b 2
c 3
dtype: int64
1
________
a 1
b 2
c 3
dtype: int64
Process finished with exit code 0
"""
import pandas as pd
import numpy as np
s2 = pd.Series([1,2,3,4,5],index=['a','b','c','d','e'])
print(s2.iloc[0:3]) #按照默認索引訪問
print("--------------")
print(s2.loc['a':'c']) #按照自定義的index訪問
"""
D:\Anaconda3\python.exe D:/Python_file_forAnconda3_python/數據分析/自定義學習/Pandas01.py
a 1
b 2
c 3
dtype: int64
--------------
a 1
b 2
c 3
dtype: int64
Process finished with exit code 0
"""
標簽取值-列 df.A df['A'] 索引位置-行 df.loc['a'] #該方法用的是自定義的index值來索引 df.iloc[0] #該方法用的是默認的index來索引 索引位置多行-多列: df.loc[:,['B','C','D']] 二維選擇： 點：df.loc['a','A'] 塊：df.loc['a':'b','A':'C']
import pandas as pd
import numpy as np
df = pd.DataFrame(np.random.randn(4,4),index=['a','b','c','d'],columns=['A','B','C','D'])
#按“列”來取數據
print(df.A) # 標簽取值-列
print("-----")
print(df['A']) # 標簽取值-列
"""
D:\Anaconda3\python.exe D:/Python_file_forAnconda3_python/數據分析/自定義學習/Pandas01.py
a -0.931263
b -0.648751
c 0.438436
d -1.481929
Name: A, dtype: float64
-----
a -0.931263
b -0.648751
c 0.438436
d -1.481929
Name: A, dtype: float64
"""
import pandas as pd
import numpy as np
df = pd.DataFrame(np.random.randn(4,4),index=['a','b','c','d'],columns=['A','B','C','D'])
print(df)
print("-----")
print(df.loc[:,['B','C','D']]) # 標簽取值-多行多列 (以默認的方式)
"""
D:\Anaconda3\python.exe D:/Python_file_forAnconda3_python/數據分析/自定義學習/Pandas01.py
A B C D
a -1.205197 -0.375471 0.115681 0.111243
b -0.329662 0.001292 -0.540496 -1.274938
c -0.285998 0.122846 -0.738836 0.213211
d -1.479184 0.251340 0.322654 -0.745249
-----
B C D
a -0.375471 0.115681 0.111243
b 0.001292 -0.540496 -1.274938
c 0.122846 -0.738836 0.213211
d 0.251340 0.322654 -0.745249
"""import pandas as pd
import numpy as np
df = pd.DataFrame(np.random.randn(4,4),index=['a','b','c','d'],columns=['A','B','C','D'])
print(df)
print("-----")
print(df.loc['a','A']) # 點
print("----")
print(df.loc['a':'b','A':'C'])
"""
D:\Anaconda3\python.exe D:/Python_file_forAnconda3_python/數據分析/自定義學習/Pandas01.py
A B C D
a -0.234136 -0.458588 0.672268 -0.749685
b 0.462632 0.681731 1.438152 -0.073641
c -0.649510 0.443019 0.361910 0.589839
d -2.194516 -1.881632 -0.470177 2.606073
-----
-0.23413573419505523
----
A B C
a -0.234136 -0.458588 0.672268
b 0.462632 0.681731 1.438152
"""
該功能可以對不同索引對象進行算術運算，運算過程中缺失值在運算中傳播會以NaN值進行自動填寫。
import pandas as pd
import numpy as np
s1 = pd.Series([1,2,3,4], index=['a','b','c','d'])
s2 = pd.Series([2,3,4,5], index=['b','c','d','e'])
print(s1+s2)
'''
D:\Anaconda3\python.exe D:/Python_file_forAnconda3_python/數據分析/自定義學習/Pandas01.py
a NaN
b 4.0
c 6.0
d 8.0
e NaN
dtype: float64
'''import pandas as pd
import numpy as np
df1 = pd.DataFrame(np.arange(9).reshape(3,3),columns=list('ABC'),index=list('abc'))
df2 = pd.DataFrame(np.arange(12).reshape(3,4),columns=list('ABCE'),index=list('bcd'))
print(df1+df2)
'''
D:\Anaconda3\python.exe D:/Python_file_forAnconda3_python/數據分析/自定義學習/Pandas01.py
A B C E
a NaN NaN NaN NaN
b 3.0 5.0 7.0 NaN
c 10.0 12.0 14.0 NaN
d NaN NaN NaN NaN
'''df1.add(df2, fill_value=0)
import pandas as pd
import numpy as np
df1 = pd.DataFrame(np.arange(9).reshape(3,3),columns=list('ABC'),index=list('abc'))
df2 = pd.DataFrame(np.arange(12).reshape(3,4),columns=list('ABCE'),index=list('bcd'))
print(df1+df2)
print('------')
print(df1.add(df2, fill_value=0))
'''
D:\Anaconda3\python.exe D:/Python_file_forAnconda3_python/數據分析/自定義學習/Pandas01.py
A B C E
a NaN NaN NaN NaN
b 3.0 5.0 7.0 NaN
c 10.0 12.0 14.0 NaN
d NaN NaN NaN NaN
------
A B C E
a 0.0 1.0 2.0 NaN
b 3.0 5.0 7.0 3.0
c 10.0 12.0 14.0 7.0
d 8.0 9.0 10.0 11.0
'''5. 運算統計
統計： 類似Numpy,Series與DataFrame也可以使用各種統計方法：平均值、方差、求和等等，可通過descirbe方法可以獲取常見統計信息 A B C count 3.0 3.0 3.0 元素值得數量 mean 3.0 4.0 5.0 平均數 std 3.0 3.0 3.0 標准差 min 0.0 1.0 2.0 最小值 25% 1.5 2.5 3.5 取值百分比 50% 3.0 4.0 5.0 取值百分比 75% 4.5 5.5 6.5 取值百分比 max 6.0 7.0 8.0 最大值
import pandas as pd
import numpy as np
df1 = pd.DataFrame(np.random.randn(3,3))
df2 = pd.DataFrame(np.random.randn(3,3),index=[5,6,7])
print(pd.concat([df1,df2]))
"""
D:\Anaconda3\python.exe D:/Python_file_forAnconda3_python/數據分析/自定義學習/Pandas01.py
0 1 2
0 1.236067 0.751290 0.358762
1 -1.605407 -1.296070 -0.167892
2 1.403888 1.962560 0.766084
5 -1.118603 0.845264 -0.890752
6 -1.209584 0.006337 0.310854
7 2.104464 -0.157647 -1.805883
Process finished with exit code 0
"""df1 = pd.DataFrame({'user_id':[5248,13],'course':[12,45],'minutes':[9,36]})
df2 = pd.DataFrame({'course':[12,5], 'name':['Numpy','Pandas']})
print(pd.merge([df1,df2]))import pandas as pd
df1 = pd.DataFrame({'user_id':[5248,13,5348],'course':[12,45,23],'minutes':[9,36,45]})
a = df1[['user_id','minutes']].groupby('user_id').sum() #通過'user_id'和'minutes'來進行分組，並按'user_id'排列
print(a)
"""
D:\Anaconda3\python.exe D:/Python_file_forAnconda3_python/數據分析/自定義學習/Pandas01.py
minutes
user_id
13 36
5248 9
5348 45
Process finished with exit code 0
"""datetime屬性對象： .datetime 代表時間對象 .date 代表某一天 .timedelta 代表時間差
from datetime import datetime, timedelta
d1 = datetime(2020,3,15)
delta = timedelta(days=10) #時間為10天
print(d1+delta)
"""
D:\Anaconda3\python.exe D:/Python_file_forAnconda3_python/數據分析/自定義學習/Pandas01.py
2020-03-25 00:00:00
"""import pandas as pd
import numpy as np
from datetime import datetime, timedelta
dates = [datetime(2020,3,15),datetime(2020,3,16),datetime(2020,3,17),datetime(2020,3,18)]
ts = pd.Series(np.random.randn(4),index=dates) # 數組ts的索引index定義為dates的值
print(ts)
print('------')
print(dates)
print('------')
print(ts.index[0])
"""
D:\Anaconda3\python.exe D:/Python_file_forAnconda3_python/數據分析/自定義學習/Pandas01.py
2020-03-15 -0.185834
2020-03-16 -2.075404
2020-03-17 -1.093103
2020-03-18 0.171173
dtype: float64
------
[datetime.datetime(2020, 3, 15, 0, 0), datetime.datetime(2020, 3, 16, 0, 0), datetime.datetime(2020, 3, 17, 0, 0), datetime.datetime(2020, 3, 18, 0, 0)]
------
2020-03-15 00:00:00
"""
pandas取索引對應的值： ts[ts.index[0]] # ts.index[0] 表示的是索引值 ts['2020/3/15'] ts['3/15/2020'] ts[datetime(2020,3,15)]
pandas可生成日期范圍通過方法.date_range函數
該函數可傳參：
start: 指定日期范圍起始時間
end： 指定日期范圍截止時間
preiods： 指定日期范圍間隔時間
freq： 指定日期頻率：D-每天，H-每小時，M-每月
5D - 5天
MS- 每個月第一天
BM- 每個月最後一個工作日
1h30min 1小時30分鐘
pd.date_range('2020-1-1','2021',freq='MS')
