數(shù)據(jù)預處理(excel加載對于數(shù)值類數(shù)據(jù)進行異常值和缺失值的識別與處理)

1、數(shù)據(jù)預處理（excel加載對于數(shù)值類數(shù)據(jù)進行異常值和缺失值的識別與處理）importpandasaspdimportnumpyasnpfromscipyimportstatsfromsklearnimportdatasetsdataframe=pd.read_excel(rC:UsersShinelonDesktoppopulation.xls,sheet_name=O,header=O)#使用sklearn.datasets中的make_blobs創(chuàng)建仿真數(shù)據(jù)集feature,_=datasets.make_blobs(n_samples=10,neatures=2,centers=1,ra

2、ndom_state=1)print(查看仿真數(shù)據(jù)集feature：n)print(feature)print(n*2)#設(shè)置索引#dataframe=dataframe.set_index(dataframe姓名)print(dataframe)#print(n*2)#查看有多少行數(shù)據(jù)#print(有多少行數(shù)據(jù)(0表示行)：,dataframe.shapeO,end=n)print(有幾列(1表示列)：,dataframe.shape1,end=n)print(n*2)#按字符索引查詢#print(dataframe.loc小王)#多條件查詢#print(多條件查找：n,(dataframe

3、年齡23)&(dataframe愛好=足球)print(n*2)#自動計算數(shù)值型的列#print(自動計算數(shù)值型的列:n,dataframe.describe()print(n*2)#查看缺失值#print(查看缺失值isnulln)print(對年齡列查看缺失值：,dataframe年齡.isnull().sum()#print(處理缺失值(過濾)notnull:n)print(dataframedataframe年齡.notnull()print(n*2)print(處理缺失值取眾數(shù)平均數(shù)中位數(shù):n)print(fillna.(data.mode()data.mean()data.medi

4、an()n)zongshu=dataframe年齡.mode()#多個眾數(shù)時采用第一個zongshu=zongshu0dataframe=dataframe.fillna(zongshu,axis=1)print(dataframe)print(n*2)print(處理缺失值插值法:n)print(fillna.(method,limit),method方法limit前向或后向填充時最大的填充范Hn)print(dataframe.fillna(method=ffill)print(n*2)print(*3)print(識別異常值：n)featureO,O=1e5feature0,1=1e5#

5、print(feature)print(四分位法：n)defindicies_of_outerliers(x):q1,q3=np.percentile(x,25,75)iqr=q3-q1lower_bound=q1-1.5*iqrupper_bound=q3+1.5*iqrreturnnp.where(xupper_bound)|(xlower_bound)outelier=indicies_of_outerliers(feature)num_outelier=0foriteminenumerate(outelier):print(item)num_outelier+=1print(共：,nu

6、m_outelier,個異常值)print(n*2)print(異常值處理)print(1.丟棄)foriteminenumerate(outelier):drop_row=item1print(drop_row)after_drop_feature=np.delete(feature,drop_row0,axis=0)print(afterdrop:)print(after_drop_feature)print(2.置空)NaN_outlier_feature=featureforiteminenumerate(outelier):index_outlier=item1print(index_outlier)NaN_outlier_featurein