[LightGBM] [Warning] min_data_in_leaf is set=60, min_child_samples=30 will be ignored. Current value: min_data_in_leaf=60 [LightGBM] [Warning] min_data_in_leaf is set=60, min_child_samples=30 will be ignored. Current value: min_data_in_leaf=60 [LightGBM] [Warning] min_data_in_leaf is set=60, min_child_samples=30 will be ignored. Current value: min_data_in_leaf=60 [LightGBM] [Warning] min_data_in_leaf is set=60, min_child_samples=30 will be ignored. Current value: min_data_in_leaf=60 stage 1: time cost : 0.12420 sec stage 2: time cost : 0.12090 sec stage 3: time cost : 0.12970 sec stage 4: time cost : 0.13302 sec stage 1: time cost : 0.00954 sec stage 2: time cost : 0.00869 sec stage 3: time cost : 0.00879 sec stage 4: time cost : 0.00902 sec value feature generate successfully videoid feature generate successfully lenth tag_dict: 47609 tag tfidf feature generate successfully tag dimension: 59393 lenth words_dict: 189467 tag tfidf feature generate successfully words dimension: 59390 folds 0 Training until validation scores don't improve for 200 rounds [100] training's rmse: 0.226424 valid_1's rmse: 0.225884 [200] training's rmse: 0.0973668 valid_1's rmse: 0.0971656 [300] training's rmse: 0.057936 valid_1's rmse: 0.0586671 [400] training's rmse: 0.0475353 valid_1's rmse: 0.0492602 [500] training's rmse: 0.0447344 valid_1's rmse: 0.0471132 [600] training's rmse: 0.0435196 valid_1's rmse: 0.0465598 [700] training's rmse: 0.0426755 valid_1's rmse: 0.0462999 [800] training's rmse: 0.0419673 valid_1's rmse: 0.0461619 [900] training's rmse: 0.0413599 valid_1's rmse: 0.0461027 [1000] training's rmse: 0.0408172 valid_1's rmse: 0.0460891 [1100] training's rmse: 0.0403146 valid_1's rmse: 0.0461019 Early stopping, best iteration is: [977] training's rmse: 0.0409576 valid_1's rmse: 0.046076 folds 1 Training until validation scores don't improve for 200 rounds [100] training's rmse: 0.226544 valid_1's rmse: 0.225426 [200] training's rmse: 0.0971282 valid_1's rmse: 0.0981563 [300] training's rmse: 0.0573839 valid_1's rmse: 0.0604765 [400] training's rmse: 0.0468133 valid_1's rmse: 0.0513215 [500] training's rmse: 0.04392 valid_1's rmse: 0.0493315 [600] training's rmse: 0.0426995 valid_1's rmse: 0.0488075 [700] training's rmse: 0.0418527 valid_1's rmse: 0.0485998 [800] training's rmse: 0.0411682 valid_1's rmse: 0.0485575 [900] training's rmse: 0.040566 valid_1's rmse: 0.048484 [1000] training's rmse: 0.0400298 valid_1's rmse: 0.0484444 [1100] training's rmse: 0.0395374 valid_1's rmse: 0.0484197 [1200] training's rmse: 0.0390424 valid_1's rmse: 0.0484038 [1300] training's rmse: 0.0385941 valid_1's rmse: 0.0483951 [1400] training's rmse: 0.0381847 valid_1's rmse: 0.048402 [1500] training's rmse: 0.0377891 valid_1's rmse: 0.048371 [1600] training's rmse: 0.037364 valid_1's rmse: 0.0483954 [1700] training's rmse: 0.0370085 valid_1's rmse: 0.0483913 Early stopping, best iteration is: [1502] training's rmse: 0.0377785 valid_1's rmse: 0.0483691 folds 2 Training until validation scores don't improve for 200 rounds [100] training's rmse: 0.227932 valid_1's rmse: 0.229577 [200] training's rmse: 0.100058 valid_1's rmse: 0.101833 [300] training's rmse: 0.0584845 valid_1's rmse: 0.060349 [400] training's rmse: 0.0480128 valid_1's rmse: 0.050103 [500] training's rmse: 0.0451386 valid_1's rmse: 0.0474865 [600] training's rmse: 0.0438463 valid_1's rmse: 0.0467042 [700] training's rmse: 0.0430083 valid_1's rmse: 0.0463218 [800] training's rmse: 0.0423034 valid_1's rmse: 0.0461414 [900] training's rmse: 0.0416902 valid_1's rmse: 0.0460446 [1000] training's rmse: 0.0411211 valid_1's rmse: 0.0460113 [1100] training's rmse: 0.0406169 valid_1's rmse: 0.0459762 [1200] training's rmse: 0.0401204 valid_1's rmse: 0.0459549 [1300] training's rmse: 0.0396321 valid_1's rmse: 0.0459738 [1400] training's rmse: 0.0392071 valid_1's rmse: 0.0459784 Early stopping, best iteration is: [1206] training's rmse: 0.0400926 valid_1's rmse: 0.0459515 folds 3 Training until validation scores don't improve for 200 rounds [100] training's rmse: 0.226307 valid_1's rmse: 0.227556 [200] training's rmse: 0.0974512 valid_1's rmse: 0.098307 [300] training's rmse: 0.0581506 valid_1's rmse: 0.0590657 [400] training's rmse: 0.047742 valid_1's rmse: 0.048951 [500] training's rmse: 0.0448986 valid_1's rmse: 0.0465751 [600] training's rmse: 0.0436506 valid_1's rmse: 0.0459655 [700] training's rmse: 0.0427668 valid_1's rmse: 0.0457833 [800] training's rmse: 0.0421014 valid_1's rmse: 0.0456473 [900] training's rmse: 0.0414596 valid_1's rmse: 0.0455921 [1000] training's rmse: 0.0408817 valid_1's rmse: 0.0455654 [1100] training's rmse: 0.0403698 valid_1's rmse: 0.0455323 [1200] training's rmse: 0.0398803 valid_1's rmse: 0.0455336 [1300] training's rmse: 0.0394193 valid_1's rmse: 0.0455275 Early stopping, best iteration is: [1164] training's rmse: 0.0400544 valid_1's rmse: 0.0455123 oof_rmse: 0.046490504968166625 oof_mse: 0.0021613670521951254 test_rmse: 0.10498955445882613 test_mse: 0.011022806545462817 oof_mape: [0.04896782] test_mape: [0.13560733] verification r2: 0.99351663488233 test r2: 0.9630887068109539 regre ranking shape (27116, 2) [LightGBM] [Warning] min_data_in_leaf is set=60, min_child_samples=30 will be ignored. Current value: min_data_in_leaf=60 [LightGBM] [Warning] min_data_in_leaf is set=60, min_child_samples=30 will be ignored. Current value: min_data_in_leaf=60 [LightGBM] [Warning] min_data_in_leaf is set=60, min_child_samples=30 will be ignored. Current value: min_data_in_leaf=60 [LightGBM] [Warning] min_data_in_leaf is set=60, min_child_samples=30 will be ignored. Current value: min_data_in_leaf=60 20210919 rov_feature_add_v1 feature table finish 20210918 rov_feature_add_v1 feature table finish 20210917 rov_feature_add_v1 feature table finish 20210916 rov_feature_add_v1 feature table finish 20210915 rov_feature_add_v1 feature table finish 20210914 rov_feature_add_v1 feature table finish 20210913 rov_feature_add_v1 feature table finish 20210912 rov_feature_add_v1 feature table finish 20210911 rov_feature_add_v1 feature table finish 20210910 rov_feature_add_v1 feature table finish 20210909 rov_feature_add_v1 feature table finish 20210908 rov_feature_add_v1 feature table finish 20210907 rov_feature_add_v1 feature table finish 20210906 rov_feature_add_v1 feature table finish 20210905 rov_feature_add_v1 feature table finish 20210904 rov_feature_add_v1 feature table finish 20210903 rov_feature_add_v1 feature table finish 20210902 rov_feature_add_v1 feature table finish 20210901 rov_feature_add_v1 feature table finish 20210831 rov_feature_add_v1 feature table finish 20210830 rov_feature_add_v1 feature table finish 20210829 rov_feature_add_v1 feature table finish 20210828 rov_feature_add_v1 feature table finish 20210827 rov_feature_add_v1 feature table finish 20210826 rov_feature_add_v1 feature table finish 20210825 rov_feature_add_v1 feature table finish 20210824 rov_feature_add_v1 feature table finish 20210823 rov_feature_add_v1 feature table finish 20210822 rov_feature_add_v1 feature table finish 20210821 rov_feature_add_v1 feature table finish 20210925 rov_predict_table_add_v1 feature table finish stage 1: time cost : 0.07594 sec stage 2: time cost : 0.07060 sec stage 3: time cost : 0.08065 sec stage 4: time cost : 0.08395 sec stage 1: time cost : 0.00955 sec stage 2: time cost : 0.00875 sec stage 3: time cost : 0.00866 sec stage 4: time cost : 0.00883 sec 151 121 (776799, 121) 151 121 (21060, 121) 0 5.953243 1 6.812345 2 4.634729 3 4.990433 4 3.931826 ... 21055 0.000000 21056 0.000000 21057 0.000000 21058 0.000000 21059 0.000000 Name: weighted_retrn_log, Length: 21060, dtype: float64 value feature generate successfully videoid feature generate successfully lenth tag_dict: 47609 tag tfidf feature generate successfully tag dimension: 59393 lenth words_dict: 189467 tag tfidf feature generate successfully words dimension: 59390 folds 0 Training until validation scores don't improve for 200 rounds [100] training's rmse: 0.217296 valid_1's rmse: 0.219055 [200] training's rmse: 0.0933997 valid_1's rmse: 0.0957921 [300] training's rmse: 0.054594 valid_1's rmse: 0.0577442 [400] training's rmse: 0.044312 valid_1's rmse: 0.0479771 [500] training's rmse: 0.0415086 valid_1's rmse: 0.0457105 [600] training's rmse: 0.0402481 valid_1's rmse: 0.0450285 [700] training's rmse: 0.0393805 valid_1's rmse: 0.0447715 [800] training's rmse: 0.0386728 valid_1's rmse: 0.0446799 [900] training's rmse: 0.0380495 valid_1's rmse: 0.0446337 [1000] training's rmse: 0.0374717 valid_1's rmse: 0.0446179 [1100] training's rmse: 0.0369606 valid_1's rmse: 0.0445955 [1200] training's rmse: 0.0364684 valid_1's rmse: 0.0445982 [1300] training's rmse: 0.0359899 valid_1's rmse: 0.0446047 [1400] training's rmse: 0.0355138 valid_1's rmse: 0.0446262 Early stopping, best iteration is: [1220] training's rmse: 0.0363808 valid_1's rmse: 0.0445906 folds 1 Training until validation scores don't improve for 200 rounds [100] training's rmse: 0.215789 valid_1's rmse: 0.214513 [200] training's rmse: 0.0931099 valid_1's rmse: 0.092297 [300] training's rmse: 0.0546024 valid_1's rmse: 0.0552234 [400] training's rmse: 0.0443371 valid_1's rmse: 0.0464082 [500] training's rmse: 0.0414791 valid_1's rmse: 0.0446326 [600] training's rmse: 0.0402743 valid_1's rmse: 0.0442435 [700] training's rmse: 0.0394273 valid_1's rmse: 0.0440974 [800] training's rmse: 0.0387161 valid_1's rmse: 0.0440401 [900] training's rmse: 0.0380969 valid_1's rmse: 0.044004 [1000] training's rmse: 0.0375615 valid_1's rmse: 0.0439732 [1100] training's rmse: 0.0370498 valid_1's rmse: 0.0439087 [1200] training's rmse: 0.0365759 valid_1's rmse: 0.0438935 [1300] training's rmse: 0.036118 valid_1's rmse: 0.0439083 Early stopping, best iteration is: [1172] training's rmse: 0.036709 valid_1's rmse: 0.0438807 folds 2 Training until validation scores don't improve for 200 rounds [100] training's rmse: 0.220769 valid_1's rmse: 0.220882 [200] training's rmse: 0.0971585 valid_1's rmse: 0.0977065 [300] training's rmse: 0.0570418 valid_1's rmse: 0.0587242 [400] training's rmse: 0.04556 valid_1's rmse: 0.0483838 [500] training's rmse: 0.0421882 valid_1's rmse: 0.0457825 [600] training's rmse: 0.0408069 valid_1's rmse: 0.0449982 [700] training's rmse: 0.0398654 valid_1's rmse: 0.0446595 [800] training's rmse: 0.0391188 valid_1's rmse: 0.0444782 [900] training's rmse: 0.0384418 valid_1's rmse: 0.0443802 [1000] training's rmse: 0.0378518 valid_1's rmse: 0.0443594 [1100] training's rmse: 0.037322 valid_1's rmse: 0.0443377 [1200] training's rmse: 0.036808 valid_1's rmse: 0.0443101 [1300] training's rmse: 0.0363635 valid_1's rmse: 0.0442829 [1400] training's rmse: 0.0359017 valid_1's rmse: 0.0442798 [1500] training's rmse: 0.0354922 valid_1's rmse: 0.0443043 Early stopping, best iteration is: [1324] training's rmse: 0.036253 valid_1's rmse: 0.0442693 folds 3 Training until validation scores don't improve for 200 rounds [100] training's rmse: 0.215786 valid_1's rmse: 0.216227 [200] training's rmse: 0.0932123 valid_1's rmse: 0.094359 [300] training's rmse: 0.0548723 valid_1's rmse: 0.0562601 [400] training's rmse: 0.0448026 valid_1's rmse: 0.0462832 [500] training's rmse: 0.0420321 valid_1's rmse: 0.0437981 [600] training's rmse: 0.0408233 valid_1's rmse: 0.0429686 [700] training's rmse: 0.0400402 valid_1's rmse: 0.042586 [800] training's rmse: 0.0393215 valid_1's rmse: 0.0424025 [900] training's rmse: 0.0386931 valid_1's rmse: 0.0422996 [1000] training's rmse: 0.0381022 valid_1's rmse: 0.0422256 [1100] training's rmse: 0.0375841 valid_1's rmse: 0.0421743 [1200] training's rmse: 0.0370754 valid_1's rmse: 0.042115 [1300] training's rmse: 0.0365871 valid_1's rmse: 0.0420674 [1400] training's rmse: 0.0361483 valid_1's rmse: 0.0420477 [1500] training's rmse: 0.0356979 valid_1's rmse: 0.0420595 [1600] training's rmse: 0.0352963 valid_1's rmse: 0.0420646 Early stopping, best iteration is: [1430] training's rmse: 0.0360223 valid_1's rmse: 0.042041 oof_rmse: 0.04370656860219149 oof_mse: 0.0019102641389780713 test_rmse: 0.030188856957077185 test_mse: 0.0009113670843748676 oof_mape: [0.04676871] test_mape: [0.13009724] verification r2: 0.9937893006007936 test r2: 0.9634331570023774 regre ranking shape (21060, 2)