Ver código fonte

分桶文件更新

zhangbo 11 meses atrás
pai
commit
de02836815

Diferenças do arquivo suprimidas por serem muito extensas
+ 0 - 2
src/main/resources/20240609_bucket_274.txt


Diferenças do arquivo suprimidas por serem muito extensas
+ 2 - 0
src/main/resources/20240609_bucket_274_old.txt


+ 1 - 1
src/main/resources/part-00000

@@ -271,4 +271,4 @@ total_time	200	19.0,20.0,21.0,23.0,30.0,41.0,56.0,59.0,64.0,67.0,69.0,70.0,72.0,
 bit_rate	200	118000.0,131000.0,141000.0,148000.0,151000.0,153000.0,155000.0,163000.0,168000.0,173000.0,181000.0,183000.0,187000.0,189000.0,194000.0,195000.0,200000.0,209000.0,211000.0,216000.0,221000.0,222000.0,229000.0,232000.0,235000.0,237000.0,238000.0,239000.0,240000.0,254000.0,261000.0,268000.0,269000.0,279000.0,293000.0,295000.0,309000.0,312000.0,322000.0,326000.0,335000.0,337000.0,339000.0,340000.0,345000.0,353000.0,361000.0,365000.0,369000.0,374000.0,377000.0,388000.0,395000.0,396000.0,399000.0,400000.0,401000.0,410000.0,415000.0,426000.0,428000.0,441000.0,448000.0,449000.0,451000.0,462000.0,469000.0,474000.0,491000.0,523000.0,531000.0,533000.0,534000.0,537000.0,539000.0,548000.0,558000.0,592000.0,614000.0,634000.0,641000.0,673000.0,674000.0,683000.0,700000.0,703000.0,711000.0,714000.0,717000.0,743000.0,753000.0,761000.0,764000.0,775000.0,779000.0,819000.0,829000.0,900000.0,904000.0,976000.0,979000.0,1030000.0,1071000.0,1135000.0,1165000.0,1270000.0,1474000.0,1528000.0,1724000.0,1821000.0,2046000.0,2056000.0,2143000.0,2159000.0,2539000.0,2619000.0,2628000.0,2712000.0,3046000.0,5526000.0,8355000.0,9585000.0,9762000.0,9831000.0,1.0119E7,1.6696E7,7.1248E7
 playcnt_6h	200	2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0,11.0,12.0,13.0,14.0,15.0,16.0,17.0,18.0,19.0,20.0,21.0,22.0,23.0,24.0,25.0,26.0,27.0,28.0,29.0,30.0,31.0,32.0,33.0,34.0,35.0,37.0,38.0,40.0,42.0,45.0,48.0,52.0,57.0,65.0,79.0,240.0
 playcnt_1d	200	2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0,11.0,12.0,13.0,14.0,15.0,16.0,17.0,18.0,19.0,20.0,21.0,22.0,23.0,24.0,25.0,26.0,27.0,28.0,29.0,30.0,31.0,32.0,33.0,34.0,35.0,36.0,37.0,38.0,39.0,40.0,42.0,43.0,44.0,46.0,48.0,50.0,52.0,54.0,57.0,60.0,62.0,66.0,71.0,76.0,83.0,92.0,107.0,135.0,497.0
-playcnt_3d	200	2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0,11.0,12.0,13.0,14.0,15.0,16.0,17.0,18.0,19.0,20.0,21.0,22.0,23.0,24.0,25.0,26.0,27.0,28.0,29.0,30.0,31.0,32.0,33.0,34.0,35.0,36.0,37.0,38.0,39.0,40.0,41.0,42.0,44.0,45.0,46.0,47.0,49.0,50.0,52.0,53.0,55.0,57.0,58.0,60.0,62.0,65.0,67.0,70.0,72.0,75.0,79.0,83.0,87.0,91.0,97.0,104.0,111.0,119.0,131.0,146.0,163.0,194.0,260.0,1157.0
+playcnt_3d	200	2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0,11.0,12.0,13.0,14.0,15.0,16.0,17.0,18.0,19.0,20.0,21.0,22.0,23.0,24.0,25.0,26.0,27.0,28.0,29.0,30.0,31.0,32.0,33.0,34.0,35.0,36.0,37.0,38.0,39.0,40.0,41.0,42.0,44.0,45.0,46.0,47.0,49.0,50.0,52.0,53.0,55.0,57.0,58.0,60.0,62.0,65.0,67.0,70.0,72.0,75.0,79.0,83.0,87.0,91.0,97.0,104.0,111.0,119.0,131.0,146.0,163.0,194.0,260.0,1157.0

+ 2 - 2
src/main/scala/com/aliyun/odps/spark/examples/临时记录的脚本

@@ -90,8 +90,8 @@ nohup /opt/apps/SPARK2/spark-2.4.8-hadoop3.2-1.0.8/bin/spark-class2 org.apache.s
 --master yarn --driver-memory 16G --executor-memory 1G --executor-cores 1 --num-executors 16 \
 --conf spark.driver.maxResultSize=16G \
 ./target/spark-examples-1.0.0-SNAPSHOT-shaded.jar \
-readPath:/dw/recommend/model/14_feature_data/20240606/ fileName:20240606_200_v2 \
-bucketNum:200 sampleRate:0.01 \
+readPath:/dw/recommend/model/14_feature_data/20240606/ fileName:20240606_200_v3 \
+bucketNum:200 sampleRate:0.1 \
 > p15_data2.log 2>&1 &
 
 

Alguns arquivos não foram mostrados porque muitos arquivos mudaram nesse diff