Ana Sayfa Keşfet Yardım
Giriş Yap
algorithm
/
recommend-emr-dataprocess
12
0
Çatalla 0
Dosyalar Sorunlar 0 Değişiklik İstekleri 1 Wiki
Ağaç: 1ec2edc3a0
Dallar Biçim İmleri
recommend-em...
/
src
/
main
/
scala
/
com
/
aliyun
/
odps
/
spark
/
examples
/
临时记录的脚本-广告

临时记录的脚本-广告 2.2 KB
Geçmiş Ham

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960 
  1. 

  2. 

  3. 

  4. nohup /opt/apps/SPARK2/spark-2.4.8-hadoop3.2-1.0.8/bin/spark-class2 org.apache.spark.deploy.SparkSubmit \
    5. 

  5. --class com.aliyun.odps.spark.examples.makedata_ad.makedata_ad_31_originData_20240620 \
    6. 

  6. --master yarn --driver-memory 1G --executor-memory 2G --executor-cores 1 --num-executors 16 \
    7. 

  7. ./target/spark-examples-1.0.0-SNAPSHOT-shaded.jar \
    8. 

  8. tablePart:64 repartition:32 \
    9. 

  9. beginStr:2024062008 endStr:2024062223 \
    10. 

  10. savePath:/dw/recommend/model/31_ad_sample_data_fix/ \
    11. 

  11. table:alg_recsys_ad_sample_all \
    12. 

  12. > p31_2024062008.log 2>&1 &
    13. 

  13. 

  14. 

  15. 

  16. 

  17. nohup /opt/apps/SPARK2/spark-2.4.8-hadoop3.2-1.0.8/bin/spark-class2 org.apache.spark.deploy.SparkSubmit \
    18. 

  18. --class com.aliyun.odps.spark.examples.makedata_ad.makedata_ad_32_bucket_20240622 \
    19. 

  19. --master yarn --driver-memory 16G --executor-memory 1G --executor-cores 1 --num-executors 16 \
    20. 

  20. --conf spark.driver.maxResultSize=16G \
    21. 

  21. ./target/spark-examples-1.0.0-SNAPSHOT-shaded.jar \
    22. 

  22. fileName:20240620_100_fix \
    23. 

  23. savePath:/dw/recommend/model/32_bucket_data/ \
    24. 

  24. > p32_data.log 2>&1 &
    25. 

  25. 

  26. 

  27. nohup /opt/apps/SPARK2/spark-2.4.8-hadoop3.2-1.0.8/bin/spark-class2 org.apache.spark.deploy.SparkSubmit \
    28. 

  28. --class com.aliyun.odps.spark.examples.makedata_ad.makedata_ad_33_bucketData_20240622 \
    29. 

  29. --master yarn --driver-memory 2G --executor-memory 4G --executor-cores 1 --num-executors 16 \
    30. 

  30. ./target/spark-examples-1.0.0-SNAPSHOT-shaded.jar \
    31. 

  31. beginStr:20240623 endStr:20240623 repartition:400 \
    32. 

  32. filterNames:XXXXXXX \
    33. 

  33. > p33_data.log 2>&1 &
    34. 

  34. 

  35. 

  36. filterNames:cid_,adid_,adverid_,targeting_conversion_ \
    37. 

  37. savePath:/dw/recommend/model/33_ad_train_data_nosparse/ \
    38. 

  38. 

  39. 

  40. /dw/recommend/model/31_ad_sample_data/
    41. 

  41. /dw/recommend/model/33_ad_train_data/
    42. 

  42. 

  43. /dw/recommend/model/31_ad_sample_data_fix/
    44. 

  44. 

  45. 

  46. 

  47. 

  48. 

  49. nohup /opt/apps/SPARK2/spark-2.4.8-hadoop3.2-1.0.8/bin/spark-class2 org.apache.spark.deploy.SparkSubmit \
    50. 

  50. --class com.aliyun.odps.spark.examples.makedata_ad.makedata_ad_33_bucketDataPrint_20240628 \
    51. 

  51. --master yarn --driver-memory 2G --executor-memory 4G --executor-cores 1 --num-executors 16 \
    52. 

  52. ./target/spark-examples-1.0.0-SNAPSHOT-shaded.jar \
    53. 

  53. beginStr:2024062717 endStr:2024062723 \
    54. 

  54. readDate:20240627 \
    55. 

  55. table:alg_recsys_ad_sample_all_new \
    56. 

  56. savePath:/dw/recommend/model/33_for_check/ \
    57. 

  57. > p33_data_check.log 2>&1 &
    58. 

  58. 

  59. 

  60. /dw/recommend/model/33_for_check_v1/
    61.