dnn test

recommend-model-produce/src/main/python/models/dnn/README.md

@@ -0,0 +1,149 @@
+# 基于DNN模型的点击率预估模型
+
+**[AI Studio在线运行环境](https://aistudio.baidu.com/aistudio/projectdetail/3240347)**
+
+以下是本例的简要目录结构及说明： 
+
+```
+├── data #样例数据
+    ├── sample_data #样例数据
+        ├── train
+            ├── sample_train.txt #训练数据样例
+├── __init__.py
+├── README.md #文档
+├── config.yaml # sample数据配置
+├── config_bigdata.yaml # 全量数据配置
+├── net.py # 模型核心组网（动静统一）
+├── criteo_reader.py #数据读取程序
+├── static_model.py # 构建静态图
+├── dygraph_model.py # 构建动态图
+├── benchmark.yaml # benchmark的配置
+├── benchmark_data.sh # benchmark的数据处理
+├── benchmark_reader.py # benchmark的数据读取程序
+```
+
+注：在阅读该示例前，建议您先了解以下内容：
+
+[paddlerec入门教程](https://github.com/PaddlePaddle/PaddleRec/blob/master/README.md)
+
+## 内容
+
+- [模型简介](#模型简介)
+- [数据准备](#数据准备)
+- [运行环境](#运行环境)
+- [快速开始](#快速开始)
+- [模型组网](#模型组网)
+- [效果复现](#效果复现)
+- [进阶使用](#进阶使用)
+- [FAQ](#FAQ)
+
+
+## 模型简介
+`CTR(Click Through Rate)`，即点击率，是“推荐系统/计算广告”等领域的重要指标，对其进行预估是商品推送/广告投放等决策的基础。简单来说，CTR预估对每次广告的点击情况做出预测，预测用户是点击还是不点击。CTR预估模型综合考虑各种因素、特征，在大量历史数据上训练，最终对商业决策提供帮助。
+
+## 数据准备
+
+训练及测试数据集选用[Display Advertising Challenge](https://www.kaggle.com/c/criteo-display-ad-challenge/)所用的Criteo数据集。该数据集包括两部分：训练集和测试集。训练集包含一段时间内Criteo的部分流量，测试集则对应训练数据后一天的广告点击流量。
+每一行数据格式如下所示：
+```
+<label> <integer feature 1> ... <integer feature 13> <categorical feature 1> ... <categorical feature 26>
+```
+其中```<label>```表示广告是否被点击，点击用1表示，未点击用0表示。```<integer feature>```代表数值特征（连续特征），共有13个连续特征。```<categorical feature>```代表分类特征（离散特征），共有26个离散特征。相邻两个特征用```\t```分隔，缺失特征用空格表示。测试集中```<label>```特征已被移除。  
+在模型目录的data目录下为您准备了快速运行的示例数据，若需要使用全量数据可以参考下方[效果复现](#效果复现)部分。
+
+## 运行环境
+PaddlePaddle>=2.0
+
+python 2.7/3.5/3.6/3.7
+
+os : windows/linux/macos 
+
+## 快速开始
+本文提供了样例数据可以供您快速体验，在任意目录下均可执行。在DNN模型目录的快速执行命令如下： 
+```bash
+# 进入模型目录
+# cd models/rank/dnn # 在任意目录均可运行
+# 动态图训练
+python -u ../../../tools/trainer.py -m config.yaml # 全量数据运行config_bigdata.yaml 
+# 动态图预测
+python -u ../../../tools/infer.py -m config.yaml 
+
+# 静态图训练
+python -u ../../../tools/static_trainer.py -m config.yaml # 全量数据运行config_bigdata.yaml 
+# 静态图预测
+python -u ../../../tools/static_infer.py -m config.yaml 
+``` 
+## 模型组网
+### 数据输入声明
+正如数据准备章节所介绍，Criteo数据集中，分为连续数据与离散（稀疏）数据，所以整体而言，CTR-DNN模型的数据输入层包括三个，分别是：`dense_input`用于输入连续数据，维度由超参数`dense_input_dim`指定，数据类型是归一化后的浮点型数据。`sparse_inputs`用于记录离散数据，在Criteo数据集中，共有26个slot，所以我们创建了名为`1~26`的26个稀疏参数输入，数据类型为整数；最后是每条样本的`label`，代表了是否被点击，数据类型是整数，0代表负样例，1代表正样例。
+
+### CTR-DNN模型组网
+
+CTR-DNN模型的组网比较直观，本质是一个二分类任务，代码参考`net.py`。模型主要组成是一个`Embedding`层，四个`FC`层，以及相应的分类任务的loss计算和auc计算。
+
+#### Embedding层
+首先介绍Embedding层的搭建方式：`Embedding`层的输入是`sparse_input`，由超参的`sparse_feature_number`和`sparse_feature_dimshape`定义。需要特别解释的是`is_sparse`参数，当我们指定`is_sprase=True`后，计算图会将该参数视为稀疏参数，反向更新以及分布式通信时，都以稀疏的方式进行，会极大的提升运行效率，同时保证效果一致。
+
+各个稀疏的输入通过Embedding层后，将其合并起来，置于一个list内，以方便进行concat的操作。
+
+```
+self.embedding = paddle.nn.Embedding(
+            self.sparse_feature_number,
+            self.sparse_feature_dim,
+            sparse=True,
+            weight_attr=paddle.ParamAttr(
+                name="SparseFeatFactors",
+                initializer=paddle.nn.initializer.Uniform()))
+```
+
+#### FC层
+将离散数据通过embedding查表得到的值，与连续数据的输入进行`concat`操作，合为一个整体输入，作为全链接层的原始输入。我们共设计了4层FC，每层FC的输出维度由超参`fc_sizes`指定，每层FC都后接一个`relu`激活函数，每层FC的初始化方式为符合正态分布的随机初始化，标准差与上一层的输出维度的平方根成反比。
+```
+sizes = [sparse_feature_dim * num_field + dense_feature_dim
+            ] + self.layer_sizes + [2]
+acts = ["relu" for _ in range(len(self.layer_sizes))] + [None]
+self._mlp_layers = []
+for i in range(len(layer_sizes) + 1):
+    linear = paddle.nn.Linear(
+        in_features=sizes[i],
+        out_features=sizes[i + 1],
+        weight_attr=paddle.ParamAttr(
+            initializer=paddle.nn.initializer.Normal(
+                std=1.0 / math.sqrt(sizes[i]))))
+    self.add_sublayer('linear_%d' % i, linear)
+    self._mlp_layers.append(linear)
+    if acts[i] == 'relu':
+        act = paddle.nn.ReLU()
+        self.add_sublayer('act_%d' % i, act)
+        self._mlp_layers.append(act)
+
+```
+#### Loss及Auc计算
+- 预测的结果通过一个输出shape为2的FC层给出，该FC层的激活函数是softmax，会给出每条样本分属于正负样本的概率。
+- 每条样本的损失由交叉熵给出。
+- 我们同时还会计算预测的auc。
+
+### 效果复现
+为了方便使用者能够快速的跑通每一个模型，我们在每个模型下都提供了样例数据。如果需要复现readme中的效果,请按如下步骤依次操作即可。 
+在全量数据下模型的指标如下：
+| 模型 | auc | batch_size | epoch_num| Time of each epoch |
+| :------| :------ | :------ | :------| :------ | 
+| dnn | 0.795+ | 512 | 4 | 约3小时 |
+
+1. 确认您当前所在目录为PaddleRec/models/rank/dnn 
+2. 进入paddlerec/datasets/criteo目录下，执行该脚本，会从国内源的服务器上下载我们预处理完成的criteo全量数据集，并解压到指定文件夹。
+``` bash
+cd ../../../datasets/criteo
+sh run.sh
+``` 
+3. 切回模型目录,执行命令运行全量数据
+```bash
+cd - # 切回模型目录
+# 动态图训练
+python -u ../../../tools/trainer.py -m config_bigdata.yaml # 全量数据运行config_bigdata.yaml 
+python -u ../../../tools/infer.py -m config_bigdata.yaml # 全量数据运行config_bigdata.yaml 
+```
+
+## 进阶使用
+  
+## FAQ

recommend-model-produce/src/main/python/models/dnn/__init__.py

@@ -0,0 +1,13 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.

recommend-model-produce/src/main/python/models/dnn/benchmark.yaml

@@ -0,0 +1,53 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+hyper_parameters:
+  optimizer:
+    class: Adam
+    learning_rate: 0.0001
+    adam_lazy_mode: True
+  sparse_inputs_slots: 27
+  sparse_feature_number: 1000001
+  sparse_feature_dim: 10
+  dense_input_dim: 13
+  fc_sizes: [400, 400, 400]
+
+runner:
+  epochs: 15
+  print_interval: 100
+
+  geo_step: 400
+  sync_mode: "async"  # sync / async /geo / heter
+  thread_num: 16
+  use_gpu: 0
+  
+  model_path: "static_model.py"
+  reader_type: "QueueDataset"  # DataLoader / QueueDataset / RecDataset
+  pipe_command: "python benchmark_reader.py"
+  dataset_debug: False
+  split_file_list: False
+
+  train_batch_size: 1000
+  train_data_dir: "train_data"
+  train_reader_path: "benchmark_reader"
+  model_save_path: "model"
+
+  infer_batch_size: 1000
+  test_data_dir: "test_data"
+  infer_reader_path: "benchmark_reader"
+  infer_load_path: "model"
+  infer_start_epoch: 0
+  infer_end_epoch: 1
+
+  

recommend-model-produce/src/main/python/models/dnn/benchmark_data.sh

@@ -0,0 +1,5 @@
+echo "Begin DownLoad Criteo Data"
+wget --no-check-certificate https://paddlerec.bj.bcebos.com/benchmark/criteo_benchmark_data.tar.gz
+echo "Begin Unzip Criteo Data"
+tar -xf criteo_benchmark_data.tar.gz
+echo "Get Criteo Data Success"

recommend-model-produce/src/main/python/models/dnn/benchmark_gpubox.yaml

@@ -0,0 +1,55 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# workspace
+#workspace: "models/rank/dnn"
+
+
+runner:
+  train_data_dir: "train_data"
+  train_reader_path: "criteo_reader" # importlib format
+  use_gpu: True
+  use_auc: True
+  train_batch_size: 2048
+  epochs: 3
+  print_interval: 10
+  model_save_path: "output_model_dnn_queue"
+
+  sync_mode: "gpubox"
+  thread_num: 30
+  reader_type: "InmemoryDataset"  # DataLoader / QueueDataset / RecDataset / InmemoryDataset
+  pipe_command: "python models/rank/dnn/benchmark_reader.py"
+  dataset_debug: False
+  split_file_list: False
+
+  infer_batch_size: 2
+  infer_reader_path: "criteo_reader" # importlib format
+  test_data_dir: "data/sample_data/train"
+  infer_load_path: "output_model_dnn_queue"
+  infer_start_epoch: 0
+  infer_end_epoch: 3
+# hyper parameters of user-defined network
+hyper_parameters:
+  # optimizer config
+  optimizer:
+    class: Adam
+    learning_rate: 0.001
+    strategy: async
+  # user-defined <key, value> pairs
+  sparse_inputs_slots: 27
+  sparse_feature_number: 1024
+  sparse_feature_dim: 11
+  dense_input_dim: 13
+  fc_sizes: [512, 256, 128, 32]
+  distributed_embedding: 0

recommend-model-produce/src/main/python/models/dnn/benchmark_reader.py

@@ -0,0 +1,93 @@
+# Copyright (c) 2019 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import sys
+import yaml
+import six
+import os
+import copy
+import xxhash
+import paddle.distributed.fleet as fleet
+import logging
+
+cont_min_ = [0, -3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
+cont_max_ = [20, 600, 100, 50, 64000, 500, 100, 50, 500, 10, 10, 10, 50]
+cont_diff_ = [20, 603, 100, 50, 64000, 500, 100, 50, 500, 10, 10, 10, 50]
+hash_dim_ = 1000001
+continuous_range_ = range(1, 14)
+categorical_range_ = range(14, 40)
+
+logging.basicConfig(
+    format='%(asctime)s - %(levelname)s - %(message)s', level=logging.INFO)
+logger = logging.getLogger(__name__)
+
+
+class Reader(fleet.MultiSlotDataGenerator):
+    def init(self, config):
+        self.config = config
+
+    def line_process(self, line):
+        features = line.rstrip('\n').split('\t')
+        dense_feature = []
+        sparse_feature = []
+        for idx in continuous_range_:
+            if features[idx] == "":
+                dense_feature.append(0.0)
+            else:
+                dense_feature.append(
+                    (float(features[idx]) - cont_min_[idx - 1]) /
+                    cont_diff_[idx - 1])
+        for idx in categorical_range_:
+            sparse_feature.append([
+                xxhash.xxh32(str(idx) + features[idx]).intdigest() % hash_dim_
+            ])
+        label = [int(features[0])]
+        return [label] + sparse_feature + [dense_feature]
+
+    def generate_sample(self, line):
+        "Dataset Generator"
+
+        def reader():
+            input_data = self.line_process(line)
+            feature_name = ["dense_input"]
+            for idx in categorical_range_:
+                feature_name.append("C" + str(idx - 13))
+            feature_name.append("label")
+            yield zip(feature_name, input_data)
+
+        return reader
+
+    def dataloader(self, file_list):
+        "DataLoader Pyreader Generator"
+
+        def reader():
+            for file in file_list:
+                with open(file, 'r') as f:
+                    for line in f:
+                        input_data = self.line_process(line)
+                        yield input_data
+
+        return reader
+
+
+if __name__ == "__main__":
+    yaml_path = sys.argv[1]
+    utils_path = sys.argv[2]
+    sys.path.append(utils_path)
+    import common_ps
+    yaml_helper = common_ps.YamlHelper()
+    config = yaml_helper.load_yaml(yaml_path)
+
+    r = Reader()
+    r.init(config)
+    r.run_from_stdin()

recommend-model-produce/src/main/python/models/dnn/config.yaml

@@ -0,0 +1,60 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# workspace
+#workspace: "models/rank/dnn"
+
+
+runner:
+  train_data_dir: "data/sample_data/train"
+  train_reader_path: "criteo_reader" # importlib format
+  use_gpu: False
+  use_auc: True
+  train_batch_size: 2
+  epochs: 1
+  print_interval: 2
+  model_save_path: "output_model_dnn"
+  infer_batch_size: 2
+  infer_reader_path: "criteo_reader" # importlib format
+  test_data_dir: "data/sample_data/train"
+  infer_load_path: "output_model_dnn"
+  infer_start_epoch: 0
+  infer_end_epoch: 3
+  num_workers: 0
+  #use inference save model
+  # model_init_path: "output_model_dnn/2" # init model
+  use_inference: False
+  save_inference_feed_varnames: ["1","2","3","4","5","6","7","8","9","10","11","12","13","14","15","16","17","18","19","20","21","22","23","24","25","26","dense_input"]
+  save_inference_fetch_varnames: ["sigmoid_0.tmp_0"]
+
+  # distribute_config
+  sync_mode: "async"
+  split_file_list: False
+  thread_num: 1
+
+
+# hyper parameters of user-defined network
+hyper_parameters:
+  # optimizer config
+  optimizer:
+    class: Adam
+    learning_rate: 0.001
+    strategy: async
+  # user-defined <key, value> pairs
+  sparse_inputs_slots: 27
+  sparse_feature_number: 1000001
+  sparse_feature_dim: 9
+  dense_input_dim: 13
+  fc_sizes: [256, 256, 128]
+  distributed_embedding: 0

recommend-model-produce/src/main/python/models/dnn/config_bigdata.yaml

@@ -0,0 +1,55 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# workspace
+#workspace: "models/rank/dnn"
+
+
+runner:
+  train_data_dir: "../../../datasets/criteo/slot_train_data_full"
+  train_reader_path: "criteo_reader" # importlib format
+  use_gpu: False
+  use_auc: True
+  train_batch_size: 512
+  epochs: 4
+  print_interval: 10
+  model_save_path: "output_model_dnn_all"
+  infer_reader_path: "criteo_reader" # importlib format
+  test_data_dir: "../../../datasets/criteo/slot_test_data_full"
+  infer_batch_size: 512
+  infer_load_path: "output_model_dnn_all"
+  infer_start_epoch: 0
+  infer_end_epoch: 4
+  num_workers: 0
+
+  #thread_num: 5
+  #reader_type: "QueueDataset"  # DataLoader / QueueDataset / RecDataset
+  #pipe_command: "python3.7 queuedataset_reader.py"
+  #dataset_debug: False
+  #split_file_list: False
+
+# hyper parameters of user-defined network
+hyper_parameters:
+  # optimizer config
+  optimizer:
+    class: Adam
+    learning_rate: 0.001
+    strategy: async
+  # user-defined <key, value> pairs
+  sparse_inputs_slots: 27
+  sparse_feature_number: 1000001
+  sparse_feature_dim: 9
+  dense_input_dim: 13
+  fc_sizes: [512, 256, 128, 32]
+  distributed_embedding: 0

recommend-model-produce/src/main/python/models/dnn/config_cpups.yaml

@@ -0,0 +1,52 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+hyper_parameters:
+  optimizer:
+    learning_rate: 0.0001
+    class: Adam
+    strategy: async
+  dense_input_dim: 13
+  sparse_inputs_slots: 27
+  sparse_feature_number: 1000001
+  sparse_feature_dim: 10
+  fc_sizes: [400, 400, 400]
+
+runner:
+  epochs: 15
+  print_interval: 100
+
+  geo_step: 400
+  sync_mode: "async"  # sync / async /geo / heter
+  thread_num: 16
+  use_gpu: False
+  use_auc: True
+  
+  model_path: "static_model.py"
+  reader_type: "QueueDataset"  # DataLoader / QueueDataset / RecDataset
+  pipe_command: "python models/rank/dnn/queuedataset_reader.py"
+  dataset_debug: False
+  split_file_list: False
+
+  train_batch_size: 1000
+  train_data_dir: "data/sample_data/train"
+  train_reader_path: "criteo_reader"
+  model_save_path: "model"
+
+  infer_reader_type: "DataLoader"  # DataLoader / QueueDataset / RecDataset
+  infer_batch_size: 1000
+  test_data_dir: "data/sample_data/train"
+  infer_reader_path: "criteo_reader"
+  infer_load_path: "model"
+  infer_start_epoch: 0
+  infer_end_epoch: 1

recommend-model-produce/src/main/python/models/dnn/config_gpubox.yaml

@@ -0,0 +1,55 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# workspace
+#workspace: "models/rank/dnn"
+
+
+runner:
+  train_data_dir: "data/sample_data/train"
+  train_reader_path: "criteo_reader" # importlib format
+  use_gpu: True
+  use_auc: True
+  train_batch_size: 32
+  epochs: 3
+  print_interval: 10
+  model_save_path: "output_model_dnn_queue"
+
+  sync_mode: "gpubox"
+  thread_num: 30
+  reader_type: "InmemoryDataset"  # DataLoader / QueueDataset / RecDataset / InmemoryDataset
+  pipe_command: "python models/rank/dnn/queuedataset_reader.py"
+  dataset_debug: False
+  split_file_list: False
+
+  infer_batch_size: 2
+  infer_reader_path: "criteo_reader" # importlib format
+  test_data_dir: "data/sample_data/train"
+  infer_load_path: "output_model_dnn_queue"
+  infer_start_epoch: 0
+  infer_end_epoch: 3
+# hyper parameters of user-defined network
+hyper_parameters:
+  # optimizer config
+  optimizer:
+    class: Adam
+    learning_rate: 0.001
+    strategy: async
+  # user-defined <key, value> pairs
+  sparse_inputs_slots: 27
+  sparse_feature_number: 1024
+  sparse_feature_dim: 9
+  dense_input_dim: 13
+  fc_sizes: [512, 256, 128, 32]
+  distributed_embedding: 0

recommend-model-produce/src/main/python/models/dnn/config_queuedataset.yaml

@@ -0,0 +1,54 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# workspace
+#workspace: "models/rank/dnn"
+
+
+runner:
+  train_data_dir: "data/sample_data/train"
+  train_reader_path: "criteo_reader" # importlib format
+  use_gpu: False
+  use_auc: True
+  train_batch_size: 2
+  epochs: 3
+  print_interval: 10
+  model_save_path: "output_model_dnn_queue"
+
+  thread_num: 1
+  reader_type: "QueueDataset"  # DataLoader / QueueDataset / RecDataset
+  pipe_command: "python3.7 queuedataset_reader.py"
+  dataset_debug: False
+  split_file_list: False
+
+  infer_batch_size: 2
+  infer_reader_path: "criteo_reader" # importlib format
+  test_data_dir: "data/sample_data/train"
+  infer_load_path: "output_model_dnn_queue"
+  infer_start_epoch: 0
+  infer_end_epoch: 3
+# hyper parameters of user-defined network
+hyper_parameters:
+  # optimizer config
+  optimizer:
+    class: Adam
+    learning_rate: 0.001
+    strategy: async
+  # user-defined <key, value> pairs
+  sparse_inputs_slots: 27
+  sparse_feature_number: 1000001
+  sparse_feature_dim: 9
+  dense_input_dim: 13
+  fc_sizes: [512, 256, 128, 32]
+  distributed_embedding: 0

recommend-model-produce/src/main/python/models/dnn/config_to_static.yaml

@@ -0,0 +1,55 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# workspace
+#workspace: "models/rank/dnn"
+
+
+runner:
+  train_data_dir: "data/sample_data/train"
+  train_reader_path: "criteo_reader" # importlib format
+  model_init_path: "output_model_dnn/0" # model_init
+  use_gpu: False
+  use_auc: True
+  train_batch_size: 2
+  epochs: 1
+  print_interval: 2
+  model_save_path: "output_model_dnn2" # save path
+  infer_batch_size: 2
+  infer_reader_path: "criteo_reader" # importlib format
+  test_data_dir: "data/sample_data/train"
+  infer_load_path: "output_model_dnn"
+  infer_start_epoch: 0
+  infer_end_epoch: 3
+
+  # distribute_config
+  sync_mode: "async"
+  split_file_list: False
+  thread_num: 1
+
+
+# hyper parameters of user-defined network
+hyper_parameters:
+  # optimizer config
+  optimizer:
+    class: Adam
+    learning_rate: 0.001
+    strategy: async
+  # user-defined <key, value> pairs
+  sparse_inputs_slots: 27
+  sparse_feature_number: 1000001
+  sparse_feature_dim: 9
+  dense_input_dim: 13
+  fc_sizes: [512, 256, 128, 32]
+  distributed_embedding: 0

recommend-model-produce/src/main/python/models/dnn/criteo_reader.py

@@ -0,0 +1,81 @@
+#   Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from __future__ import print_function
+import numpy as np
+
+from paddle.io import IterableDataset
+
+
+class RecDataset(IterableDataset):
+    def __init__(self, file_list, config):
+        super(RecDataset, self).__init__()
+        self.file_list = file_list
+        self.init()
+
+    def init(self):
+        from operator import mul
+        padding = 0
+        sparse_slots = "click 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26"
+        self.sparse_slots = sparse_slots.strip().split(" ")
+        self.dense_slots = ["dense_feature"]
+        self.dense_slots_shape = [13]
+        self.slots = self.sparse_slots + self.dense_slots
+        self.slot2index = {}
+        self.visit = {}
+        for i in range(len(self.slots)):
+            self.slot2index[self.slots[i]] = i
+            self.visit[self.slots[i]] = False
+        self.padding = padding
+
+    def __iter__(self):
+        full_lines = []
+        self.data = []
+        for file in self.file_list:
+            with open(file, "r") as rf:
+                for l in rf:
+                    line = l.strip().split(" ")
+                    output = [(i, []) for i in self.slots]
+                    for i in line:
+                        slot_feasign = i.split(":")
+                        slot = slot_feasign[0]
+                        if slot not in self.slots:
+                            continue
+                        if slot in self.sparse_slots:
+                            feasign = int(slot_feasign[1])
+                        else:
+                            feasign = float(slot_feasign[1])
+                        output[self.slot2index[slot]][1].append(feasign)
+                        self.visit[slot] = True
+                    for i in self.visit:
+                        slot = i
+                        if not self.visit[slot]:
+                            if i in self.dense_slots:
+                                output[self.slot2index[i]][1].extend(
+                                    [self.padding] *
+                                    self.dense_slots_shape[self.slot2index[i]])
+                            else:
+                                output[self.slot2index[i]][1].extend(
+                                    [self.padding])
+                        else:
+                            self.visit[slot] = False
+                    # sparse
+                    output_list = []
+                    for key, value in output[:-1]:
+                        output_list.append(np.array(value).astype('int64'))
+                    # dense
+                    output_list.append(
+                        np.array(output[-1][1]).astype("float32"))
+                    # list
+                    yield output_list

recommend-model-produce/src/main/python/models/dnn/data/sample_data/train/sample_train.txt

@@ -0,0 +1,80 @@
+click:0 dense_feature:0.0 dense_feature:0.00497512437811 dense_feature:0.05 dense_feature:0.08 dense_feature:0.207421875 dense_feature:0.028 dense_feature:0.35 dense_feature:0.08 dense_feature:0.082 dense_feature:0.0 dense_feature:0.4 dense_feature:0.0 dense_feature:0.08 1:737395 2:210498 3:903564 4:286224 5:286835 6:906818 7:906116 8:67180 9:27346 10:51086 11:142177 12:95024 13:157883 14:873363 15:600281 16:812592 17:228085 18:35900 19:880474 20:984402 21:100885 22:26235 23:410878 24:798162 25:499868 26:306163
+click:1 dense_feature:0.0 dense_feature:0.932006633499 dense_feature:0.02 dense_feature:0.14 dense_feature:0.0395625 dense_feature:0.328 dense_feature:0.98 dense_feature:0.12 dense_feature:1.886 dense_feature:0.0 dense_feature:1.8 dense_feature:0.0 dense_feature:0.14 1:715353 2:761523 3:432904 4:892267 5:515218 6:948614 7:266726 8:67180 9:27346 10:266081 11:286126 12:789480 13:49621 14:255651 15:47663 16:79797 17:342789 18:616331 19:880474 20:984402 21:242209 22:26235 23:669531 24:26284 25:269955 26:187951
+click:0 dense_feature:0.0 dense_feature:0.00829187396352 dense_feature:0.08 dense_feature:0.06 dense_feature:0.14125 dense_feature:0.076 dense_feature:0.05 dense_feature:0.22 dense_feature:0.208 dense_feature:0.0 dense_feature:0.2 dense_feature:0.0 dense_feature:0.06 1:737395 2:952384 3:511141 4:271077 5:286835 6:948614 7:903547 8:507110 9:27346 10:56047 11:612953 12:747707 13:977426 14:671506 15:158148 16:833738 17:342789 18:427155 19:880474 20:537425 21:916237 22:26235 23:468277 24:676936 25:751788 26:363967
+click:0 dense_feature:0.0 dense_feature:0.124378109453 dense_feature:0.02 dense_feature:0.04 dense_feature:0.0 dense_feature:0.0 dense_feature:0.0 dense_feature:0.08 dense_feature:0.024 dense_feature:0.0 dense_feature:0.0 dense_feature:0.0 dense_feature:0.04 1:210127 2:286436 3:183920 4:507656 5:286835 6:906818 7:199553 8:67180 9:502607 10:708281 11:809876 12:888238 13:375164 14:202774 15:459895 16:475933 17:555571 18:847163 19:26230 20:26229 21:808836 22:191474 23:410878 24:315120 25:26224 26:26223
+click:0 dense_feature:0.1 dense_feature:0.0149253731343 dense_feature:0.34 dense_feature:0.32 dense_feature:0.016421875 dense_feature:0.098 dense_feature:0.04 dense_feature:0.96 dense_feature:0.202 dense_feature:0.1 dense_feature:0.2 dense_feature:0.0 dense_feature:0.32 1:230803 2:817085 3:539110 4:388629 5:286835 6:948614 7:586040 8:67180 9:27346 10:271155 11:176640 12:827381 13:36881 14:202774 15:397299 16:411672 17:342789 18:474060 19:880474 20:984402 21:216871 22:26235 23:761351 24:787115 25:884722 26:904135
+click:0 dense_feature:0.0 dense_feature:0.00829187396352 dense_feature:0.13 dense_feature:0.04 dense_feature:0.246203125 dense_feature:0.108 dense_feature:0.05 dense_feature:0.04 dense_feature:0.03 dense_feature:0.0 dense_feature:0.1 dense_feature:0.0 dense_feature:0.04 1:737395 2:64837 3:259267 4:336976 5:515218 6:154084 7:847938 8:67180 9:27346 10:708281 11:776766 12:964800 13:324323 14:873363 15:212708 16:637238 17:681378 18:895034 19:673458 20:984402 21:18600 22:26235 23:410878 24:787115 25:884722 26:355412
+click:0 dense_feature:0.0 dense_feature:0.028192371476 dense_feature:0.0 dense_feature:0.0 dense_feature:0.0245625 dense_feature:0.016 dense_feature:0.04 dense_feature:0.12 dense_feature:0.016 dense_feature:0.0 dense_feature:0.1 dense_feature:0.0 dense_feature:0.0 1:737395 2:554760 3:661483 4:263696 5:938478 6:906818 7:786926 8:67180 9:27346 10:245862 11:668197 12:745676 13:432600 14:413795 15:751427 16:272410 17:342789 18:422136 19:26230 20:26229 21:452501 22:26235 23:51381 24:776636 25:26224 26:26223
+click:0 dense_feature:0.0 dense_feature:0.00497512437811 dense_feature:1.95 dense_feature:0.28 dense_feature:0.092828125 dense_feature:0.57 dense_feature:0.06 dense_feature:0.4 dense_feature:0.4 dense_feature:0.0 dense_feature:0.2 dense_feature:0.0 dense_feature:0.4 1:371155 2:817085 3:773609 4:555449 5:938478 6:906818 7:166117 8:507110 9:27346 10:545822 11:316654 12:172765 13:989600 14:255651 15:792372 16:606361 17:342789 18:566554 19:880474 20:984402 21:235256 22:191474 23:700326 24:787115 25:884722 26:569095
+click:0 dense_feature:0.0 dense_feature:0.0912106135987 dense_feature:0.01 dense_feature:0.02 dense_feature:0.06625 dense_feature:0.018 dense_feature:0.05 dense_feature:0.06 dense_feature:0.098 dense_feature:0.0 dense_feature:0.4 dense_feature:0.0 dense_feature:0.04 1:230803 2:531472 3:284417 4:661677 5:938478 6:553107 7:21150 8:49466 9:27346 10:526914 11:164508 12:631773 13:882348 14:873363 15:523948 16:687081 17:342789 18:271301 19:26230 20:26229 21:647160 22:26235 23:410878 24:231695 25:26224 26:26223
+click:1 dense_feature:0.0 dense_feature:0.00663349917081 dense_feature:0.01 dense_feature:0.02 dense_feature:0.02153125 dense_feature:0.092 dense_feature:0.05 dense_feature:0.68 dense_feature:0.472 dense_feature:0.0 dense_feature:0.3 dense_feature:0.0 dense_feature:0.02 1:737395 2:532829 3:320762 4:887282 5:286835 6:25207 7:640357 8:67180 9:27346 10:695831 11:739268 12:835325 13:402539 14:873363 15:125813 16:168896 17:342789 18:374414 19:26230 20:26229 21:850229 22:26235 23:410878 24:480027 25:26224 26:26223
+click:0 dense_feature:0.0 dense_feature:0.00497512437811 dense_feature:0.05 dense_feature:0.04 dense_feature:0.086125 dense_feature:0.098 dense_feature:0.15 dense_feature:0.06 dense_feature:0.228 dense_feature:0.0 dense_feature:0.2 dense_feature:0.0 dense_feature:0.04 1:210127 2:999497 3:646348 4:520638 5:938478 6:906818 7:438398 8:67180 9:27346 10:975902 11:532544 12:708828 13:815045 14:255651 15:896230 16:663630 17:342789 18:820094 19:687226 20:537425 21:481536 22:26235 23:761351 24:888170 25:250729 26:381125
+click:1 dense_feature:0.1 dense_feature:0.00331674958541 dense_feature:0.02 dense_feature:0.02 dense_feature:0.00078125 dense_feature:0.002 dense_feature:0.73 dense_feature:0.08 dense_feature:0.254 dense_feature:0.1 dense_feature:1.4 dense_feature:0.0 dense_feature:0.02 1:715353 2:342833 3:551901 4:73418 5:286835 6:446063 7:219517 8:67180 9:27346 10:668726 11:40711 12:921745 13:361076 14:15048 15:214564 16:400893 17:228085 18:393370 19:26230 20:26229 21:383046 22:26235 23:700326 24:369764 25:26224 26:26223
+click:0 dense_feature:0.0 dense_feature:0.142620232172 dense_feature:0.04 dense_feature:0.1 dense_feature:0.08853125 dense_feature:0.028 dense_feature:0.01 dense_feature:0.1 dense_feature:0.028 dense_feature:0.0 dense_feature:0.1 dense_feature:0.0 dense_feature:0.1 1:737395 2:583707 3:519411 4:19103 5:286835 6:906818 7:801403 8:67180 9:27346 10:35743 11:626052 12:142351 13:988058 14:873363 15:617333 16:850339 17:276641 18:696084 19:26230 20:26229 21:121620 22:191474 23:468277 24:18340 25:26224 26:26223
+click:0 dense_feature:0.0 dense_feature:0.00995024875622 dense_feature:0.0 dense_feature:0.22 dense_feature:0.0251875 dense_feature:0.0 dense_feature:0.0 dense_feature:0.8 dense_feature:0.182 dense_feature:0.0 dense_feature:0.0 dense_feature:0.0 dense_feature:0.84 1:737395 2:19359 3:166075 4:381832 5:286835 6:446063 7:816009 8:67180 9:27346 10:708281 11:619790 12:524128 13:826787 14:202774 15:371495 16:392894 17:644532 18:271180 19:26230 20:26229 21:349978 22:26235 23:761351 24:517170 25:26224 26:26223
+click:0 dense_feature:0.0 dense_feature:0.0149253731343 dense_feature:0.52 dense_feature:0.1 dense_feature:6.25153125 dense_feature:0.0 dense_feature:0.0 dense_feature:0.3 dense_feature:0.03 dense_feature:0.0 dense_feature:0.0 dense_feature:0.0 dense_feature:0.1 1:230803 2:24784 3:519411 4:19103 5:843054 6:948614 7:529143 8:67180 9:502607 10:708281 11:430027 12:142351 13:529101 14:202774 15:618316 16:850339 17:644532 18:95370 19:880474 20:31181 21:121620 22:26235 23:744389 24:18340 25:269955 26:683431
+click:0 dense_feature:0.0 dense_feature:0.0480928689884 dense_feature:0.12 dense_feature:0.22 dense_feature:0.541703125 dense_feature:1.062 dense_feature:0.01 dense_feature:0.24 dense_feature:0.054 dense_feature:0.0 dense_feature:0.1 dense_feature:0.0 dense_feature:0.22 1:737395 2:378661 3:21539 4:552097 5:286835 6:553107 7:512138 8:67180 9:27346 10:708281 11:91094 12:516991 13:150114 14:873363 15:450569 16:353024 17:228085 18:539379 19:26230 20:26229 21:410733 22:26235 23:700326 24:272703 25:26224 26:26223
+click:0 dense_feature:0.0 dense_feature:0.016583747927 dense_feature:0.06 dense_feature:0.0 dense_feature:0.209625 dense_feature:0.0 dense_feature:0.0 dense_feature:0.0 dense_feature:0.09 dense_feature:0.0 dense_feature:0.0 dense_feature:0.0 dense_feature:0.0 1:737395 2:750131 3:807749 4:905739 5:286835 6:906818 7:11935 8:67180 9:27346 10:708281 11:505199 12:285350 13:724106 14:255651 15:625913 16:511836 17:644532 18:102288 19:26230 20:26229 21:726818 22:179327 23:744389 24:176417 25:26224 26:26223
+click:0 dense_feature:0.0 dense_feature:0.00663349917081 dense_feature:0.05 dense_feature:0.14 dense_feature:0.226703125 dense_feature:0.12 dense_feature:0.05 dense_feature:0.14 dense_feature:0.112 dense_feature:0.0 dense_feature:0.1 dense_feature:0.0 dense_feature:0.14 1:736218 2:690313 3:757279 4:763330 5:286835 6:553107 7:89560 8:642551 9:27346 10:128328 11:281593 12:246510 13:200341 14:255651 15:899145 16:807138 17:342789 18:659853 19:26230 20:26229 21:399608 22:26235 23:669531 24:787115 25:26224 26:26223
+click:0 dense_feature:0.0 dense_feature:0.00829187396352 dense_feature:0.3 dense_feature:0.2 dense_feature:0.021296875 dense_feature:0.83 dense_feature:0.2 dense_feature:0.56 dense_feature:1.122 dense_feature:0.0 dense_feature:0.5 dense_feature:0.0 dense_feature:0.2 1:715353 2:283434 3:523722 4:590869 5:286835 6:948614 7:25472 8:67180 9:27346 10:340404 11:811342 12:679454 13:897590 14:813514 15:578769 16:962576 17:342789 18:267210 19:310188 20:537425 21:746185 22:179327 23:761351 24:416923 25:253255 26:249672
+click:1 dense_feature:0.05 dense_feature:0.0149253731343 dense_feature:0.03 dense_feature:0.24 dense_feature:0.0 dense_feature:0.008 dense_feature:0.4 dense_feature:0.62 dense_feature:0.82 dense_feature:0.1 dense_feature:1.4 dense_feature:0.0 dense_feature:0.08 1:715353 2:532829 3:716475 4:940968 5:286835 6:948614 7:38171 8:67180 9:27346 10:619455 11:515541 12:779426 13:711791 14:255651 15:881750 16:408550 17:342789 18:612540 19:26230 20:26229 21:23444 22:26235 23:410878 24:88425 25:26224 26:26223
+click:0 dense_feature:0.0 dense_feature:0.00497512437811 dense_feature:0.11 dense_feature:0.08 dense_feature:0.135265625 dense_feature:0.426 dense_feature:0.06 dense_feature:0.06 dense_feature:0.42 dense_feature:0.0 dense_feature:0.1 dense_feature:0.0 dense_feature:0.08 1:737395 2:817085 3:506158 4:48876 5:286835 6:948614 7:95506 8:67180 9:27346 10:75825 11:220591 12:613471 13:159874 14:255651 15:121379 16:889290 17:681378 18:532453 19:880474 20:537425 21:717912 22:26235 23:270873 24:450199 25:884722 26:382723
+click:0 dense_feature:0.0 dense_feature:0.0829187396352 dense_feature:0.0 dense_feature:0.0 dense_feature:0.555859375 dense_feature:0.318 dense_feature:0.03 dense_feature:0.0 dense_feature:0.02 dense_feature:0.0 dense_feature:0.1 dense_feature:0.0 dense_feature:0.0 1:715353 2:465222 3:974451 4:892661 5:938478 6:948614 7:651987 8:67180 9:27346 10:708281 11:229311 12:545057 13:875629 14:149134 15:393524 16:213237 17:681378 18:540092 19:26230 20:26229 21:483290 22:26235 23:700326 24:946673 25:26224 26:26223
+click:1 dense_feature:0.05 dense_feature:0.854063018242 dense_feature:0.01 dense_feature:0.04 dense_feature:0.000171875 dense_feature:0.004 dense_feature:0.01 dense_feature:0.04 dense_feature:0.004 dense_feature:0.1 dense_feature:0.1 dense_feature:0.0 dense_feature:0.04 1:737395 2:99294 3:681584 4:398205 5:914075 6:906818 7:620358 8:67180 9:27346 10:147441 11:364583 12:535262 13:516341 14:813514 15:281303 16:714384 17:276641 18:443922 19:26230 20:26229 21:948746 22:26235 23:700326 24:928903 25:26224 26:26223
+click:0 dense_feature:0.0 dense_feature:0.00331674958541 dense_feature:0.0 dense_feature:0.0 dense_feature:0.45190625 dense_feature:0.048 dense_feature:0.01 dense_feature:0.16 dense_feature:0.044 dense_feature:0.0 dense_feature:0.1 dense_feature:0.0 dense_feature:0.0 1:737395 2:792512 3:676584 4:995262 5:938478 6:906818 7:888723 8:67180 9:27346 10:708281 11:310529 12:951172 13:885793 14:873363 15:62698 16:672021 17:276641 18:11502 19:880474 20:984402 21:501083 22:191474 23:744389 24:398029 25:218743 26:991064
+click:0 dense_feature:0.0 dense_feature:0.00663349917081 dense_feature:0.51 dense_feature:0.0 dense_feature:0.2689375 dense_feature:0.0 dense_feature:0.0 dense_feature:0.02 dense_feature:0.006 dense_feature:0.0 dense_feature:0.0 dense_feature:0.0 dense_feature:0.0 1:230803 2:239052 3:323170 4:474182 5:140103 6:553107 7:757837 8:524745 9:27346 10:743444 11:883533 12:123023 13:621127 14:255651 15:570872 16:883618 17:924903 18:984920 19:964183 20:984402 21:260134 22:179327 23:410878 24:787860 25:269955 26:949924
+click:0 dense_feature:0.0 dense_feature:0.273631840796 dense_feature:0.0 dense_feature:0.0 dense_feature:0.066453125 dense_feature:0.052 dense_feature:0.04 dense_feature:0.06 dense_feature:0.01 dense_feature:0.0 dense_feature:0.1 dense_feature:0.0 dense_feature:0.0 1:737395 2:531472 3:747313 4:362684 5:843054 6:553107 7:863980 8:718499 9:27346 10:881217 11:371751 12:168971 13:290788 14:202774 15:316669 16:269663 17:342789 18:136775 19:26230 20:26229 21:76865 22:26235 23:761351 24:441421 25:26224 26:26223
+click:0 dense_feature:0.0 dense_feature:0.116086235489 dense_feature:0.43 dense_feature:0.36 dense_feature:0.000953125 dense_feature:0.0 dense_feature:0.0 dense_feature:0.36 dense_feature:0.036 dense_feature:0.0 dense_feature:0.0 dense_feature:0.0 dense_feature:0.36 1:737395 2:24784 3:677469 4:820784 5:286835 6:553107 7:715520 8:718499 9:27346 10:708281 11:670424 12:122926 13:724619 14:873363 15:845517 16:488791 17:644532 18:183573 19:880474 20:31181 21:46761 22:26235 23:700326 24:629361 25:269955 26:862373
+click:0 dense_feature:2.55 dense_feature:0.0348258706468 dense_feature:0.01 dense_feature:0.38 dense_feature:0.001453125 dense_feature:0.046 dense_feature:1.11 dense_feature:0.44 dense_feature:2.312 dense_feature:0.2 dense_feature:1.1 dense_feature:0.0 dense_feature:0.46 1:594517 2:194636 3:496284 4:323209 5:286835 6:553107 7:259696 8:760861 9:27346 10:698046 11:478868 12:576074 13:635369 14:201966 15:926692 16:972906 17:342789 18:409802 19:26230 20:26229 21:395694 22:26235 23:410878 24:844671 25:26224 26:26223
+click:0 dense_feature:0.0 dense_feature:0.144278606965 dense_feature:0.43 dense_feature:0.22 dense_feature:0.00309375 dense_feature:0.15 dense_feature:0.14 dense_feature:0.54 dense_feature:0.152 dense_feature:0.0 dense_feature:0.2 dense_feature:0.1 dense_feature:0.22 1:737395 2:239052 3:456744 4:736474 5:286835 6:948614 7:13277 8:67180 9:27346 10:958384 11:778183 12:497627 13:136915 14:201966 15:757961 16:747483 17:228085 18:984920 19:905920 20:537425 21:472149 22:179327 23:410878 24:709155 25:269955 26:618673
+click:0 dense_feature:0.0 dense_feature:0.0132669983416 dense_feature:0.4 dense_feature:0.3 dense_feature:0.36440625 dense_feature:1.492 dense_feature:0.07 dense_feature:0.3 dense_feature:1.048 dense_feature:0.0 dense_feature:0.3 dense_feature:0.0 dense_feature:0.3 1:737395 2:19959 3:661391 4:748753 5:286835 6:948614 7:848540 8:67180 9:27346 10:708281 11:703964 12:72024 13:336272 14:255651 15:835686 16:703858 17:342789 18:274368 19:26230 20:26229 21:765452 22:26235 23:700326 24:815200 25:26224 26:26223
+click:0 dense_feature:0.0 dense_feature:0.0116086235489 dense_feature:0.01 dense_feature:0.0 dense_feature:0.0 dense_feature:0.0 dense_feature:0.0 dense_feature:0.0 dense_feature:0.0 dense_feature:0.0 dense_feature:0.0 dense_feature:0.0 dense_feature:0.0 1:210127 2:662691 3:334228 4:857003 5:286835 6:25207 7:280499 8:67180 9:502607 10:708281 11:195094 12:870026 13:783566 14:873363 15:139595 16:214259 17:555571 18:208248 19:880474 20:984402 21:471770 22:26235 23:744389 24:507551 25:383787 26:797121
+click:1 dense_feature:0.0 dense_feature:0.0348258706468 dense_feature:0.03 dense_feature:0.02 dense_feature:0.066140625 dense_feature:0.006 dense_feature:0.17 dense_feature:0.02 dense_feature:0.236 dense_feature:0.0 dense_feature:0.5 dense_feature:0.0 dense_feature:0.02 1:230803 2:999497 3:25361 4:892267 5:286835 6:906818 7:356528 8:67180 9:27346 10:5856 11:157692 12:554754 13:442501 14:255651 15:896230 16:248781 17:342789 18:820094 19:905920 20:984402 21:916436 22:26235 23:669531 24:26284 25:884722 26:187951
+click:0 dense_feature:0.0 dense_feature:4.62852404643 dense_feature:0.07 dense_feature:0.0 dense_feature:0.022671875 dense_feature:0.0 dense_feature:0.01 dense_feature:0.0 dense_feature:0.0 dense_feature:0.0 dense_feature:0.1 dense_feature:0.0 dense_feature:0.0 1:624252 2:344887 3:238747 4:308366 5:286835 6:553107 7:69291 8:67180 9:27346 10:781054 11:258240 12:546906 13:772337 14:873363 15:807640 16:525695 17:276641 18:613203 19:438655 20:984402 21:415123 22:191474 23:700326 24:729290 25:218743 26:953507
+click:0 dense_feature:0.0 dense_feature:0.00663349917081 dense_feature:0.06 dense_feature:0.02 dense_feature:0.06878125 dense_feature:0.044 dense_feature:0.01 dense_feature:0.22 dense_feature:0.044 dense_feature:0.0 dense_feature:0.1 dense_feature:0.0 dense_feature:0.02 1:737395 2:7753 3:871178 4:183530 5:286835 6:906818 7:273988 8:507110 9:27346 10:708281 11:942072 12:775997 13:612590 14:873363 15:669921 16:639940 17:681378 18:421122 19:880474 20:984402 21:410471 22:26235 23:410878 24:228420 25:269955 26:616000
+click:0 dense_feature:0.0 dense_feature:0.212271973466 dense_feature:0.02 dense_feature:0.28 dense_feature:0.113421875 dense_feature:0.06 dense_feature:0.02 dense_feature:0.28 dense_feature:0.194 dense_feature:0.0 dense_feature:0.2 dense_feature:0.0 dense_feature:0.28 1:210127 2:228963 3:692240 4:389834 5:938478 6:948614 7:125690 8:507110 9:27346 10:708281 11:549232 12:308284 13:262461 14:255651 15:629185 16:280660 17:276641 18:886164 19:26230 20:26229 21:367919 22:191474 23:700326 24:520083 25:26224 26:26223
+click:0 dense_feature:0.0 dense_feature:1.01658374793 dense_feature:0.01 dense_feature:0.02 dense_feature:0.11759375 dense_feature:0.08 dense_feature:0.02 dense_feature:0.02 dense_feature:0.024 dense_feature:0.0 dense_feature:0.1 dense_feature:0.0 dense_feature:0.02 1:230803 2:7753 3:194720 4:831884 5:286835 6:553107 7:620358 8:67180 9:27346 10:843010 11:424144 12:615986 13:516341 14:813514 15:782575 16:775856 17:342789 18:421122 19:880474 20:984402 21:110090 22:191474 23:700326 24:784174 25:269955 26:101161
+click:0 dense_feature:0.0 dense_feature:0.00663349917081 dense_feature:0.59 dense_feature:0.06 dense_feature:0.04321875 dense_feature:0.192 dense_feature:0.02 dense_feature:0.08 dense_feature:0.014 dense_feature:0.0 dense_feature:0.1 dense_feature:0.0 dense_feature:0.06 1:230803 2:532829 3:26258 4:853241 5:938478 6:948614 7:877607 8:67180 9:27346 10:613723 11:246387 12:538673 13:377975 14:873363 15:659013 16:601478 17:681378 18:199271 19:26230 20:26229 21:300137 22:26235 23:410878 24:372458 25:26224 26:26223
+click:0 dense_feature:0.0 dense_feature:0.06135986733 dense_feature:0.0 dense_feature:0.0 dense_feature:0.294671875 dense_feature:0.212 dense_feature:0.26 dense_feature:0.0 dense_feature:0.034 dense_feature:0.0 dense_feature:0.1 dense_feature:0.0 dense_feature:0.0 1:737395 2:154478 3:982044 4:501457 5:819883 6:906818 7:445051 8:67180 9:27346 10:976970 11:783630 12:609883 13:358461 14:15048 15:409791 16:756307 17:342789 18:480228 19:26230 20:26229 21:845147 22:26235 23:669531 24:124290 25:26224 26:26223
+click:1 dense_feature:0.05 dense_feature:0.537313432836 dense_feature:0.0 dense_feature:0.02 dense_feature:0.018578125 dense_feature:0.016 dense_feature:0.16 dense_feature:0.22 dense_feature:0.192 dense_feature:0.1 dense_feature:0.3 dense_feature:0.0 dense_feature:0.02 1:737395 2:194636 3:274597 4:418981 5:286835 6:553107 7:553528 8:67180 9:27346 10:901359 11:110700 12:108037 13:915461 14:255651 15:951604 16:421384 17:342789 18:728110 19:26230 20:26229 21:772733 22:191474 23:761351 24:844671 25:26224 26:26223
+click:0 dense_feature:0.1 dense_feature:0.00663349917081 dense_feature:0.16 dense_feature:0.26 dense_feature:0.00509375 dense_feature:0.122 dense_feature:0.03 dense_feature:0.94 dense_feature:0.526 dense_feature:0.1 dense_feature:0.1 dense_feature:0.0 dense_feature:1.1 1:210127 2:344887 3:343793 4:917598 5:286835 6:948614 7:220413 8:67180 9:27346 10:912799 11:370606 12:722621 13:569604 14:255651 15:499545 16:159495 17:342789 18:613203 19:305384 20:984402 21:844602 22:26235 23:410878 24:695516 25:218743 26:729263
+click:0 dense_feature:0.0 dense_feature:0.00497512437811 dense_feature:0.09 dense_feature:0.16 dense_feature:0.11221875 dense_feature:0.51 dense_feature:0.09 dense_feature:0.48 dense_feature:0.088 dense_feature:0.0 dense_feature:0.4 dense_feature:0.0 dense_feature:0.16 1:737395 2:532829 3:579624 4:980109 5:286835 6:948614 7:927736 8:67180 9:27346 10:970644 11:931289 12:377125 13:539272 14:873363 15:555779 16:405069 17:342789 18:701770 19:26230 20:26229 21:201088 22:26235 23:410878 24:113994 25:26224 26:26223
+click:0 dense_feature:0.0 dense_feature:0.182421227197 dense_feature:0.01 dense_feature:0.02 dense_feature:0.000109375 dense_feature:0.978 dense_feature:0.01 dense_feature:0.02 dense_feature:0.062 dense_feature:0.0 dense_feature:0.1 dense_feature:0.0 dense_feature:0.02 1:478318 2:158107 3:508317 4:452336 5:286835 6:948614 7:620358 8:67180 9:27346 10:147441 11:364583 12:34025 13:516341 14:873363 15:502825 16:683439 17:681378 18:889198 19:26230 20:26229 21:234451 22:26235 23:700326 24:256238 25:26224 26:26223
+click:0 dense_feature:0.0 dense_feature:0.469320066335 dense_feature:0.2 dense_feature:0.2 dense_feature:0.0705 dense_feature:0.102 dense_feature:0.05 dense_feature:0.22 dense_feature:0.194 dense_feature:0.0 dense_feature:0.1 dense_feature:0.0 dense_feature:0.2 1:715353 2:846239 3:573061 4:508181 5:286835 6:553107 7:892443 8:718499 9:27346 10:639370 11:866496 12:791636 13:895012 14:873363 15:362079 16:16082 17:228085 18:994402 19:880474 20:984402 21:35513 22:26235 23:669531 24:520197 25:934391 26:625657
+click:0 dense_feature:0.0 dense_feature:0.0729684908789 dense_feature:0.06 dense_feature:0.04 dense_feature:5.620296875 dense_feature:0.0 dense_feature:0.0 dense_feature:0.06 dense_feature:0.0 dense_feature:0.0 dense_feature:0.0 dense_feature:0.0 dense_feature:0.04 1:399845 2:239052 3:334610 4:593315 5:286835 6:948614 7:751495 8:67180 9:502607 10:111048 11:244081 12:115252 13:915518 14:873363 15:817451 16:296052 17:276641 18:984920 19:774721 20:984402 21:930636 22:26235 23:700326 24:975048 25:269955 26:266439
+click:1 dense_feature:0.05 dense_feature:0.0265339966833 dense_feature:0.07 dense_feature:0.22 dense_feature:1.5625e-05 dense_feature:0.008 dense_feature:0.04 dense_feature:0.36 dense_feature:0.088 dense_feature:0.1 dense_feature:0.3 dense_feature:0.0 dense_feature:0.08 1:737395 2:64837 3:534435 4:555449 5:286835 6:25207 7:661236 8:67180 9:27346 10:708281 11:785752 12:47348 13:524553 14:117289 15:776971 16:293528 17:681378 18:102169 19:758208 20:31181 21:27506 22:26235 23:410878 24:787115 25:884722 26:605635
+click:1 dense_feature:0.1 dense_feature:0.0464344941957 dense_feature:0.0 dense_feature:0.04 dense_feature:0.00059375 dense_feature:0.004 dense_feature:0.02 dense_feature:0.04 dense_feature:0.004 dense_feature:0.1 dense_feature:0.1 dense_feature:0.0 dense_feature:0.04 1:230803 2:7753 3:529866 4:437169 5:938478 6:948614 7:17274 8:67180 9:27346 10:461781 11:452641 12:302471 13:49621 14:873363 15:543432 16:858509 17:681378 18:402164 19:880474 20:984402 21:650184 22:191474 23:410878 24:492581 25:269955 26:217228
+click:0 dense_feature:0.55 dense_feature:0.00829187396352 dense_feature:0.03 dense_feature:0.0 dense_feature:0.0014375 dense_feature:0.004 dense_feature:0.36 dense_feature:0.0 dense_feature:0.042 dense_feature:0.1 dense_feature:0.4 dense_feature:0.0 dense_feature:0.0 1:26973 2:817085 3:961160 4:355882 5:843054 6:906818 7:417593 8:67180 9:27346 10:708281 11:402889 12:899379 13:552051 14:202774 15:532679 16:545549 17:342789 18:562805 19:880474 20:31181 21:355920 22:26235 23:700326 24:787115 25:884722 26:115004
+click:1 dense_feature:0.0 dense_feature:0.00663349917081 dense_feature:0.01 dense_feature:0.02 dense_feature:0.089296875 dense_feature:0.362 dense_feature:0.23 dense_feature:0.04 dense_feature:0.338 dense_feature:0.0 dense_feature:0.4 dense_feature:0.0 dense_feature:0.02 1:230803 2:977337 3:853759 4:880273 5:515218 6:25207 7:414263 8:437731 9:27346 10:205124 11:108170 12:676869 13:388798 14:255651 15:247232 16:172895 17:228085 18:543219 19:26230 20:26229 21:860937 22:179327 23:669531 24:959959 25:26224 26:26223
+click:0 dense_feature:0.0 dense_feature:0.0945273631841 dense_feature:0.62 dense_feature:0.24 dense_feature:0.11840625 dense_feature:0.368 dense_feature:0.07 dense_feature:0.24 dense_feature:0.144 dense_feature:0.0 dense_feature:0.1 dense_feature:0.0 dense_feature:0.48 1:737395 2:532829 3:805087 4:186661 5:286835 6:154084 7:468059 8:718499 9:27346 10:708281 11:968875 12:8177 13:47822 14:255651 15:979316 16:956543 17:342789 18:541633 19:26230 20:26229 21:646669 22:26235 23:410878 24:184909 25:26224 26:26223
+click:0 dense_feature:0.3 dense_feature:0.00497512437811 dense_feature:0.12 dense_feature:0.12 dense_feature:0.002890625 dense_feature:0.074 dense_feature:0.06 dense_feature:0.14 dense_feature:0.074 dense_feature:0.1 dense_feature:0.1 dense_feature:0.0 dense_feature:0.74 1:737395 2:64837 3:967865 4:249418 5:938478 6:948614 7:228716 8:67180 9:27346 10:627362 11:722606 12:193782 13:348283 14:255651 15:928582 16:221557 17:342789 18:895034 19:384556 20:984402 21:475712 22:26235 23:410878 24:492875 25:884722 26:468964
+click:0 dense_feature:0.0 dense_feature:0.177446102819 dense_feature:0.01 dense_feature:0.02 dense_feature:0.041859375 dense_feature:0.0 dense_feature:0.0 dense_feature:0.16 dense_feature:0.036 dense_feature:0.0 dense_feature:0.0 dense_feature:0.0 dense_feature:0.02 1:154064 2:834620 3:25206 4:25205 5:938478 6:948614 7:134101 8:92608 9:27346 10:708281 11:505199 12:25711 13:724106 14:671506 15:42927 16:25723 17:644532 18:1957 19:26230 20:26229 21:26236 22:26235 23:744389 24:26233 25:26224 26:26223
+click:0 dense_feature:0.0 dense_feature:5.61691542289 dense_feature:0.0 dense_feature:0.1 dense_feature:0.043796875 dense_feature:0.302 dense_feature:0.13 dense_feature:0.22 dense_feature:0.3 dense_feature:0.0 dense_feature:0.1 dense_feature:0.0 dense_feature:0.22 1:154184 2:19359 3:166075 4:381832 5:286835 6:906818 7:348227 8:49466 9:27346 10:645596 11:951584 12:524128 13:277250 14:255651 15:853732 16:392894 17:342789 18:619939 19:26230 20:26229 21:349978 22:26235 23:700326 24:517170 25:26224 26:26223
+click:1 dense_feature:0.0 dense_feature:0.00331674958541 dense_feature:0.0 dense_feature:0.0 dense_feature:0.093234375 dense_feature:0.022 dense_feature:0.04 dense_feature:0.02 dense_feature:0.02 dense_feature:0.0 dense_feature:0.2 dense_feature:0.0 dense_feature:0.0 1:715353 2:485136 3:386313 4:208181 5:286835 6:25207 7:227715 8:49466 9:27346 10:437476 11:733250 12:721260 13:389832 14:255651 15:47178 16:761962 17:342789 18:813169 19:26230 20:26229 21:464938 22:26235 23:410878 24:833196 25:26224 26:26223
+click:0 dense_feature:0.0 dense_feature:0.134328358209 dense_feature:0.0 dense_feature:0.14 dense_feature:0.00015625 dense_feature:0.0 dense_feature:0.0 dense_feature:0.14 dense_feature:0.014 dense_feature:0.0 dense_feature:0.0 dense_feature:0.0 dense_feature:0.14 1:737395 2:488655 3:221719 4:442408 5:286835 6:25207 7:898902 8:718499 9:27346 10:457066 11:290973 12:533168 13:949027 14:873363 15:270294 16:934635 17:924903 18:763017 19:880474 20:31181 21:517486 22:26235 23:410878 24:588215 25:499868 26:980179
+click:1 dense_feature:0.0 dense_feature:0.00331674958541 dense_feature:0.0 dense_feature:0.0 dense_feature:0.023578125 dense_feature:0.0 dense_feature:0.04 dense_feature:0.0 dense_feature:0.046 dense_feature:0.0 dense_feature:0.3 dense_feature:0.0 dense_feature:0.0 1:737395 2:729012 3:691820 4:351286 5:938478 6:553107 7:21150 8:67180 9:27346 10:947459 11:164508 12:205079 13:882348 14:255651 15:178324 16:282716 17:342789 18:193902 19:880474 20:31181 21:604480 22:191474 23:669531 24:727223 25:499868 26:236426
+click:1 dense_feature:0.1 dense_feature:0.00331674958541 dense_feature:0.0 dense_feature:0.0 dense_feature:0.00859375 dense_feature:0.006 dense_feature:1.55 dense_feature:0.16 dense_feature:0.06 dense_feature:0.2 dense_feature:1.6 dense_feature:0.0 dense_feature:0.0 1:712372 2:235347 3:483718 4:382039 5:914075 6:906818 7:727609 8:154004 9:27346 10:116648 11:40711 12:658199 13:361076 14:15048 15:15058 16:644988 17:342789 18:544170 19:26230 20:26229 21:251535 22:26235 23:700326 24:114111 25:26224 26:26223
+click:1 dense_feature:0.25 dense_feature:0.192371475954 dense_feature:0.06 dense_feature:0.36 dense_feature:0.0 dense_feature:0.02 dense_feature:0.09 dense_feature:0.42 dense_feature:0.042 dense_feature:0.2 dense_feature:0.3 dense_feature:0.3 dense_feature:0.0 1:737395 2:288975 3:885137 4:368487 5:515218 6:906818 7:569753 8:799133 9:27346 10:635043 11:883202 12:780104 13:492605 14:873363 15:234451 16:94894 17:796504 18:653705 19:880474 20:984402 21:400692 22:26235 23:410878 24:767424 25:934391 26:958132
+click:1 dense_feature:0.15 dense_feature:0.0398009950249 dense_feature:0.02 dense_feature:0.04 dense_feature:1.5625e-05 dense_feature:0.0 dense_feature:0.06 dense_feature:0.04 dense_feature:0.026 dense_feature:0.1 dense_feature:0.3 dense_feature:0.0 dense_feature:0.0 1:715353 2:532829 3:721632 4:377785 5:286835 6:553107 7:959856 8:718499 9:27346 10:737746 11:432444 12:706936 13:169268 14:873363 15:896219 16:461005 17:342789 18:286597 19:26230 20:26229 21:602049 22:26235 23:700326 24:510447 25:26224 26:26223
+click:0 dense_feature:0.0 dense_feature:0.00663349917081 dense_feature:0.05 dense_feature:0.08 dense_feature:0.155421875 dense_feature:0.55 dense_feature:0.08 dense_feature:0.24 dense_feature:1.73 dense_feature:0.0 dense_feature:0.3 dense_feature:0.0 dense_feature:0.08 1:737395 2:288975 3:385122 4:57409 5:286835 6:25207 7:339181 8:67180 9:27346 10:284863 11:531306 12:229544 13:32168 14:117289 15:632422 16:615549 17:342789 18:240865 19:880474 20:984402 21:253725 22:26235 23:410878 24:837371 25:934391 26:948190
+click:0 dense_feature:0.0 dense_feature:0.0398009950249 dense_feature:0.06 dense_feature:0.12 dense_feature:0.11359375 dense_feature:0.55 dense_feature:0.03 dense_feature:0.12 dense_feature:0.186 dense_feature:0.0 dense_feature:0.2 dense_feature:0.0 dense_feature:0.12 1:737395 2:158107 3:738359 4:343895 5:286835 6:948614 7:513189 8:760861 9:27346 10:741641 11:214926 12:142871 13:753229 14:873363 15:502825 16:864586 17:681378 18:889198 19:26230 20:26229 21:368414 22:191474 23:410878 24:256238 25:26224 26:26223
+click:1 dense_feature:0.25 dense_feature:0.00663349917081 dense_feature:0.03 dense_feature:0.04 dense_feature:7.8125e-05 dense_feature:0.0 dense_feature:0.48 dense_feature:0.06 dense_feature:0.004 dense_feature:0.2 dense_feature:1.3 dense_feature:0.0 dense_feature:0.0 1:737395 2:414770 3:100889 4:981572 5:286835 6:446063 7:600430 8:507110 9:27346 10:566014 11:40711 12:330691 13:361076 14:15048 15:176957 16:759140 17:342789 18:212244 19:26230 20:26229 21:688637 22:26235 23:634287 24:762432 25:26224 26:26223
+click:0 dense_feature:0.0 dense_feature:0.00663349917081 dense_feature:0.04 dense_feature:0.02 dense_feature:0.109765625 dense_feature:0.202 dense_feature:0.13 dense_feature:0.02 dense_feature:0.078 dense_feature:0.0 dense_feature:0.1 dense_feature:0.1 dense_feature:0.02 1:737395 2:7753 3:871178 4:183530 5:286835 6:948614 7:358953 8:718499 9:27346 10:837400 11:432444 12:775997 13:169268 14:255651 15:250644 16:639940 17:342789 18:421122 19:880474 20:984402 21:410471 22:26235 23:410878 24:228420 25:269955 26:870795
+click:0 dense_feature:0.05 dense_feature:0.162520729685 dense_feature:0.28 dense_feature:0.16 dense_feature:0.001046875 dense_feature:0.028 dense_feature:1.03 dense_feature:0.84 dense_feature:0.534 dense_feature:0.1 dense_feature:2.3 dense_feature:0.0 dense_feature:0.28 1:737395 2:334074 3:108983 4:898979 5:286835 6:948614 7:600430 8:718499 9:27346 10:668726 11:40711 12:62821 13:361076 14:202774 15:722413 16:688170 17:342789 18:746785 19:957809 20:984402 21:96056 22:191474 23:410878 24:703372 25:129305 26:591537
+click:0 dense_feature:0.2 dense_feature:0.0945273631841 dense_feature:0.02 dense_feature:0.18 dense_feature:0.021078125 dense_feature:0.046 dense_feature:0.52 dense_feature:0.44 dense_feature:0.18 dense_feature:0.1 dense_feature:0.8 dense_feature:0.0 dense_feature:0.22 1:663372 2:532829 3:714247 4:673800 5:286835 6:906818 7:219517 8:67180 9:27346 10:161916 11:40711 12:441505 13:361076 14:255651 15:992961 16:137571 17:796504 18:395194 19:26230 20:26229 21:800938 22:179327 23:410878 24:719782 25:26224 26:26223
+click:1 dense_feature:0.15 dense_feature:0.24543946932 dense_feature:0.0 dense_feature:0.12 dense_feature:0.0001875 dense_feature:0.004 dense_feature:0.08 dense_feature:0.12 dense_feature:0.072 dense_feature:0.1 dense_feature:0.4 dense_feature:0.0 dense_feature:0.04 1:663372 2:70321 3:202829 4:415480 5:286835 6:553107 7:32934 8:67180 9:27346 10:1873 11:699999 12:55775 13:371214 14:873363 15:685332 16:719499 17:342789 18:135819 19:26230 20:26229 21:973542 22:852086 23:410878 24:635223 25:26224 26:26223
+click:0 dense_feature:0.0 dense_feature:0.0679933665008 dense_feature:0.02 dense_feature:0.02 dense_feature:0.20015625 dense_feature:0.016 dense_feature:0.03 dense_feature:0.02 dense_feature:0.014 dense_feature:0.0 dense_feature:0.1 dense_feature:0.0 dense_feature:0.02 1:737395 2:229199 3:956202 4:475901 5:286835 6:948614 7:614385 8:718499 9:27346 10:171202 11:670646 12:566018 13:386065 14:873363 15:936716 16:825279 17:681378 18:758631 19:26230 20:26229 21:113534 22:26235 23:410878 24:551443 25:26224 26:26223
+click:1 dense_feature:0.05 dense_feature:0.00497512437811 dense_feature:0.04 dense_feature:0.22 dense_feature:0.015921875 dense_feature:0.022 dense_feature:0.04 dense_feature:0.4 dense_feature:0.182 dense_feature:0.1 dense_feature:0.2 dense_feature:0.0 dense_feature:0.22 1:737395 2:64837 3:751736 4:291977 5:286835 6:25207 7:377931 8:718499 9:27346 10:724396 11:433484 12:517940 13:439712 14:201966 15:628624 16:780717 17:342789 18:895034 19:880474 20:31181 21:463725 22:26235 23:410878 24:787115 25:884722 26:164940
+click:1 dense_feature:0.0 dense_feature:0.00995024875622 dense_feature:0.15 dense_feature:0.48 dense_feature:0.051375 dense_feature:0.0 dense_feature:0.0 dense_feature:0.06 dense_feature:0.556 dense_feature:0.0 dense_feature:0.0 dense_feature:0.0 dense_feature:0.5 1:737395 2:532829 3:158777 4:112926 5:286835 6:948614 7:764249 8:67180 9:27346 10:795273 11:330644 12:524443 13:78129 14:873363 15:127209 16:146094 17:342789 18:976129 19:26230 20:26229 21:901094 22:26235 23:410878 24:259263 25:26224 26:26223
+click:1 dense_feature:0.0 dense_feature:0.00497512437811 dense_feature:1.75 dense_feature:0.0 dense_feature:0.922828125 dense_feature:1.078 dense_feature:0.0 dense_feature:0.0 dense_feature:0.112 dense_feature:0.0 dense_feature:0.0 dense_feature:0.0 dense_feature:0.0 1:26973 2:62956 3:428206 4:935291 5:286835 6:446063 7:360307 8:437731 9:502607 10:957425 11:626052 12:641189 13:988058 14:217110 15:637914 16:293992 17:342789 18:832710 19:774721 20:537425 21:516798 22:191474 23:700326 24:204648 25:884722 26:776972
+click:1 dense_feature:1.95 dense_feature:0.00829187396352 dense_feature:0.08 dense_feature:0.1 dense_feature:0.01878125 dense_feature:0.044 dense_feature:0.42 dense_feature:0.24 dense_feature:0.358 dense_feature:0.1 dense_feature:0.2 dense_feature:0.1 dense_feature:0.26 1:737395 2:638265 3:526671 4:362576 5:938478 6:948614 7:999918 8:67180 9:27346 10:806276 11:181589 12:688684 13:367155 14:255651 15:709602 16:386859 17:228085 18:204112 19:668832 20:537425 21:541553 22:191474 23:410878 24:606704 25:49230 26:68113
+click:0 dense_feature:0.0 dense_feature:0.00331674958541 dense_feature:0.0 dense_feature:0.0 dense_feature:0.38159375 dense_feature:0.022 dense_feature:0.18 dense_feature:0.0 dense_feature:0.016 dense_feature:0.0 dense_feature:0.1 dense_feature:0.0 dense_feature:0.0 1:737395 2:841163 3:284187 4:385559 5:286835 6:446063 7:311604 8:67180 9:27346 10:38910 11:76230 12:520869 13:429321 14:255651 15:296507 16:542357 17:342789 18:377250 19:880474 20:31181 21:325494 22:26235 23:410878 24:26284 25:499868 26:467348
+click:0 dense_feature:0.0 dense_feature:0.00663349917081 dense_feature:0.08 dense_feature:0.0 dense_feature:0.077125 dense_feature:0.0 dense_feature:0.0 dense_feature:0.0 dense_feature:0.03 dense_feature:0.0 dense_feature:0.0 dense_feature:0.0 dense_feature:0.0 1:737395 2:238813 3:821667 4:209184 5:286835 6:906818 7:261420 8:67180 9:27346 10:748867 11:277196 12:790086 13:495408 14:873363 15:572266 16:281532 17:342789 18:99340 19:880474 20:537425 21:815896 22:26235 23:669531 24:17430 25:734238 26:251811
+click:0 dense_feature:0.0 dense_feature:0.210613598673 dense_feature:0.01 dense_feature:0.0 dense_feature:0.041375 dense_feature:0.0 dense_feature:0.0 dense_feature:0.08 dense_feature:0.026 dense_feature:0.0 dense_feature:0.0 dense_feature:0.0 dense_feature:0.0 1:737395 2:532829 3:559456 4:565823 5:286835 6:948614 7:48897 8:67180 9:27346 10:708281 11:214000 12:431427 13:477774 14:873363 15:637383 16:678446 17:276641 18:849284 19:26230 20:26229 21:758879 22:26235 23:410878 24:399458 25:26224 26:26223
+click:1 dense_feature:0.2 dense_feature:0.00331674958541 dense_feature:0.0 dense_feature:0.0 dense_feature:0.00440625 dense_feature:0.036 dense_feature:0.04 dense_feature:0.3 dense_feature:0.03 dense_feature:0.1 dense_feature:0.1 dense_feature:0.0 dense_feature:0.0 1:715353 2:532829 3:967094 4:707735 5:286835 6:948614 7:555710 8:154004 9:27346 10:708281 11:514992 12:158604 13:780149 14:255651 15:285282 16:149708 17:342789 18:553067 19:26230 20:26229 21:229985 22:26235 23:700326 24:777746 25:26224 26:26223
+click:1 dense_feature:0.0 dense_feature:0.00331674958541 dense_feature:0.0 dense_feature:0.0 dense_feature:0.23178125 dense_feature:0.222 dense_feature:0.06 dense_feature:0.0 dense_feature:0.408 dense_feature:0.0 dense_feature:0.2 dense_feature:0.0 dense_feature:0.0 1:715353 2:227084 3:456811 4:828682 5:286835 6:948614 7:406567 8:67180 9:27346 10:66123 11:598531 12:527138 13:731439 14:813514 15:35257 16:43339 17:342789 18:918487 19:26230 20:26229 21:580653 22:26235 23:410878 24:495283 25:26224 26:26223
+click:0 dense_feature:0.15 dense_feature:0.462686567164 dense_feature:0.08 dense_feature:0.22 dense_feature:0.00015625 dense_feature:0.022 dense_feature:0.03 dense_feature:0.52 dense_feature:0.022 dense_feature:0.1 dense_feature:0.1 dense_feature:0.0 dense_feature:0.22 1:576931 2:99294 3:263211 4:501662 5:938478 6:154084 7:128918 8:67180 9:27346 10:912799 11:801006 12:506258 13:378182 14:201966 15:150934 16:240427 17:681378 18:393279 19:26230 20:26229 21:152038 22:26235 23:700326 24:551443 25:26224 26:26223
+click:0 dense_feature:0.0 dense_feature:0.00331674958541 dense_feature:0.0 dense_feature:0.0 dense_feature:0.181484375 dense_feature:0.06 dense_feature:0.01 dense_feature:0.0 dense_feature:0.056 dense_feature:0.0 dense_feature:0.1 dense_feature:0.0 dense_feature:0.0 1:230803 2:283434 3:367596 4:197992 5:938478 6:948614 7:268098 8:67180 9:27346 10:870993 11:632267 12:139817 13:718764 14:255651 15:884839 16:80117 17:276641 18:556463 19:880474 20:537425 21:271358 22:26235 23:410878 24:488077 25:253255 26:584828
+click:0 dense_feature:0.0 dense_feature:0.00497512437811 dense_feature:0.0 dense_feature:0.16 dense_feature:4.790078125 dense_feature:0.0 dense_feature:0.0 dense_feature:0.28 dense_feature:0.016 dense_feature:0.0 dense_feature:0.0 dense_feature:0.0 dense_feature:0.2 1:737395 2:532829 3:158777 4:112926 5:286835 6:948614 7:277312 8:67180 9:502607 10:708281 11:755513 12:524443 13:4029 14:873363 15:503814 16:146094 17:644532 18:121590 19:26230 20:26229 21:901094 22:191474 23:744389 24:259263 25:26224 26:26223
+click:0 dense_feature:0.0 dense_feature:3.30845771144 dense_feature:0.0 dense_feature:0.04 dense_feature:0.022671875 dense_feature:0.062 dense_feature:0.01 dense_feature:0.4 dense_feature:0.062 dense_feature:0.0 dense_feature:0.1 dense_feature:0.0 dense_feature:0.04 1:663372 2:529436 3:511823 4:942782 5:286835 6:906818 7:190054 8:67180 9:27346 10:708281 11:32527 12:494263 13:652478 14:873363 15:616057 16:17325 17:342789 18:325238 19:26230 20:26229 21:256747 22:179327 23:410878 24:169709 25:26224 26:26223
+click:0 dense_feature:0.0 dense_feature:0.00829187396352 dense_feature:0.01 dense_feature:0.16 dense_feature:0.206765625 dense_feature:0.328 dense_feature:0.13 dense_feature:0.16 dense_feature:0.176 dense_feature:0.0 dense_feature:0.7 dense_feature:0.0 dense_feature:0.16 1:737395 2:552854 3:606082 4:267619 5:286835 6:948614 7:918889 8:67180 9:27346 10:708281 11:400024 12:972010 13:66330 14:255651 15:432931 16:650209 17:506108 18:212910 19:26230 20:26229 21:107726 22:26235 23:410878 24:718419 25:26224 26:26223

recommend-model-produce/src/main/python/models/dnn/dygraph_model.py

@@ -0,0 +1,96 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import paddle
+import paddle.nn as nn
+import paddle.nn.functional as F
+import math
+
+import net
+
+
+class DygraphModel():
+    # define model
+    def create_model(self, config):
+        sparse_feature_number = config.get(
+            "hyper_parameters.sparse_feature_number")
+        sparse_feature_dim = config.get("hyper_parameters.sparse_feature_dim")
+        fc_sizes = config.get("hyper_parameters.fc_sizes")
+        sparse_fea_num = config.get('hyper_parameters.sparse_fea_num')
+        dense_feature_dim = config.get('hyper_parameters.dense_input_dim')
+        sparse_input_slot = config.get('hyper_parameters.sparse_inputs_slots')
+
+        dnn_model = net.DNNLayer(sparse_feature_number, sparse_feature_dim,
+                                 dense_feature_dim, sparse_input_slot - 1,
+                                 fc_sizes)
+        return dnn_model
+
+    # define feeds which convert numpy of batch data to paddle.tensor
+    def create_feeds(self, batch_data, config):
+        dense_feature_dim = config.get('hyper_parameters.dense_input_dim')
+        sparse_tensor = []
+        for b in batch_data[:-1]:
+            sparse_tensor.append(
+                paddle.to_tensor(b.numpy().astype('int64').reshape(-1, 1)))
+        dense_tensor = paddle.to_tensor(batch_data[-1].numpy().astype(
+            'float32').reshape(-1, dense_feature_dim))
+        label = sparse_tensor[0]
+        return label, sparse_tensor[1:], dense_tensor
+
+    # define loss function by predicts and label
+    def create_loss(self, raw_predict_2d, label):
+        cost = paddle.nn.functional.cross_entropy(
+            input=raw_predict_2d, label=label)
+        avg_cost = paddle.mean(x=cost)
+        return avg_cost
+
+    # define optimizer
+    def create_optimizer(self, dy_model, config):
+        lr = config.get("hyper_parameters.optimizer.learning_rate", 0.001)
+        optimizer = paddle.optimizer.Adam(
+            learning_rate=lr, parameters=dy_model.parameters())
+        return optimizer
+
+    # define metrics such as auc/acc
+    # multi-task need to define multi metric
+    def create_metrics(self):
+        metrics_list_name = ["auc"]
+        auc_metric = paddle.metric.Auc("ROC")
+        metrics_list = [auc_metric]
+        return metrics_list, metrics_list_name
+
+    # construct train forward phase
+    def train_forward(self, dy_model, metrics_list, batch_data, config):
+        label, sparse_tensor, dense_tensor = self.create_feeds(batch_data,
+                                                               config)
+
+        raw_pred_2d = dy_model.forward(sparse_tensor, dense_tensor)
+        loss = self.create_loss(raw_pred_2d, label)
+        # update metrics
+        predict_2d = paddle.nn.functional.softmax(raw_pred_2d)
+        metrics_list[0].update(preds=predict_2d.numpy(), labels=label.numpy())
+
+        print_dict = {'loss': loss}
+        # print_dict = None
+        return loss, metrics_list, print_dict
+
+    def infer_forward(self, dy_model, metrics_list, batch_data, config):
+        label, sparse_tensor, dense_tensor = self.create_feeds(batch_data,
+                                                               config)
+
+        raw_pred_2d = dy_model.forward(sparse_tensor, dense_tensor)
+        # update metrics
+        predict_2d = paddle.nn.functional.softmax(raw_pred_2d)
+        metrics_list[0].update(preds=predict_2d.numpy(), labels=label.numpy())
+        return metrics_list, None

recommend-model-produce/src/main/python/models/dnn/net.py

@@ -0,0 +1,142 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import paddle
+import paddle.nn as nn
+import paddle.nn.functional as F
+import math
+
+
+class DNNLayer(nn.Layer):
+    def __init__(self,
+                 sparse_feature_number,
+                 sparse_feature_dim,
+                 dense_feature_dim,
+                 num_field,
+                 layer_sizes,
+                 sync_mode=None):
+        super(DNNLayer, self).__init__()
+        self.sync_mode = sync_mode
+        self.sparse_feature_number = sparse_feature_number
+        self.sparse_feature_dim = sparse_feature_dim
+        self.dense_feature_dim = dense_feature_dim
+        self.num_field = num_field
+        self.layer_sizes = layer_sizes
+
+        use_sparse = True
+        if paddle.is_compiled_with_custom_device('npu'):
+            use_sparse = False
+
+        self.embedding = paddle.nn.Embedding(
+            self.sparse_feature_number,
+            self.sparse_feature_dim,
+            sparse=use_sparse,
+            weight_attr=paddle.ParamAttr(
+                name="SparseFeatFactors",
+                initializer=paddle.nn.initializer.Uniform()))
+
+        sizes = [sparse_feature_dim * num_field + dense_feature_dim
+                 ] + self.layer_sizes + [2]
+        acts = ["relu" for _ in range(len(self.layer_sizes))] + [None]
+        self._mlp_layers = []
+        for i in range(len(layer_sizes) + 1):
+            linear = paddle.nn.Linear(
+                in_features=sizes[i],
+                out_features=sizes[i + 1],
+                weight_attr=paddle.ParamAttr(
+                    initializer=paddle.nn.initializer.Normal(
+                        std=1.0 / math.sqrt(sizes[i]))))
+            self.add_sublayer('linear_%d' % i, linear)
+            self._mlp_layers.append(linear)
+            if acts[i] == 'relu':
+                act = paddle.nn.ReLU()
+                self.add_sublayer('act_%d' % i, act)
+                self._mlp_layers.append(act)
+
+    def forward(self, sparse_inputs, dense_inputs, show_click=None):
+
+        sparse_embs = []
+        for s_input in sparse_inputs:
+            if self.sync_mode == "gpubox":
+                emb = paddle.static.nn.sparse_embedding(
+                    input=s_input,
+                    size=[
+                        self.sparse_feature_number, self.sparse_feature_dim + 2
+                    ],
+                    param_attr=paddle.ParamAttr(name="embedding"))
+                emb = paddle.static.nn.continuous_value_model(emb, show_click,
+                                                              False)
+            else:
+                emb = self.embedding(s_input)
+            emb = paddle.reshape(emb, shape=[-1, self.sparse_feature_dim])
+            sparse_embs.append(emb)
+
+        y_dnn = paddle.concat(x=sparse_embs + [dense_inputs], axis=1)
+
+        for n_layer in self._mlp_layers:
+            y_dnn = n_layer(y_dnn)
+
+        return y_dnn
+
+
+class StaticDNNLayer(nn.Layer):
+    def __init__(self, sparse_feature_number, sparse_feature_dim,
+                 dense_feature_dim, num_field, layer_sizes):
+        super(StaticDNNLayer, self).__init__()
+        self.sparse_feature_number = sparse_feature_number
+        self.sparse_feature_dim = sparse_feature_dim
+        self.dense_feature_dim = dense_feature_dim
+        self.num_field = num_field
+        self.layer_sizes = layer_sizes
+
+        #self.embedding = paddle.nn.Embedding(
+        #    self.sparse_feature_number,
+        #    self.sparse_feature_dim,
+        #    sparse=True,
+        #    weight_attr=paddle.ParamAttr(
+        #        name="SparseFeatFactors",
+        #        initializer=paddle.nn.initializer.Uniform()))
+
+        sizes = [sparse_feature_dim * num_field + dense_feature_dim
+                 ] + self.layer_sizes + [2]
+        acts = ["relu" for _ in range(len(self.layer_sizes))] + [None]
+        self._mlp_layers = []
+        for i in range(len(layer_sizes) + 1):
+            linear = paddle.nn.Linear(
+                in_features=sizes[i],
+                out_features=sizes[i + 1],
+                weight_attr=paddle.ParamAttr(
+                    initializer=paddle.nn.initializer.Normal(
+                        std=1.0 / math.sqrt(sizes[i]))))
+            self.add_sublayer('linear_%d' % i, linear)
+            self._mlp_layers.append(linear)
+            if acts[i] == 'relu':
+                act = paddle.nn.ReLU()
+                self.add_sublayer('act_%d' % i, act)
+                self._mlp_layers.append(act)
+
+    def forward(self, sparse_embs, dense_inputs):
+
+        #sparse_embs = []
+        #for s_input in sparse_inputs:
+        #    emb = self.embedding(s_input)
+        #    emb = paddle.reshape(emb, shape=[-1, self.sparse_feature_dim])
+        #    sparse_embs.append(emb)
+
+        y_dnn = paddle.concat(x=sparse_embs + [dense_inputs], axis=1)
+
+        for n_layer in self._mlp_layers:
+            y_dnn = n_layer(y_dnn)
+
+        return y_dnn

recommend-model-produce/src/main/python/models/dnn/online.yaml

@@ -0,0 +1,49 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+hyper_parameters:
+  optimizer:
+    class: Adam
+    learning_rate: 0.0001
+    adam_lazy_mode: True
+  sparse_inputs_slots: 27
+  sparse_feature_number: 1000001
+  sparse_feature_dim: 10
+  dense_input_dim: 13
+  fc_sizes: [400, 400, 400]
+
+runner:
+  train_data_dir: ["data/sample_data/train/"]
+  days: "{20191225..20191227}"
+  pass_per_day: 24
+
+  train_batch_size: 12
+  train_thread_num: 16
+  geo_step: 400
+  sync_mode: "async"  # sync / async /geo / heter
+
+  pipe_command: "python benchmark_reader.py"
+  print_interval: 100
+
+  use_gpu: 0
+
+  model_path: "static_model.py"
+  dataset_debug: False
+  model_save_path: "model"
+
+  # knock-in and knock-out
+  # create_num_threshold: 1      # knock-in
+  # max_keep_days: 60            # knock-out
+
+  

recommend-model-produce/src/main/python/models/dnn/queuedataset_reader.py

@@ -0,0 +1,95 @@
+# Copyright (c) 2019 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import sys
+import yaml
+import six
+import os
+import copy
+import paddle.distributed.fleet as fleet
+import logging
+import numpy as np
+
+logging.basicConfig(
+    format='%(asctime)s - %(levelname)s - %(message)s', level=logging.INFO)
+logger = logging.getLogger(__name__)
+
+
+class Reader(fleet.MultiSlotDataGenerator):
+    def init(self, config):
+        self.config = config
+        padding = 0
+        sparse_slots = "click 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26"
+        self.sparse_slots = sparse_slots.strip().split(" ")
+        self.dense_slots = ["dense_feature"]
+        self.dense_slots_shape = [13]
+        self.slots = self.sparse_slots + self.dense_slots
+        self.slot2index = {}
+        self.visit = {}
+        for i in range(len(self.slots)):
+            self.slot2index[self.slots[i]] = i
+            self.visit[self.slots[i]] = False
+        self.padding = padding
+        logger.info("pipe init success")
+
+    def line_process(self, line):
+        line = line.strip().split(" ")
+        output = [(i, []) for i in self.slots]
+        for i in line:
+            slot_feasign = i.split(":")
+            slot = slot_feasign[0]
+            if slot not in self.slots:
+                continue
+            if slot in self.sparse_slots:
+                feasign = int(slot_feasign[1])
+            else:
+                feasign = float(slot_feasign[1])
+            output[self.slot2index[slot]][1].append(feasign)
+            self.visit[slot] = True
+        for i in self.visit:
+            slot = i
+            if not self.visit[slot]:
+                if i in self.dense_slots:
+                    output[self.slot2index[i]][1].extend(
+                        [self.padding] *
+                        self.dense_slots_shape[self.slot2index[i]])
+                else:
+                    output[self.slot2index[i]][1].extend([self.padding])
+            else:
+                self.visit[slot] = False
+
+        return output
+        #return [label] + sparse_feature + [dense_feature]
+    def generate_sample(self, line):
+        r"Dataset Generator"
+
+        def reader():
+            output_dict = self.line_process(line)
+            # {key, value} dict format: {'labels': [1], 'sparse_slot1': [2, 3], 'sparse_slot2': [4, 5, 6, 8], 'dense_slot': [1,2,3,4]} 
+            # dict must match static_model.create_feed()
+            yield output_dict
+
+        return reader
+
+
+if __name__ == "__main__":
+    yaml_path = sys.argv[1]
+    utils_path = sys.argv[2]
+    sys.path.append(utils_path)
+    import common_ps
+    yaml_helper = common_ps.YamlHelper()
+    config = yaml_helper.load_yaml(yaml_path)
+
+    r = Reader()
+    r.init(config)
+    r.run_from_stdin()

recommend-model-produce/src/main/python/models/dnn/static_model.py

@@ -0,0 +1,129 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import math
+import paddle
+
+from net import DNNLayer, StaticDNNLayer
+
+
+class StaticModel():
+    def __init__(self, config):
+        self.cost = None
+        self.infer_target_var = None
+        self.config = config
+        self._init_hyper_parameters()
+        self.sync_mode = config.get("runner.sync_mode")
+
+    def _init_hyper_parameters(self):
+        self.is_distributed = False
+        self.distributed_embedding = False
+
+        if self.config.get("hyper_parameters.distributed_embedding", 0) == 1:
+            self.distributed_embedding = True
+
+        self.sparse_feature_number = self.config.get(
+            "hyper_parameters.sparse_feature_number")
+        self.sparse_feature_dim = self.config.get(
+            "hyper_parameters.sparse_feature_dim")
+        self.sparse_inputs_slots = self.config.get(
+            "hyper_parameters.sparse_inputs_slots")
+        self.dense_input_dim = self.config.get(
+            "hyper_parameters.dense_input_dim")
+        self.learning_rate = self.config.get(
+            "hyper_parameters.optimizer.learning_rate")
+        self.fc_sizes = self.config.get("hyper_parameters.fc_sizes")
+
+    def create_feeds(self, is_infer=False):
+        dense_input = paddle.static.data(
+            name="dense_input",
+            shape=[None, self.dense_input_dim],
+            dtype="float32")
+
+        # sparse_input_ids = [
+        #     paddle.static.data(
+        #         name="C" + str(i), shape=[None, 1], dtype="int64")
+        #     for i in range(1, self.sparse_inputs_slots)
+        # ]
+
+        sparse_input_ids = [
+            paddle.static.data(
+                name=str(i), shape=[None, 1], dtype="int64")
+            for i in range(1, self.sparse_inputs_slots)
+        ]
+
+        label = paddle.static.data(
+            name="label", shape=[None, 1], dtype="int64")
+
+        feeds_list = [label] + sparse_input_ids + [dense_input]
+        return feeds_list
+
+    def net(self, input, is_infer=False):
+        self.label_input = input[0]
+        self.sparse_inputs = input[1:self.sparse_inputs_slots]
+        self.dense_input = input[-1]
+        sparse_number = self.sparse_inputs_slots - 1
+
+        dnn_model = DNNLayer(
+            self.sparse_feature_number,
+            self.sparse_feature_dim,
+            self.dense_input_dim,
+            sparse_number,
+            self.fc_sizes,
+            sync_mode=self.sync_mode)
+
+        self.cast_label = paddle.cast(self.label_input, dtype='float32')
+        ones = paddle.full_like(
+            self.label_input, fill_value=1, dtype="float32")
+        show_click = paddle.cast(
+            paddle.concat(
+                [ones, self.cast_label], axis=1), dtype='float32')
+        show_click.stop_gradient = True
+        raw_predict_2d = dnn_model.forward(self.sparse_inputs,
+                                           self.dense_input, show_click)
+
+        predict_2d = paddle.nn.functional.softmax(raw_predict_2d)
+
+        self.predict = predict_2d
+
+        auc, batch_auc, [
+            self.batch_stat_pos, self.batch_stat_neg, self.stat_pos,
+            self.stat_neg
+        ] = paddle.static.auc(input=self.predict,
+                              label=self.label_input,
+                              num_thresholds=2**12,
+                              slide_steps=20)
+        self.inference_target_var = auc
+        if is_infer:
+            fetch_dict = {'auc': auc}
+            return fetch_dict
+
+        cost = paddle.nn.functional.cross_entropy(
+            input=raw_predict_2d, label=self.label_input)
+        avg_cost = paddle.mean(x=cost)
+        self._cost = avg_cost
+
+        fetch_dict = {'cost': avg_cost, 'auc': auc}
+        return fetch_dict
+
+    def create_optimizer(self, strategy=None):
+        optimizer = paddle.optimizer.Adam(
+            learning_rate=self.learning_rate, lazy_mode=True)
+        if strategy != None:
+            import paddle.distributed.fleet as fleet
+            optimizer = fleet.distributed_optimizer(optimizer, strategy)
+        optimizer.minimize(self._cost)
+
+    def infer_net(self, input):
+        return self.net(input, is_infer=True)

recommend-model-produce/src/main/python/models/dnn/static_model_lod.py

@@ -0,0 +1,138 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import math
+import paddle
+
+from net import DNNLayer, StaticDNNLayer
+
+
+class StaticModel():
+    def __init__(self, config):
+        self.cost = None
+        self.infer_target_var = None
+        self.config = config
+        self._init_hyper_parameters()
+
+    def _init_hyper_parameters(self):
+        self.is_distributed = False
+        self.distributed_embedding = False
+
+        if self.config.get("hyper_parameters.distributed_embedding", 0) == 1:
+            self.distributed_embedding = True
+
+        self.sparse_feature_number = self.config.get(
+            "hyper_parameters.sparse_feature_number")
+        self.sparse_feature_dim = self.config.get(
+            "hyper_parameters.sparse_feature_dim")
+        self.sparse_inputs_slots = self.config.get(
+            "hyper_parameters.sparse_inputs_slots")
+        self.dense_input_dim = self.config.get(
+            "hyper_parameters.dense_input_dim")
+        self.learning_rate = self.config.get(
+            "hyper_parameters.optimizer.learning_rate")
+        self.fc_sizes = self.config.get("hyper_parameters.fc_sizes")
+
+    def create_feeds(self, is_infer=False):
+        dense_input = paddle.static.data(
+            name="dense_input",
+            shape=[None, self.dense_input_dim],
+            dtype="float32")
+
+        sparse_input_ids = [
+            paddle.static.data(
+                name="C" + str(i), shape=[None, 1], lod_level=1, dtype="int64")
+            for i in range(1, self.sparse_inputs_slots)
+        ]
+
+        label = paddle.static.data(
+            name="label", shape=[None, 1], dtype="int64")
+
+        feeds_list = [label] + sparse_input_ids + [dense_input]
+        return feeds_list
+
+    def net(self, input, is_infer=False):
+        self.label_input = input[0]
+        self.sparse_inputs = input[1:self.sparse_inputs_slots]
+        self.dense_input = input[-1]
+        sparse_number = self.sparse_inputs_slots - 1
+
+        def embedding_layer(input):
+            if self.distributed_embedding:
+                emb = paddle.static.nn.sparse_embedding(
+                    input=input,
+                    size=[
+                        self.sparse_feature_number, self.sparse_feature_dim
+                    ],
+                    param_attr=paddle.ParamAttr(
+                        name="SparseFeatFactors",
+                        initializer=paddle.nn.initializer.Uniform()))
+            else:
+                paddle.static.Print(input)
+
+                emb = paddle.static.nn.embedding(
+                    input=input,
+                    is_sparse=True,
+                    is_distributed=self.is_distributed,
+                    size=[
+                        self.sparse_feature_number, self.sparse_feature_dim
+                    ],
+                    param_attr=paddle.ParamAttr(
+                        name="SparseFeatFactors",
+                        initializer=paddle.nn.initializer.Uniform()))
+            emb_sum = paddle.static.nn.sequence_pool(
+                input=emb, pool_type='sum')
+            sequeeze_emb_sum = paddle.squeeze(emb_sum, axis=1)
+            #return emb_sum
+            return sequeeze_emb_sum
+
+        sparse_embs = list(map(embedding_layer, self.sparse_inputs))
+
+        dnn_model = StaticDNNLayer(
+            self.sparse_feature_number, self.sparse_feature_dim,
+            self.dense_input_dim, sparse_number, self.fc_sizes)
+
+        raw_predict_2d = dnn_model.forward(sparse_embs, self.dense_input)
+
+        predict_2d = paddle.nn.functional.softmax(raw_predict_2d)
+
+        self.predict = predict_2d
+
+        auc, batch_auc, _ = paddle.static.auc(input=self.predict,
+                                              label=self.label_input,
+                                              num_thresholds=2**12,
+                                              slide_steps=20)
+        self.inference_target_var = auc
+        if is_infer:
+            fetch_dict = {'auc': auc}
+            return fetch_dict
+
+        cost = paddle.nn.functional.cross_entropy(
+            input=raw_predict_2d, label=self.label_input)
+        avg_cost = paddle.mean(x=cost)
+        self._cost = avg_cost
+
+        fetch_dict = {'cost': avg_cost, 'auc': auc}
+        return fetch_dict
+
+    def create_optimizer(self, strategy=None):
+        optimizer = paddle.optimizer.Adam(
+            learning_rate=self.learning_rate, lazy_mode=True)
+        if strategy != None:
+            import paddle.distributed.fleet as fleet
+            optimizer = fleet.distributed_optimizer(optimizer, strategy)
+        optimizer.minimize(self._cost)
+
+    def infer_net(self, input):
+        return self.net(input, is_infer=True)