NLP Status Report 2017-3-13 - 版本历史

2017年3月14日 (二) 01:46 Zhangandy

2017-03-14T01:46:23Z

2017年3月13日 (一) 08:01 Zhangsy

2017-03-13T08:01:36Z

2017年3月13日 (一) 07:10 Zhangjy

2017-03-13T07:10:10Z

2017年3月13日 (一) 07:07 Zhangjy

2017-03-13T07:07:15Z

2017年3月13日 (一) 05:50 Fengyang

2017-03-13T05:50:00Z

2017年3月13日 (一) 05:33 Fengyang

2017-03-13T05:33:06Z

Fengyang：以“{| class="wikitable" !Date !! People !! Last Week !! This Week |- | rowspan="6"|2017/1/3 |Yang Feng || * ran experiments on the CS-EN data set (200k pairs) with tota...”为内容创建页面

2017-03-13T05:26:58Z

以“{| class="wikitable" !Date !! People !! Last Week !! This Week |- | rowspan="6"|2017/1/3 |Yang Feng || * ran experiments on the CS-EN data set (200k pairs) with tota...”为内容创建页面

新页面

{| class="wikitable"
!Date !! People !! Last Week !! This Week
|-
| rowspan="6"|2017/1/3
|Yang Feng ||
* ran experiments on the CS-EN data set (200k pairs) with totally identical initialization as the paper. on the sampled 2k training sentences, the bleu is 19.5 (not converged yet). (the bleu on the test set expected to be 26)
* add the alpha and gamma score and do multi-task training. Without multi-task training, the loss didn't decline on the training data, but with multi-task training, the loss did decline.
* prepared for Huilan's inspection.
||
* Analyze the reason that the loss didn't decline with alpha and gamma
* test for multi-task training;
* improve the baseline for the CS-EN
|-
|Jiyuan Zhang ||
* reproduced planning neural network [http://cslt.riit.tsinghua.edu.cn/mediawiki/images/3/38/Planning_neural_network_initial_decode.pdf results]
||
*reproduce planning neural network
|-
|Andi Zhang ||
* added source masks in attention_decoder where calculates attention and in gru_cell where calculates new states.
* found the attribute sentence_length, perhaps it works better than my code
||

|-
|Shiyue Zhang ||
* figured out the problem of attention: the initial value of V should be around 0
* tested different modification, such as add mask, init b with 0.
* Compared the results, and concluded only change the initial value of V is the best.
||
* try to get right attention on memory
|-
|Peilun Xiao ||

||

|}

@@ 第27行： / 第27行： @@
 |-
 |Shiyue Zhang ||
+* added trained memory-attention model to neural model(43.0) and got 2+ blue gain (45.19), but need more validation and improvement
 ||
+* go on with baseline on big data: get results of cs-en and enfr data, train on zh-en data from [http://www.statmt.org/wmt17/translation-task.html#download WMT17]
 |-
 |Peilun Xiao ||

@@ 第13行： / 第13行： @@
 |Jiyuan Zhang ||
 *completed to reproduce planning neural network
-*chose best attention_memory model for huilian  and ran big train dataset(about 370k)
+*chose best attention_memory model for huilian  and ran big train dataset(about 370k) [http://cslt.riit.tsinghua.edu.cn/mediawiki/images/b/b9/Model_with_different_dataset.pdf  result]
 ||

@@ 第6行： / 第6行： @@
 * tested and analyzed the results on the cs-en data set (30.4 on the heldout-training set and 7.3 on the dev set);
 * added masks to the baseline (44.4 on the cn-en);
-* added masks to alpha-gamma method and fixed the bugs. Got an improvement of 0.5 again the masked baseline [[http://cslt.riit.tsinghua.edu.cn/mediawiki/images/b/b8/Nmt_mn_report_continue.pdf report]];
+* added encoder-masks and memory-masks to alpha-gamma method and fixed the bugs. Got an improvement of 0.5 again the masked baseline [[http://cslt.riit.tsinghua.edu.cn/mediawiki/images/b/b8/Nmt_mn_report_continue.pdf report]];
 * To avoid doing softmax twice, rewrite the softmax_cross_entropy function myself. (under-training)
 ||

@@ 第22行： / 第22行： @@
 |-
 |Andi Zhang ||
+*ran baseline without mask, found that the model with masks has a slightly better bleu score.
 ||
+*try to fix the problem
 |-
 |Shiyue Zhang ||

@@ 第4行： / 第4行： @@
 | rowspan="6"|2017/1/3
 |Yang Feng ||
-* ran experiments on the CS-EN data set (200k pairs) with totally identical initialization as the paper. on the sampled 2k training sentences, the bleu is 19.5 (not converged yet). (the bleu on the test set expected to be 26)
+* tested and analyzed the results on the cs-en data set (30.4 on the heldout-training set and 7.3 on the dev set);
-* add the alpha and gamma score and do multi-task training. Without multi-task training, the loss didn't decline on the training data, but with multi-task training, the loss did decline.
+* added masks to the baseline (44.4 on the cn-en);
-* prepared for Huilan's inspection.
+* added masks to alpha-gamma method and fixed the bugs. Got an improvement of 0.5 again the masked baseline [[http://cslt.riit.tsinghua.edu.cn/mediawiki/images/b/b8/Nmt_mn_report_continue.pdf report]];
 ||
-* Analyze the reason that the loss didn't decline with alpha and gamma
+* analyze and improve the alpha-gamma method.
-* test for multi-task training;
-* improve the baseline for the CS-EN
 |-
 |Jiyuan Zhang ||
-*  reproduced planning neural network [http://cslt.riit.tsinghua.edu.cn/mediawiki/images/3/38/Planning_neural_network_initial_decode.pdf results]
 ||
 |-
 |Andi Zhang ||
-* added source masks in attention_decoder where calculates attention and in gru_cell where calculates new states.
-* found the attribute sentence_length, perhaps it works better than my code
 ||
 |-
 |Shiyue Zhang ||
-* figured out the problem of attention: the initial value of V should be around 0
-* tested different modification, such as add mask, init b with 0.
-* Compared the results, and concluded only change the initial value of V is the best.
 ||
-* try to get right attention on memory
 |-
 |Peilun Xiao ||