深度學習課件

1、深度學習人工智能的起點: 達特茅斯會議1919-20011927-20111927-20161916-2011Nathaniel Rochester人工智能的階段 1950s 1980s 2000s Future 自動計算機如何為計算機編程使其能夠使用語言神經(jīng)網(wǎng)絡(luò)計算規(guī)模理論自我提升抽象隨機性與創(chuàng)造性基于規(guī)則的專家系統(tǒng)通用智能123人工智能的當前技術(shù): 存在的問題 依賴大量的標注數(shù)據(jù)“窄人工智能” 訓練完成特定的任務(wù)不夠穩(wěn)定，安全不具備解釋能力，模型不透明人工智能的當前狀態(tài): 應(yīng)用人工智能成為熱點的原因: 深度學習, 強化學習大規(guī)模的,復雜的,流式的數(shù)據(jù)概要解析白宮人工智能研發(fā)戰(zhàn)略計劃3. 深

2、度學習及最新進展2. 解析十家技術(shù)公司的的人工智能戰(zhàn)略4. 強化學習及最新進展5. 深度學習在企業(yè)數(shù)據(jù)分析中的應(yīng)用美國人工智能戰(zhàn)略規(guī)劃美國人工智能研發(fā)戰(zhàn)略規(guī)劃策略- I : 在人工智能研究領(lǐng)域做長期研發(fā)投資 目標：. 確保美國的世界領(lǐng)導地位 . 優(yōu)先投資下一代人工智能技術(shù)推動以數(shù)據(jù)為中心的知識發(fā)現(xiàn)技術(shù)高效的數(shù)據(jù)清潔技術(shù)以，確保用于訓練系統(tǒng)的數(shù)據(jù)的可信性(varascty)和正確性(appropriateness) 綜合考慮 數(shù)據(jù)，元數(shù)據(jù)，以及人的反饋或知識異構(gòu)數(shù)據(jù)，多模態(tài)數(shù)據(jù)分析和挖掘，離散數(shù)據(jù)，連續(xù)數(shù)據(jù)，時間域數(shù)據(jù)，空間域數(shù)據(jù)，時空數(shù)據(jù)，圖數(shù)據(jù)小數(shù)據(jù)挖掘，強調(diào)小概率事件的重要性數(shù)據(jù)和知識尤其

3、領(lǐng)域知識庫的融合使用策略- I : 在人工智能研究領(lǐng)域做長期研發(fā)投資 目標：. 確保美國的世界領(lǐng)導地位 . 優(yōu)先投資下一代人工智能技術(shù)推動以數(shù)據(jù)為中心的知識發(fā)現(xiàn)技術(shù)2. 增強系統(tǒng)的感知能力硬件或算法能提升系統(tǒng)感知能力的穩(wěn)健性和可靠性提升在復雜動態(tài)環(huán)境中對物體的檢測，分類，辨別，識別能力提升傳感器或算法對人的感知，以便系統(tǒng)更好地跟人的合作計算和傳播感知系統(tǒng)的不確定性給系統(tǒng)以便更好的判斷策略- I : 在人工智能研究領(lǐng)域做長期研發(fā)投資 目標：. 確保美國的世界領(lǐng)導地位 . 優(yōu)先投資下一代人工智能技術(shù)推動以數(shù)據(jù)為中心的知識發(fā)現(xiàn)技術(shù)2. 增強系統(tǒng)的感知能力當前硬件環(huán)境和算法框架下AI的理論上限學習能力

4、語言能力感知能力推理能力創(chuàng)造力計劃，規(guī)劃能力3. 理論能力和上限策略- I : 在人工智能研究領(lǐng)域做長期研發(fā)投資 目標：. 確保美國的世界領(lǐng)導地位 . 優(yōu)先投資下一代人工智能技術(shù)推動以數(shù)據(jù)為中心的知識發(fā)現(xiàn)技術(shù)2. 增強系統(tǒng)的感知能力目前的AI系統(tǒng)均為窄人工智能， “Narrow AI”而不是“General AI”GAI: 靈活， 多任務(wù)， 有自由意志，在多認知任務(wù)中的通用能力（學習能力， 語言能力，感知能力，推理能力，創(chuàng)造力，計劃，規(guī)劃能力遷移學習3. 理論能力和上限4. 通用AI策略- I : 在人工智能研究領(lǐng)域做長期研發(fā)投資 目標：. 確保美國的世界領(lǐng)導地位 . 優(yōu)先投資下一代人工智能技

5、術(shù)推動以數(shù)據(jù)為中心的知識發(fā)現(xiàn)技術(shù)2. 增強系統(tǒng)的感知能力多AI系統(tǒng)的協(xié)同分布式計劃和控制技術(shù)3. 理論能力和上限4. 通用AI5. 規(guī)?；疉I系統(tǒng)策略- I : 在人工智能研究領(lǐng)域做長期研發(fā)投資 目標：. 確保美國的世界領(lǐng)導地位 . 優(yōu)先投資下一代人工智能技術(shù)推動以數(shù)據(jù)為中心的知識發(fā)現(xiàn)技術(shù)2. 增強系統(tǒng)的感知能力AI系統(tǒng)的自我解釋能力目前AI系統(tǒng)的學習方法：大數(shù)據(jù)，黑盒人的學習方法：小數(shù)據(jù)，接受正規(guī)的指導規(guī)則以及各種暗示仿人的AI系統(tǒng)，可以做智能助理，智能輔導3. 理論能力和上限4. 通用AI5. 規(guī)模化AI系統(tǒng)6. 仿人類的AI技術(shù)策略- I : 在人工智能研究領(lǐng)域做長期研發(fā)投資 目標：.

6、確保美國的世界領(lǐng)導地位 . 優(yōu)先投資下一代人工智能技術(shù)推動以數(shù)據(jù)為中心的知識發(fā)現(xiàn)技術(shù)2. 增強系統(tǒng)的感知能力提升機器人的感知能力，更智能的同復雜的物理世界交互3. 理論能力和上限4. 通用AI5. 規(guī)?；疉I系統(tǒng)6. 仿人類的AI技術(shù)7. 研發(fā)實用，可靠，易用的機器人策略- I : 在人工智能研究領(lǐng)域做長期研發(fā)投資 目標：. 確保美國的世界領(lǐng)導地位 . 優(yōu)先投資下一代人工智能技術(shù)推動以數(shù)據(jù)為中心的知識發(fā)現(xiàn)技術(shù)2. 增強系統(tǒng)的感知能力提升機器人的感知能力，更智能的同復雜的物理世界交互 GPU：提升的內(nèi)存，輸入輸出，時鐘 速度，并行能力，節(jié)能“類神經(jīng)元”處理器處理基于流式，動態(tài)數(shù)據(jù)利用AI技術(shù)提升

7、硬件能力：高性能計算，優(yōu)化能源消耗，增強計算性能，自我智能配置，優(yōu)化數(shù)據(jù)在多核處理器和內(nèi)存直接移動3. 理論能力和上限4. 通用AI5. 規(guī)模化AI系統(tǒng)6. 仿人類的AI技術(shù)7. 研發(fā)實用，可靠，易用的機器人8. AI和硬件的相互推動策略-II： 開發(fā)有效的人機合作方法. 不是替代人，而是跟人合作，強調(diào)人和AI系統(tǒng)之間的互補作用輔助人類的人工智能技術(shù)AI系統(tǒng)的設(shè)計很多是為人所用復制人類計算，決策，認知策略-II： 開發(fā)有效的人機合作方法. 不是替代人，而是跟人合作，強調(diào)人和AI系統(tǒng)之間的互補作用輔助人類的人工智能技術(shù)2. 開發(fā)增強人類的技術(shù)穩(wěn)態(tài)設(shè)備穿戴設(shè)備植入設(shè)備輔助數(shù)據(jù)理解策略-II： 開發(fā)

8、有效的人機合作方法. 不是替代人，而是跟人合作，強調(diào)人和AI系統(tǒng)之間的互補作用輔助人類的人工智能技術(shù)2. 開發(fā)增強人類的技術(shù)數(shù)據(jù)和信息的可視化，以人可以理解的方式展現(xiàn)提升人和系統(tǒng)通信的效率3. 可視化， AI-人之間的友好界面策略-II： 開發(fā)有效的人機合作方法. 不是替代人，而是跟人合作，強調(diào)人和AI系統(tǒng)之間的互補作用輔助人類的人工智能技術(shù)2. 開發(fā)增強人類的技術(shù)已成功：安靜環(huán)境下的流暢的語音識未解決的：噪聲環(huán)境下的識別，遠場語音識別，口音，兒童語音識別，受損語音識別，語言理解，對話能力3. 可視化， AI-人之間的友好界面4. 研發(fā)更有效的語言處理系統(tǒng)策略 III： 理解并重點關(guān)注人工智能

9、可能帶來的倫理， 法律， 社會方面的影響研究人工智能技術(shù)可能帶來的倫理， 法律，社會方面的影響期待其符合人的類規(guī)范AI系統(tǒng)從設(shè)計上需要符合人類的道德標準：公平，正義，透明，責任感策略 III： 理解并重點關(guān)注人工智能可能帶來的倫理， 法律， 社會方面的影響研究人工智能技術(shù)可能帶來的倫理， 法律，社會方面的影響期待其符合人的類規(guī)范AI系統(tǒng)從設(shè)計上需要符合人類的道德標準：公平，正義，透明，責任感2. 構(gòu)建符合道德的AI技術(shù)如何將道德量化， 由模糊變?yōu)榫_的系統(tǒng)和算法設(shè)計道德通常是模糊的，隨文化， 宗教和信仰而不同策略 III： 理解并重點關(guān)注人工智能可能帶來的倫理， 法律， 社會方面的影響研究人工

10、智能技術(shù)可能帶來的倫理， 法律，社會方面的影響期待其符合人的類規(guī)范AI系統(tǒng)從設(shè)計上需要符合人類的道德標準：公平，正義，透明，責任感2. 構(gòu)建符合道德的AI技術(shù)兩層架構(gòu)： 由一層專門負責道德建設(shè)道德標準植入每一個工程AI步驟3. 符合道德標準的AI技術(shù)的實現(xiàn)框架策略 - IV: 確保人工智能系統(tǒng)的自身和對周圍環(huán)境安全性在人工智能系統(tǒng)廣泛使用之前，必須確保系統(tǒng)的安全性研究創(chuàng)造穩(wěn)定， 可依靠，可信賴，可理解，可控制的人工智能系統(tǒng)所面臨的挑戰(zhàn)及解決辦法提升AI系統(tǒng) 的可解釋性和透明度2. 建立信任3. 增強verification 和 validation4. 自我監(jiān)控，自我診斷，自我修正5. 意外處

11、理能力， 防攻擊能力策略-V: 發(fā)展人工智能技術(shù)所需的共享的數(shù)據(jù)集和共享的模擬環(huán)境一件重要的公益事業(yè)， 同時要充分尊重企業(yè)和個人在數(shù)據(jù)中的權(quán)利和利益鼓勵開源策略-VI: 評價和評測人工智能技術(shù)的標準開發(fā)恰當?shù)脑u級策略和方法策略- VII: 更好的理解國家在人工智能研發(fā)方面的人力需求保證足夠的人才資源大數(shù)據(jù)和人工智能 數(shù)據(jù)是人工智能的來源 大數(shù)據(jù)并行計算，流計算等技術(shù)是人工智能能實用化的保障 人工智能是大數(shù)據(jù)， 尤其復雜數(shù)據(jù)分析的主要方法. Top 10 家技術(shù)公司的布局Google: AI-First StrategyGoogle 化4億美金購買英國倫敦大學人工智能創(chuàng)業(yè)公司：DeepMindA

12、lphaGoGNCWaveNetQ-Learning2011年成立1. 語音識別，合成 ； 2. 機器翻譯；3. 無人駕駛車. 4. 谷歌眼鏡. 5. Google Now. 6. 收購 Api.uiFacebook共享深度學習開源代碼：TorchFacbook M 數(shù)字助理研究和應(yīng)用：FAIR & AML Apple AIApple SiriApple bought Emotient and Vocal IQ?Partnership on AIIt will “conduct research, recommend best practices, and publish research u

13、nder an open license in areas such as ethics, fairness and inclusivity; transparency, privacy, and interoperability; collaboration between people and AI systems; and the trustworthiness, reliability and robustness of the technology” 2016年9月29日Elon Musk : OpenAIPaypal, Telsla, SpaceX , SolarCity 四家公司

14、CEO ， 投資十個億美金成立OpenAIMicrosoftIBM百度國內(nèi)技術(shù)巨頭騰訊， 阿里， 訊飛在人工智能領(lǐng)域投入巨大5. 深度學習在企業(yè)數(shù)據(jù)分析中的案例An example: AI in Data Analytics with Deep Learning 客戶情感分析Introduction Emotion Recognition in TextEmotion Recognition in SpeechEmotion Recognition in Conversations Industrial Application DatasetsFeaturesMethodsIntroducti

15、on: Interchangeable Terms Opinion MiningSentimental AnalysisEmotion RecognitionPolarityDetectionReviewMining42Introduction: What emotions are?43Introduction: Problem DefinitionWe will only focus on document level sentimentOpinion MiningRANLP 2015, Hissar, BulgariaIntroduction: Text Examples6th Septe

16、mber 2015a thriller without a lot of thrillsAn edgy thriller that delivers a surprising punchA flawed but engrossing thrillerIts unlikely well see a better thriller this yearAn erotic thriller thats neither too erotic nor very thrilling eitherEmotions are expressed artistically with help of Negation

17、 Conjunction Words Sentimental Words, e.g. 45RANLP 2015, Hissar, BulgariaIntroduction: Text ExamplesDSE: explicitly express an opinion holders attitudeESE: indirectly express the attitude of the writer6th September 2015Emotions are expressed explicitly and indirectly. 46RANLP 2015, Hissar, BulgariaI

18、ntroduction: Text Examples6th September 2015Emotions are expressed language that is often obscured by sarcasm, ambiguity, and plays on words, all of which could be very misleading for both humans and computers A sharp tongue does not mean you have a keen mind I dont know what makes you so dumb but i

19、t really works Please, keep talking. So great. I always yawn when I am interested.47RANLP 2015, Hissar, BulgariaIntroduction: Speech Conversation Examples6th September 201548RANLP 2015, Hissar, BulgariaIntroduction: Conversation Examples6th September 201549RANLP 2015, Hissar, BulgariaTypical Approac

20、h: A Classification Task6th September 2015A DocumentFeatures:Ngrams (Uni, bigrams)POS TagsTerm FrequencySyntactic Dependency Negation Tags SVMMaxentNave BayesCRFRandom ForestPosNeuNegSupervised LearningPos-Tag Patterns + Dictionary +Mutual InfoRulesUnsupervised Learning50RANLP 2015, Hissar, Bulgaria

21、Typical Approach: A Classification Task6th September 2015Features:Prosodic features:pitch, energy, formants, etc.Voice quality features: harsh, tense, breathy, etc.Spectral features: LPC, MFCC, LPCC, etc. Teager Energy Operator (TEO)-based features: TEO- FM-var, TEO-Auto-Env, etc SVMGMMHMMDBNKNN LDA

22、 CART PosNeuNegSupervised Learning51Challenges RemainText-Based: Capture the compositional effects with higher accuracyNegating Positive sentencesNegating Negative sentencesConjunction: Speech-Based:Effective features unknown. Emotional speech segments tend to be transcribed with lower ASR accuracyO

23、verviewIntroduction Emotion Recognition in TextWord Embedding for Sentiment AnalysisCNN for Sentiment ClassificationRNN, LSTM for sentiment ClassificationPrior Knowledge + CNN/LSTMParsing + RNN Emotion Recognition in SpeechEmotion Recognition in Conversations Industrial Application How deep learning

24、 can change the game?RANLP 2015, Hissar, Bulgaria6th September 2015Emotion Classification with Deep learning approaches54RANLP 2015, Hissar, Bulgaria1. Word Embedding as Features6th September 2015Representation of text is very important for performance of many real-world applications including emoti

25、on recognition: Local representations:N-gramsBag-of-words1-of-N codingContinuous Representations:Latent Semantic Analysis Latent Dirichlet Allocation Distributed Representations: word embeddingTomas Mikolov, “Learning Representations of Text using Neural Networks”, NIPs Deep learning Workshop 2013 (

26、Bengio et al., 2006; Collobert & Weston, 2008; Mnih & Hinton, 2008; Turian et al., 2010; Mikolov et al., 2013a;c)55RANLP 2015, Hissar, Bulgaria1. Word Embedding as Features6th September 2015Representation of text is very important for performance of many real-world applications including emotion rec

27、ognition: Local representations:N-gramsBag-of-words1-of-N codingContinuous Representations:Latent Semantic Analysis Latent Dirichlet Allocation Distributed Representations: word embeddingTomas Mikolov, “Learning Representations of Text using Neural Networks”, NIPs Deep learning Workshop 201356RANLP

28、2015, Hissar, BulgariaWord Embedding6th September 2015Skip-gram ArchCBOW The hidden layer vector is the word-embedding vector for w(t) 57Word Embedding for Sentiment Detection It has been widely accepted as standard features for NLP applications including sentiment analysis since 2013 Mikolov 2013Th

29、e word vector space implicitly encodes many linguistic regularities among words: semantic and syntactic Example: Google Pre-trained word vectors with 1000Billion words Does it encode polarity similarities? great0.729151bad0.719005terrific0.688912decent0.683735nice0.683609excellent0.644293fantastic0.

30、640778better0.612073solid0.580604lousy0.576420 wonderful0.572612terrible0.560204Good0.558616Top Relevant Words to “good”Mostly Yes, but it doesnt separate antonyms well RANLP 2015, Hissar, BulgariaLearning Sentiment-Specific Word Embedding6th September 2015Tang, et al, “Learning Sentiment Specific W

31、ord Embedding for Twitter Sentiment Classification”, ACL 201459RANLP 2015, Hissar, BulgariaLearning Sentiment-Specific Word Embedding6th September 2015Tang, et al, “Learning Sentiment Specific Word Embedding for Twitter Sentiment Classification”, ACL 2014In Spirit, it is similar to multi-task learni

32、ng. It learns the same way as the regular word-embedding with loss function considering both semantic context and sentiment distance to the twitter emotion symbols. 6010 million tweets selected by positive and negative emoticons as training dataThe Twitter sentiment classification track of SemEval 2

33、013Learning Sentiment-Specific Word EmbeddingTang, et al, “Learning Sentiment Specific Word Embedding for Twitter Sentiment Classification”, ACL 2014Paragraph VectorsLe and Mikolov, “Distributional Representations of Sentences and Documents, ICML 2014 Paragraph vectors are distributional vector repr

34、esentation for pieces of text, such as sentences or paragraphsThe paragraph vectors are also asked to contribute to the prediction task of the next word given many contexts sampled from the paragraph.Each paragraph corresponds to one column in DIt acts as a memory remembering what is missing from th

35、e current context , about the topic of the paragraph Paragraph Vectors Best Results on MR Data SetLe and Mikolov, “Distributional Representations of Sentences and Documents, ICML 2014 OverviewIntroduction Emotion Recognition in TextWord Embedding for Sentiment AnalysisCNN for Sentiment Classificatio

36、nRNN, LSTM for sentiment ClassificationPrior Knowledge + CNN/LSTMDataset CollectionEmotion Recognition in SpeechEmotion Recognition in Conversations Industrial Application CNN for Sentiment Classification Ref: Yoon Kim. Convolutional Neural Networks for Sentence Classification. EMNLP, 2014.CNN for S

37、entiment Classification Ref: Yoon Kim. Convolutional Neural Networks for Sentence Classification. EMNLP, 2014. A simple CNN with One Layer of convolution on top of word vectors. Motivated by CNN has been successful on many other NLP tasksInput Layer: Word vectors are from pre-trained Google-News wor

38、d2vector Conv Layer: Window size: 3 words, 4 words, 5 words. Each with 100 feature map. 300 features in the penultimate layerPooling Layer: Max Over time Pooling at the Output layer: Fully connected softmax layer , output distribution over labelsRegularization: Drop-out on the penultimate layer with

39、 a constrain on the l2 norms of the weight vectors Fine-train embedding vectors during training Common Datasets CNN for Sentiment Classification - Results CNN-rand: Randomly initialize all word embeddingsCNN-static: word2vec, keep the embeddings fixedCNN-nonstatic: Fine-tuning embedding vectorsCNN f

40、or Sentiment Classification - Results Why it is successful?Multiple filters and multiple feature mapsEmotions are expressed in segments, instead of the spanning over the whole sentence Use pre-trained word2vec vectors as input features . Embedding word vectors are further improved for non-static tra

41、ining. Antonyms are further separated after training. Resources for This work Source Code: https:/yoonkim/CNN_sentenceImplementation in Tensorflow: /dennybritz/cnn-text-classification-tftfExtensive Experiments:/pdf/1510.03820v4.pdfpdfDynamic CNN for Sentiment Kalchbrenner et al, “A Convolutional Neu

42、ral Network for Modeling Sentences”, ACL 2014 Hyper Parameters in Experiments: K=4m=5, 14 feature mapsm=7, 6 feature mapsd=48 Dynamic CNN for Sentiment Kalchbrenner et al, “A Convolutional Neural Network for Modeling Sentences”, ACL 2014 One Dimension Convolution Two Dimension Convolution48 D word v

43、ectors randomly initiated 300 D Initiated with Google word2vectorMore complicated model architecture with dynamic poolingStraight Forward 6, 4 feature maps100-128 feature mapsJohnson and Zhang. , “Effective Use of Word Order for Text Categorization with Convolutional Neural Networks”, ACL-2015Why CN

44、N is effective A simple remedy is to use word bi-grams in addition to unigramsIt has been noted that loss of word order caused by bag-of-word vectors (bow vectors) is particularly problematic on sentiment classificationComparing SVM with Tri-gram features with 1, 2,3 window filter CNNTop 100 Feature

45、sSVMCNNUni-Grams687Bi-Grams2833Tri-Grams460SVMs cant fully take advantage of high-order ngramsSentiment Classification Considering Features beyond Text with CNN ModelsTang et al. , “Learning Semantic Representations of Users and Products for Document Level Sentiment Classification“”, ACL-2015Overvie

46、wIntroduction Emotion Recognition in TextWord Embedding for Sentiment AnalysisCNN for Sentiment ClassificationRNN, LSTM for sentiment ClassificationPrior Knowledge + CNN/LSTMDataset Collection Emotion Recognition in SpeechEmotion Recognition in Conversations Industrial Application Recursive Neural T

47、ensor NetworkSocher et al. , “Recursive Deep Models for Semantic Compositionality over a Sentiment Treebank”, EMNLP-2013 . /sentiment/The Stanford Sentiment Treeback is a corpus with fully labeled parse treesCreated to facilitate analysis of the compositional effects of sentiment in language 10,662

48、sentences from movie reviews. Parsed by stanford parser. 215,154 phrases are labeledA model called Recursive Neural Tensor Networks was proposed Recursive Neural Tensor Network- Distribution of sentiment values for N-gramsSocher et al. , “Recursive Deep Models for Semantic Compositionality over a Se

49、ntiment Treebank”, EMNLP-2013 . /sentiment/Stronger sentiment often builds up in longer phrases and the majority of the shorter phrases are neutral Recursive Neural Tensor Network (RNTN)Socher et al. , “Recursive Deep Models for Semantic Compositionality over a Sentiment Treebank”, EMNLP-2013 . /sen

50、timent/f = tanhV is the tensor directly relate input vectors , W is the regular RNN weight matrixWang et al. , “Predicting Polarities of Tweets by Composing Word Embedding with Long Short-Term Memory”, ACL-2015LSTM for Sentiment AnalysisLSTM works tremendously well on a large number of problemsSuch

51、architectures are more capable to learn a complex composition such as negation of word vectors than simple RNNs .Input, stored information, and output are controlled by three gates.Wang et al. , “Predicting Polarities of Tweets by Composing Word Embedding with Long Short-Term Memory”, ACL-2015LSTM f

52、or Sentiment AnalysisDataset: the Stanford Twitter Sentiment corpus (STS)LSTM-TLT: Word-embedding vectors as input. TLT: Trainable Look-up Table It is observed that negations can be better captured.Tang et al. , “Document Modeling with Gated Recurrent Neural Network for Sentiment Classification”, EM

53、NLP-2015Gated Recurrent UnitTang et al. , “Document Modeling with Gated Recurrent Neural Network for Sentiment Classification”, EMNLP-2015Gated Recurrent Neural NetworkUse CNN/LSTM to generate l sentence representations from word vectors Gate Recurrent Neural Network (GRU) to encode sentence relatio

54、ns for sentiment classification GRU can viewed as variant of LSTM , with output gate always on Tang et al. , “Document Modeling with Gated Recurrent Neural Network for Sentiment Classification”, EMNLP-2015Gated Recurrent Neural NetworkJ. Wang et al., Dimensional Sentiment Analysis Using a Regional C

55、NN-LSTM Model”, ACL-2016CNN-LSTMJ. Wang et al., Dimensional Sentiment Analysis Using a Regional CNN-LSTM Model”, ACL-2016CNN-LSTMThe dimensional approach represents emotional states as continuous numerical values in multiple dimensions such as the valence-arousal (VA) space (Russell, 1980). The dime

56、nsion of valence refers to the degree of positive and negative sentiment, whereas the dimension of arousal refers to the degree of calm and excitementK.S Tai et al, Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks”, ACL-2015Tree-LSTMTree-LSTM： a generalization o

57、f LSTMs to tree-structured network topologies. TreeLSTMs outperform all existing systems and strong LSTM baselines on two tasks: predicting the semantic relatedness of two sentences (SemEval 2014, Task 1) and sentiment classification (Stanford Sentiment Treebank).K.S Tai et al, Improved Semantic Rep

58、resentations From Tree-Structured Long Short-Term Memory Networks”, ACL-2015Tree-LSTMAchieve comparable accuracyConstituency-Tree based performs betterThe word vectors are initialized by Glove Word2Vectors (Trained on 840 billion tokens of Common Crawl data, /projects/glove/)OverviewIntroduction Emo

59、tion Recognition in TextWord Embedding for Sentiment AnalysisCNN for Sentiment ClassificationRNN, LSTM for sentiment ClassificationPrior Knowledge + CNN/LSTMDataset Collection Emotion Recognition in SpeechEmotion Recognition in Conversations Industrial Application RANLP 2015, Hissar, BulgariaPrior K

60、nowledge + Deep Neural Networks6th September 2015For each iteration: The teacher network is obtained by projecting the student network to a rule-regularized subspace (red dashed arrow);The student network is updated to balance between emulating the teachers output and predicting the true labels (bla