Long Short Term Memory (LSTM)

Top PDF Long Short Term Memory (LSTM):

Long short term memory on chronic laryngitis classification

Long short term memory on chronic laryngitis classification

The classification study with the use of machine learning concepts has been applied for years, and one of the aspects in which this can be applied is for the analysis of speech acoustics applied to the analysis of pathologies. Among the pathologies present, one of them is chronic laryngitis. Thus, this article aims to present the results for a classification of chronic laryngitis with the use of Long Short Term Memory as a classifier. The parameters of relative jitter, relative shimmer and autocorrelation was used as input of the LSTM. A dataset of about 1500 instances were used to train, validate and test along 4 experiments with LSTM and one feedforward Artificial Neural Network (ANN). The results of the LSTM overcome the ones of the feedforward ANN, and was about 100% accuracy, sensitivity and specificity in test set, denoting a promising future for this classification tool in the voice pathologies diagnose.
Mostrar mais

8 Ler mais

Gerando letras musicais utilizado uma rede neural recorrente LSTM - Long Short-Term Memory.

Gerando letras musicais utilizado uma rede neural recorrente LSTM - Long Short-Term Memory.

The use of machine learning has been used for various purposes, including the generation of various types of art, such as visual, sound or textual. The use of these learning algorithms makes it possible to develop natural language generation applications. Problems of natural language generation mostly use recurrent neural networks, a well-researched subject, with much proof of concepts and variations. However, a more specific application for this problem is song lyrics. In this work, a method was presented and the results of the automatic generation of lyrics using a Long Short-Term Memory (LSTM) network were discussed. For the creation of the datasets a crawler developed for this work was used, that captured 562 songs of 3 artists of different styles. The experiments were performed separately with each style. The results of the experiments showed a different grammatical structure with the RAP corpus, which began to rhyme in some of the generated sentences. It was also possible to observe the generation of some acceptable musical sentences.
Mostrar mais

47 Ler mais

Previsão de séries temporais financeiras: uma abordagem com Long Short-term Memory Deep Neural Networks

Previsão de séries temporais financeiras: uma abordagem com Long Short-term Memory Deep Neural Networks

Esta tese terá um carácter bastante inovador, principalmente a nível nacional, pois tem um tema pioneiro ao nível de mestrado, sendo o seu objetivo principal comparar os métodos tradicionais de previsão com novas metodologias de tipo LSTM (Long Short Term Memory) para Deep Learning, para dados financeiros, e onde toda a parte computacional foi desenvolvida na linguagem Python. A série temporal em estudo é proveniente da bolsa americana, nomeadamente o índice Standard & Poor’s 500.

107 Ler mais

A Comparison on Statistical Methods and Long Short Term Memory Network Forecasting the Demand of Fresh Fish Products

A Comparison on Statistical Methods and Long Short Term Memory Network Forecasting the Demand of Fresh Fish Products

A Long Short Term Memory(LSTM) is a new type of RNN, that contains information outside the normal flow of the recurrent network in a gated cell. Information can be stored in, written to, or read from a cell, much like data in a computer’s memory. The cell makes decisions about what to store, and when to allow reads, writes and erasures, via gates that open and close. Those gates act on the signals they receive, and similar to the neural network’s nodes, they block or pass on information based on its strength and import, which they filter with their own sets of weights. Those weights, like the weights that modulate input and hidden states, are adjusted via the recurrent networks learning process. That is, the cells learn when to allow data to enter, leave or be deleted through the iterative process of making guesses.
Mostrar mais

58 Ler mais

Modelo para predição dos ativos totais de uma cooperativa de crédito baseado em redes long short-term memory (LSTM)

Modelo para predição dos ativos totais de uma cooperativa de crédito baseado em redes long short-term memory (LSTM)

Predict trends or future informations were being even more important to the development of the communities that we live. The total assets from a traditional financial institution or a credit union are composed and influenced by various numbers and indexes, that by your time are also influenced by other factores. The complexity to understand the growth trend of the total assets through all those influences create an opportunity to apply predict models based on Artificial Neural Networks to look for precisely results. With this goal, a predict model was projected based on Artificial Neural Networks Long Short-Term Memory (LSTM) to predict the future value of the total assets from a credit union. The predict model was trained from real data provided by a Credit Union. For the production of the experimental results, two samples of the same database were used, having as difference the reduction of 8 indexes in the second sample. The obtained results were positive, reaching values very close to the real ones. A comparison was also made between the results of the two samples of the database, with the goal to understand the importance of the indexes that have been reduced in the second sample.
Mostrar mais

57 Ler mais

Customized Hardware for Long-Short Term Memory Networks in Embedded Systems

Customized Hardware for Long-Short Term Memory Networks in Embedded Systems

Considering this, the present work proposes the development of an LSTM network for an FPGA platform using HLS tools. Because of the large size of LSTM networks in comparison with the on-chip memory available in an FPGA, off-chip memory storage of the input pairs and the weight matrices is proposed. To avoid dealing with large latencies derived from exclusively using off-chip memory, a block-batching technique is presented, which performs computations over a batch of input pairs, and a block of weight matrices, which are buffered from off-chip into the on- chip memory using double-buffers. For obtaining a higher level of parallelism and the reuse of the elements instantiated on fabric, the implementation of the most computationally-intensive blocks is performed using pipelines. Furthermore, it is possible to vary the topology of the network by using a number of parameters for that purpose. The word bit-widths can also be adjusted according to the needs of the system.
Mostrar mais

79 Ler mais

Differential role of entorhinal and hippocampal nerve growth factor in short- and long-term memory modulation

Differential role of entorhinal and hippocampal nerve growth factor in short- and long-term memory modulation

Figure 2. Effects of nerve growth factor (NGF) infusion into the entorhinal cortex on short- (A) and long-term (B) memory in a one-trial step-down inhibitory avoidance task. Data are reported as median (inter- quartile range) latency (in sec- onds) in training and test ses- sions. N = 11-13 per group. *P < 0.05 compared with the vehicle- treated group (Mann-Whitney U-test). All groups showed sig- nificant training-test differences (P < 0.01, Wilcoxon test).

4 Ler mais

Short and long term investor synchronization caused by decoupling.

Short and long term investor synchronization caused by decoupling.

The participants were first informed about the details of the experiment and expressed informed consent to participate. Then they were given instructions concerning the $G. They were told that in each time step, their behavior influences the price. They had no time limit for making their decisions, but they knew that at Figure 2. Speculative biases in the price movements - indicated by circles - in experiments with human subjects. The onset of the speculative bias was subsequently detected in Monte Carlo simulations with agents of the $-game trading on the price data generated by the humans. The rather sharp transition in the splitting of solid and dotted lines over time (for definitions of the different lines see caption 1) can be used to mark the onset of the speculative bias before it is visible in the price history. The lengths of memory used in experiments with human subjects were m = 3 (graph A and B) and m = 6 (graph C).
Mostrar mais

8 Ler mais

Dancing bees improve colony foraging success as long-term benefits outweigh short-term costs.

Dancing bees improve colony foraging success as long-term benefits outweigh short-term costs.

A closer examination of our model and the existing literature suggests that our results of long term benefits of the dance are plausible. Recruitment in honeybees is costly [21,23,24]: recruits need to wait for a dancing bee [21], and they usually require several field excursions before locating the advertised food source after following a dance [23,58–60]. Thus, potential recruits incurred both energy and opportunity costs in our model. But after the advertised food source is located, the energetic returns are higher than for scouts. This result is consistent with empirical studies, which found that recruits tend to discover food sources of higher quality than scouts [23,24,60]. By dancing only for high quality food sources (see Figure S1 and [3,8]) foraging bees effectively filter information, which allows recruits to exploit selectively the best food sources known to the colony [3,61,62]. After successful recruitment, costs of continuing to visit the food patch are low as bees quickly locate the previously visited patches using route-memory [63,64], while benefits potentially remain high. The more stable the environment, the longer these benefits
Mostrar mais

9 Ler mais

The Effect of Zinc Supplementation of Lactating Rats on Short-Term and Long-Term Memory of Their Male Offspring

The Effect of Zinc Supplementation of Lactating Rats on Short-Term and Long-Term Memory of Their Male Offspring

Some scientists reported that enhanced zinc consumption caused memory deficits and increased brain levels of zinc. It is shown that the influx of toxic amounts of zinc to post-synaptic neurons was mainly responsi- ble for the neurodegenerative process 19 .

4 Ler mais

Interplay between short- and long-term plasticity in cell-assembly formation.

Interplay between short- and long-term plasticity in cell-assembly formation.

Although we pursued biologically plausibility in the present modeling, some assumptions of the model remain to be confirmed by experiment. We assumed that LTD of excitatory synapses has a logarithmic weight dependence, implying that synaptic weights only sublinearly influence the LTD of strong synapses. However, the weight dependence for strong synapses is still unknown. We also implicitly assumed that synaptic weights are solely modified by STDP and homeostatic plasticity within 30 minutes to 1 hour from the application of external stimuli and molecular processes for the consolidation of memory trace occur later. However, the actual synaptic mechanism of memory consolidation is more complicated and remains elusive [21]. In fact, cell assemblies could not be permanently stable in the present model with STD and STDP. Therefore, how these cell assemblies may be maintained in a longer time scale remains open for further theoretical studies. In addition, some predictions of the model should be examined by experiment. Synaptic weights displayed large fluctuations in Figure 3D, which has not been observed in previous experiments. The large-amplitude fluctuations were partly due to our choice of a relatively large learning rate and partly due to the inherent nature of the present log-STDP model. Nevertheless, these fluctuations are unlikely to be harmful to the practical function
Mostrar mais

16 Ler mais

Identification of a functional connectome for long-term fear memory in mice.

Identification of a functional connectome for long-term fear memory in mice.

One interesting observation was that the same (or very similar) behavioral output was associated with distinct functional networks, suggesting that there is a degree of degeneracy at the network level (i.e., an ability of distinct constellations of neural elements to produce same output [61,62]). This was the case for networks engaged by recall of contextual fear memory at the short vs. long delay in WT mice. Likewise, a-CaMKII +/2 mice exhibited equivalent levels of conditioned freezing compared to WT littermates at the short delay, yet underlying functional networks differed considerably. Together, this suggests that a population of network solutions may support contextual fear memory, and raise two related issues. First, because of this degeneracy, pre-training manipulations of key brain regions may not necessarily prevent the formation of a contextual fear memory. For example, pre-training lesions of the hippocampus surprisingly do not always impair acquisition of a contextual fear memory [63,64], and this is presumably because alternate networks can support this type of memory in the absence of an intact hippocampus at the time of training. If multiple networks can support the formation of a contextual fear memory, then a second issue is whether all networks perform equivalently under all conditions? In the case of animals trained without a hippocampus this does not seem to be the case. While hippocampal lesioned animals may form a contextual fear memory, acquisition is less efficient [64], and the resultant memory is less precise [63] and less durable [65]. Similarly, in our study, the a-CaMKII +/2 mice were able to form a contextual fear memory, but this memory faded rapidly over time. This highlights that not all degenerate solutions necessarily have the same robustness, although it remains to be determined which specific alterations in network organization in a-CaMKII +/2 mice might be causally related to premature forgetting.
Mostrar mais

18 Ler mais

Self-construal priming affects speed of retrieval from short-term memory.

Self-construal priming affects speed of retrieval from short-term memory.

Several researchers have demonstrated self-construal priming effects in long-term memory (LTM) as well. Wang and Ross [17] reported self-priming effects on retrieval of autobiographical memories from LTM. Participants that received the collective prime tended to retrieve memories of routine social situations of a collective nature, and those that received the individual prime tended to retrieve events centered on themselves. Sui, Zhu, and Chiu [18] examined the effects of cultural priming on recognition memory using a sample of Chinese participants. Subjects were primed with images representative of Chinese or American culture, which effectively served as collective and individual primes, respectively. The participants were then shown a series of adjectives and asked to rate each in terms of its applicability to either the subject or the subject’s mother. After a one-hour delay, participants completed a surprise recognition task in which they viewed a list of adjectives and indicated which were encountered
Mostrar mais

4 Ler mais

Participation of the classical speech areas in auditory long-term memory.

Participation of the classical speech areas in auditory long-term memory.

Accumulating evidence suggests that storing speech sounds requires transposing rapidly fluctuating sound waves into more easily encoded oromotor sequences. If so, then the clas- sical speech areas in the caudalmost portion of the temporal gyrus (pSTG) and in the inferi- or frontal gyrus (IFG) may be critical for performing this acoustic-oromotor transposition. We tested this proposal by applying repetitive transcranial magnetic stimulation (rTMS) to each of these left-hemisphere loci, as well as to a nonspeech locus, while participants listened to pseudowords. After 5 minutes these stimuli were re-presented together with new ones in a recognition test. Compared to control-site stimulation, pSTG stimulation produced a highly significant increase in recognition error rate, without affecting reaction time. By contrast, IFG stimulation led only to a weak, non-significant, trend toward recognition memory im- pairment. Importantly, the impairment after pSTG stimulation was not due to interference with perception, since the same stimulation failed to affect pseudoword discrimination ex- amined with short interstimulus intervals. Our findings suggest that pSTG is essential for transforming speech sounds into stored motor plans for reproducing the sound. Whether or not the IFG also plays a role in speech-sound recognition could not be determined from the present results.
Mostrar mais

15 Ler mais

Computational models of memory consolidation and long-term synaptic plasticity during sleep

Computational models of memory consolidation and long-term synaptic plasticity during sleep

The synaptic memory doctrine postulates that molecular modifications in the synapse strength allow the brain to store, maintain and retrieve memories ( Lisman, 2017a ) ( Takeuchi, Duszkiewicz, & Morris, 2014 ). Such a hypothesis is alternative to the idea that memories are stored as changes in the cellular properties of neurons ( Mozzachiodi & Byrne, 2010 ). Synapse strength is proportional to synapse size and exhibits an approximate 10-fold gradation ( Liu, Hagan, & Lisman, 2017 ), granting ample space for storing information. For this reason, the brain implements many di fferent mechanisms of synaptic plasticity ( Citri & Malenka, 2008 ). A standard protocol to observe changes in synaptic strength consists in comparing the amplitude of evoked post-synaptic potential (EPSP) produced with electrical stimulations of the same magnitude on two different occasions ( Nicoll & Malenka, 1999 ). Changes can be short-lived as in the processes of neural facilitation ( Del Castillo & Katz, 1954 ), synaptic augmentation ( Magleby & Zengel, 1976 ), synaptic depression ( Tsodyks & Markram, 1997 ), post- tetanic potentiation ( Bao, Kandel, & Hawkins, 1997 ) or synaptic fatigue ( Armbruster & Ryan, 2011 ). However, within the synaptic memory doctrine, one would be focused on the long-lasting changes of synaptic strength, termed Long-term potentiation (LTP) and Long-term depression (LTD) for positive and negative changes, respectively ( Bliss & Collingridge, 1993; Morris, 2003 ). The causal link between memories and LTP/LTD has been established by experiments that observed the cellular signatures of LTP after learning ( Whitlock, Heynen, & Shuler, 2006 ) and that could evoke or inhibit a memory trace by producing LTP and LTD ( Nabavi et al., 2014 ). Therefore, it is reasonable to consider that long-term synaptic changes have a direct impact on memory.
Mostrar mais

16 Ler mais

Food restriction increases long- term memory persistence in adult or aged mice

Food restriction increases long- term memory persistence in adult or aged mice

Although the effects of FR had been extensively studied during aging, its beneficial effects on memory formation in adulthood are little elucidated. Currently, there is no direct evidence from human studies showing the protective effect of either short-term or long-term FR on cognition in adults (Kretsch et al., 1997; Martin et al., 2007). Thus, while the vast majority of studies only evaluate the effects of a long- term FR protocol, we aimed to investigate the effects of FR for 12 h or 12 h/day for 2 days on learning and memory in adult and aged mice. These protocols occur more commonly in modern society, thus representing a better translational model. In addition, shortening the FR duration would constitute an advantage in clinical practice. Specifically, we investigated the influence of FR on an aversive mem- ory formation (acquisition, consolidation and retrieval) evaluated in the plus-maze discriminative avoidance task (PM-DAT). The PM-DAT was used to concomitantly evaluate learning, memory, anxiety-like behavior, and motor activity, as previously described (Kameda et al., 2007; Patti et al., 2006, 2010; Sanday et al., 2012, 2013a,b,c; Silva and Frussa-Filho, 2000).
Mostrar mais

12 Ler mais

Verbal Short-Term Memory Underlies Typical Development of “Thought Organization” Measured as Speech Connectedness

Verbal Short-Term Memory Underlies Typical Development of “Thought Organization” Measured as Speech Connectedness

The present study aims to further explore the poten- tial of speech graph analysis as a tool for the screening of learning difficulties in the school environment. As a first step, it is necessary to more deeply understand the relation- ship between speech graph attributes and cognitive devel- opment. Two prior findings (Mota et al., 2016) established the grounds for the present study. First, it was found that nonverbal IQ and ToM scores could not explain the rela- tionship between the LSC for the story narratives and read- ing (Mota et al., 2016). This result led us to speculate which other factors could explain this relationship. Secondly, an association was found between speech connectedness and reading only for the short-term memory reports, and not for long-term memory reports based on memories from days to years before the interview (Mota et al., 2016). Building on these initial observations, we hypothesized that speech con- nectedness might be related to short-term memory or/and working memory—particularly in the verbal domain. We posited that the ability to store and/or simultaneously pro- cess a large amount of information for a short period of time would be required to plan well-organized discourse, as indexed by a large LSC in a speech graph. Thus, we pre- dicted that graph connectedness and reading fluency would be correlated, replicating prior results with another measure of reading ability (the time spent to read a list of words and pseudo-words), and that this relation would be mediated by measures of verbal (but not visuo-spatial) short-term and/or working memory.
Mostrar mais

10 Ler mais

On the contribution of binocular disparity to the long-term memory for natural scenes.

On the contribution of binocular disparity to the long-term memory for natural scenes.

The long-term memory for scenes, especially if enough processing time is available, is largely mediated by their semantic content. Observers can quickly build a conceptual representation of the scene [3] and, if enough time is available for encoding (e.g., [4]), this representation is consolidated in short-term memory and eventually transferred to long-term memory. A strong evidence for the conceptual encoding of scenes comes from the fact that observers are more likely to produce false recognitions when they encounter a scene conceptually related to the memorized one [5]. Nonetheless, visual memory for scenes has been shown to be resistant to interference ([6], but see [7]), and above all, there is evidence that observers can recognize specific instances of objects within a category even after learning thousands of items, visual long-term memory is thus potentially quite detailed [8]. The visual and conceptual codes for natural images coexist in long-term memory and have similar decay times, as demonstrated by the interference effects of visually and conceptually related distractors [9]. As far as scenes are concerned, the question arises as to what specific visual features are stored in memory and to what extent they contribute to the successful recognition of the scene.
Mostrar mais

10 Ler mais

Short- and Long-term memory: Differential involvement of neurotransmitter systems and signal transduction cascades

Short- and Long-term memory: Differential involvement of neurotransmitter systems and signal transduction cascades

At this stage it is safe to say that STM and LTM pertain to and are regulated by separate subsys- tems of the brain, which belong in some cases to the same and in others to different brain structures (see Izquierdo et al. 1999), and involve a great variety of molecular mechanisms at the receptor and post- receptor level, some of which may be linked. This fits with modern concepts of memory organisation (Fuster 1998, Izquierdo & Medina 1997, Izquierdo et al. 1997), which supersede old phrenological con- cepts based on lesion studies (i.e., “hippocampal” as opposed to, say “amygdala-dependent” tasks, see references (Fuster 1998). All types of memory de- pend on the integrated activity of various brain sites and involve more than one receptor or post-receptor mechanism (Izquierdo & Medina 1997, Izquierdo et al. 1999, 2000).
Mostrar mais

12 Ler mais

Errors in nonword repetition: bridging short- and long-term memory

Errors in nonword repetition: bridging short- and long-term memory

According to the working memory model, the phonological loop is the component of working memory specialized in processing and ma- nipulating limited amounts of speech-based information. The Children’s Test of Nonword Repetition (CNRep) is a suitable measure of phonological short-term memory for English-speaking children, which was validated by the Brazilian Children’s Test of Pseudoword Repetition (BCPR) as a Portuguese-language version. The objectives of the present study were: i) to investigate developmental aspects of the phonological memory processing by error analysis in the nonword repetition task, and ii) to examine phoneme (substitution, omission and addition) and order (migration) errors made in the BCPR by 180 normal Brazilian children of both sexes aged 4-10, from preschool to 4th grade. The dominant error was substitution [F(3,525) = 180.47; P < 0.0001]. The performance was age-related [F(4,175) = 14.53; P < 0.0001]. The length effect, i.e., more errors in long than in short items, was observed [F(3,519) = 108.36; P < 0.0001]. In 5-syllable pseudo- words, errors occurred mainly in the middle of the stimuli, before the syllabic stress [F(4,16) = 6.03; P = 0.003]; substitutions appeared more at the end of the stimuli, after the stress [F(12,48) = 2.27; P = 0.02]. In conclusion, the BCPR error analysis supports the idea that phonological loop capacity is relatively constant during development, although school learning increases the efficiency of this system. Moreover, there are indications that long-term memory contributes to holding memory trace. The findings were discussed in terms of distinctiveness, clustering and redintegration hypotheses.
Mostrar mais

15 Ler mais

Show all 3919 documents...