My personal project is to build a system that forecasts the risk of avalanches in the mountains given weather data. So, input is a time series of NOAA weather (precipitation, wind, temperature) data. The target is 4 years of human predicted avalanche dangers.
As background, the snow pack in the mountain is composed of layers of the current seasons snowfall. Each snowfall leaves a distinct layer, like rings on a tree or the grand canyon. A 2-month old layer can become a weak foundation and behave like ball bearings. A blizzard comes along and loads this trigger with feet of snow and now you have high avalanche danger.
Since the danger is caused partially by something that happened in the past, I settled on a recurrent neural network ( rnn .) The NN needs to have a memory of all the layers in the snow pack. Using PyBrain, I got no results. Random guessing would be much better.
Thus, I started on a simpler rnn problem: predicting Reber grammer strings. My code is below, and again it doesn't work. Am I on the right track? I’m trying to learn from examples, but I’m missing something.
"""An example of a recurrent neural network (that doesn't work) to predict a Reber Grammer""" import csv, urllib from pybrain.datasets.supervised import SupervisedDataSet from pybrain.supervised import BackpropTrainer from pybrain.tools.shortcuts import buildNetwork from pybrain.structure import RecurrentNetwork from pybrain.structure import LinearLayer, SigmoidLayer from pybrain.structure import FullConnection # More information on Reber Grammar can be found at http://cogsci.ucd.ie/Connectionism/Exercises/Exercise3.php f=urllib.urlopen('http://cogsci.ucd.ie/Connectionism/Labs/basicProp/reber.pat','r') #--- Reading Reber Grammar data from a website. The characters have already been transformed into binary vectors. for i in range(4): #Skipping 4 rows of header information f.readline() reader = csv.reader(f,delimiter=' ',quoting=csv.QUOTE_NONE) header = [] records = [] fields = 14 #---- Feeding data in PyBrain dataset trndata = SupervisedDataSet(7, 7) for row, record in enumerate(reader): record = [int(i) for i in record if i<>''] indata = tuple(record[:7]) outdata = tuple(record[7:]) trndata.addSample(indata,outdata) #---- Building Network n = RecurrentNetwork() n.addInputModule(LinearLayer(7, name='in')) n.addModule(SigmoidLayer(7, name='hidden')) n.addOutputModule(LinearLayer(7, name='out')) n.addConnection(FullConnection(n['in'], n['hidden'], name='c1')) n.addConnection(FullConnection(n['hidden'], n['out'], name='c2')) n.addRecurrentConnection(FullConnection(n['hidden'], n['hidden'], name='c3')) n.sortModules() #---- Training Network t = BackpropTrainer(n,learningrate=0.01,momentum=0.5,verbose=True) t.trainOnDataset(trndata,5) #---- Testing by feeding ouput into input. Should create Reber Grammer string grammer, # but instead gets stuck on one character. n.reset() input = [1]+[0]*6 for i in range(100): print input input = n.activate(input) input = [int(x>=max(input)) for x in input] #Assumes that the chosen character is the one with the highest value [link][13 comments]