Experimental Haskell machine learning library
Go to file
Mahdi Dibaiee 6b9cb1fa3e relu: run notmnist using relu activation and draw the chart
[wip] word2vec: work in progress implementation of word2vec
2016-09-13 09:49:44 +04:30
app feat(crossEntropy): crossEntropy cost function 2016-07-24 10:48:04 +04:30
examples relu: run notmnist using relu activation and draw the chart 2016-09-13 09:49:44 +04:30
profiling fix(naivebayes): fix the algorithm to make it actually work 2016-08-05 23:54:36 +04:30
src relu: run notmnist using relu activation and draw the chart 2016-09-13 09:49:44 +04:30
test fix(stack): use stack build and exec instead of manual stack ghc 2016-07-18 16:33:34 +04:30
.gitignore Initial commit 2016-07-17 16:52:16 +04:30
.gitmodules feat(notmnist): notmnist example using SGD + learning rate decay 2016-09-10 00:36:15 +04:30
LICENSE fix(stack): use stack build and exec instead of manual stack ghc 2016-07-18 16:33:34 +04:30
notmnist-0.png feat(notmnist): notmnist example using SGD + learning rate decay 2016-09-10 00:36:15 +04:30
notmnist-1.png feat(notmnist): notmnist example using SGD + learning rate decay 2016-09-10 00:36:15 +04:30
notmnist-2.png feat(notmnist): notmnist example using SGD + learning rate decay 2016-09-10 00:36:15 +04:30
notmnist-relu.png relu: run notmnist using relu activation and draw the chart 2016-09-13 09:49:44 +04:30
notmnist.png relu: run notmnist using relu activation and draw the chart 2016-09-13 09:49:44 +04:30
README.md relu: run notmnist using relu activation and draw the chart 2016-09-13 09:49:44 +04:30
Setup.hs initial commit, still work in progress 2016-07-17 16:53:13 +04:30
sgd.png relu: run notmnist using relu activation and draw the chart 2016-09-13 09:49:44 +04:30
sibe.cabal relu: run notmnist using relu activation and draw the chart 2016-09-13 09:49:44 +04:30
stack.yaml feat(notmnist): notmnist example using SGD + learning rate decay 2016-09-10 00:36:15 +04:30

sibe

A simple Machine Learning library.

Simple neural network

    let a = (sigmoid, sigmoid') -- activation function
        -- random network, seed 0, values between -1 and 1,
        -- two inputs, two nodes in hidden layer and a single output
        rnetwork = randomNetwork 0 (-1, 1) 2 [(2, a)] (1, a)

        -- inputs and labels
        inputs = [vector [0, 1], vector [1, 0], vector [1, 1], vector [0, 0]]
        labels = [vector [1], vector [1], vector [0], vector [0]]

        -- define the session which includes parameters
        session = def { network = rnetwork
                      , learningRate = 0.5
                      , epochs = 1000
                      , training = zip inputs labels
                      , test = zip inputs labels
                      } :: Session

        initialCost = crossEntropy session

    -- run gradient descent
    -- you can also use `sgd`, see the notmnist example
    newsession <- run gd session

    let results = map (`forward` newsession) inputs
        rounded = map (map round . toList) results

        cost = crossEntropy newsession
    
    putStrLn $ "- initial cost (cross-entropy): " ++ show initialCost
    putStrLn $ "- actual result: " ++ show results
    putStrLn $ "- rounded result: " ++ show rounded
    putStrLn $ "- cost (cross-entropy): " ++ show cost

Examples

# neural network examples
stack exec example-xor
stack exec example-424
# notMNIST dataset, achieves ~87.5% accuracy after 9 epochs (2 minutes)
stack exec example-notmnist

# Naive Bayes document classifier, using Reuters dataset
# using Porter stemming, stopword elimination and a few custom techniques.
# The dataset is imbalanced which causes the classifier to be biased towards some classes (earn, acq, ...)
# to workaround the imbalanced dataset problem, there is a --top-ten option which classifies only top 10 popular
# classes, with evenly split datasets (100 for each), this increases F Measure significantly, along with ~10% of improved accuracy
# N-Grams don't seem to help us much here (or maybe my implementation is wrong!), using bigrams increases
# accuracy, while decreasing F-Measure slightly.
stack exec example-naivebayes-doc-classifier -- --verbose
stack exec example-naivebayes-doc-classifier -- --verbose --top-ten

notMNIST

notMNIST dataset, sigmoid hidden layer, cross-entropy loss, learning rate decay and sgd (notmnist.hs): notMNIST

notMNIST dataset, relu hidden layer, cross-entropy loss, learning rate decay and sgd (notmnist.hs): notMNIST