How to shuffle a "DenseMatrix" in F#

Question

In Matlab we can write the following code to shuffle a matrix:

data = data(:, randperm(size(data,2)));

there is what I write with Math.NET:

let csvfile = @"../UFLDL-tutorial-F#/housing.csv"
let housingAsLines = 
    File.ReadAllLines(csvfile)
        |> Array.map (fun t -> t.Split(',')
                            |> Array.map (fun t -> float t))
let housingAsMatrix= DenseMatrix.OfRowArrays housingAsLines
let housingAsMatrixT = housingAsMatrix.Transpose()

let v1 = DenseVector.Create(housingAsMatrixT.ColumnCount,1.0)
housingAsMatrixT.InsertRow(0,v1)

// How to shuffle a "DenseMatrix" in F#

To simulate matrix operation in Matlab, using the F# slice syntax and zero-based indexing. However, it doesn't work.

housingAsMatrixT.[*,0]

And I got the error message in vscode.

The field, constructor or member 'GetSlice' is not defined

Ruscio

Is there any reason you have to shuffle the data after it is a DenseMatrix? Can you load the data into an array, e.g. [], then shuffle the data, and then use the array to construct the DenseMatrix? – Tiberias

in order to prevent overfitting and underfitting with Machine Learning algorithms, I have to do that. – Ruscio

"I have to do that" I take it that you mean you have to shuffle the data for each run of the training session. The statement could be read that you have to shuffle the data once it is in the Matrix and not before. – Tiberias

Of interest: Neural Networks and Deep Learning – Tiberias

maybe you can use SelectPermutationInplace? – Demonetize

@Demonetize Nice answer. I suspect that the OP will find that what he really wants/needs is an array of matrices and that he needs to only shuffle the array and not the matrix. Or to put it another way, the OP does not need a matrix of matrices but an array of matrices. – Tiberias

Of interest: UFLDL Tutorial This is source of the question. – Tiberias

Can you provide a link to the data so that I can see it in its raw form. I might be able to provide some existing loader code I have if it is compatible. – Tiberias

Of interest: Housing Data Set I don't know if this is the dataset the OP is using, but this can be used with Neural Networks. – Tiberias

Is the raw data for each input a vector or a matrix? I have been thinking that the data is a matrix as in a representation of pixels of a digit, but if it is just a vector then change array of matrix to array of vector. – Tiberias

Answer 1

You actually have two questions, 1) how to slice Matrices ala matlab and 2) how to shuffle the columns of a matrix.

For 1) actually Issue 277 you linked in the comment does indeed provide the solution. However you might be using an old version or you might not be referencing the F# extensions correctly:

#r @"..\packages\MathNet.Numerics.3.13.1\lib\net40\MathNet.Numerics.dll"
#r @"..\packages\MathNet.Numerics.FSharp.3.13.1\lib\net40\MathNet.Numerics.FSharp.dll"

open MathNet.Numerics
open MathNet.Numerics.LinearAlgebra
open MathNet.Numerics.Distributions
open System

//let m = DenseMatrix.randomStandard<float> 5 5
let m = DenseMatrix.random<float> 5 5 (ContinuousUniform(0., 1.))
let m' = m.[*,0]
m'
//val it : Vector<float> =
//seq [0.4710989485; 0.2220238937; 0.566367266; 0.2356496324; ...]

This extracts the first column of the matrix.

Now for 2), assuming you need to shuffle the matrix or the arrays containing a matrix you can use some of the approaches below. There might be a more elegant method within mathnet.numerics.

To permute the vector above: m'.SelectPermutation() or SelectPermutationInplace for arrays. There are other convenience function like .Column(idx),.EnumerateColumnsIndexed() or EnumerateColumns(), etc.

So m'.SelectPermutation() will shuffle the elements of m'. Or to shuffle the columns (which your matlab function does):

let idx = Combinatorics.GeneratePermutation 5
idx
//val it : int [] = [|2; 0; 1; 4; 3|]
let m2 = idx |> Seq.map (fun i -> m.Column(i)) |> DenseMatrix.ofColumnSeq
m2.Column(1) = m.Column(0)
//val it : bool = true

Since the first column of the original matrix moved to the second column of the new matrix, the two should be equal.

Demonetize

As of today Matrix has PermuteColumns/PermuteRows – Sickly

Answer 2

With Neural Networks I had to shuffle an array of Matrices and used the following code. Note the base data structure is an array ([]) and each item in the array is a Matrix. This is not shuffling a matrix, but an array. It should give you some idea of how to proceed for your problem.

type Random() = 
    static member Shuffle (a : 'a[]) =
        let rand = new System.Random()
        let swap (a: _[]) x y =
            let tmp = a.[x]
            a.[x] <- a.[y]
            a.[y] <- tmp
        Array.iteri (fun i _ -> swap a i (rand.Next(i, Array.length a))) a

and called it like

Random.Shuffle trainingData

Addendum

Here is the code to convert a byte[] to a DenseMatrix of double

let byteArrayToMatrix (bytes : byte[]) : Matrix<double> =
    let (x : Vector<byte>) = Vector<byte>.Build.DenseOfArray bytes
    let (y : Vector<double>) = x.Map(fun x -> double x)
    let (z : Matrix<double>) = Matrix<double>.Build.DenseOfRowVectors y
    z

Tiberias

Thank you. By the way, how to extract a whole column/row in DenseMatrix? – Ruscio

I only gave a quick response because I know that asking Neural Network questions on SO for the F# tag get few responses and even fewer of us have working and tested code. While I can appreciate that you want to learn, having done this myself the best advise for learning about MathNet Numerics Dense Matrix is to reference these two pages. DenseMatrix for the available methods, and Matrices and Vectors for some intro documentation. – Tiberias

The way I learned to use DenseMatrix for this was to write test cases and play with the functions. Also I translated the working Python code from Neural Networks and Deep Learning to F# so I had less guess work to do on what data structures to implement. I even did one example where I used a linked list instead of an array because you always move through the data in a sequential manner then start again from the start. No randomness so no need for an array. Didn't use it though because typically the code is done with arrays and no speed improvement. – Tiberias

I have been reviewed these pages; DenseMatrix for the available methods, and Matrices and Vectors. According to "github.com/mathnet/mathnet-numerics/issues/277", slice syntax is able to write something that looks more like Matlab. – Ruscio

@NelsonMok When I loaded the raw data into the arrays and then converted them to DenseMatrix I made extensive use of Array.blit – Tiberias

Answer 3

@GuyCoder and @s952163, thanks for your help. I implemented a quick-and-dirty version. It is not good enough but it works.

Please feel free to comment. Thank you.

#load "../packages/FsLab.1.0.2/FsLab.fsx"
open System
open System.IO
open MathNet.Numerics.LinearAlgebra.Double
open MathNet.Numerics
open MathNet.Numerics.LinearAlgebra
open MathNet.Numerics.Distributions

// implementation of the Fisher-Yates shuffle by Mathias
// http://www.clear-lines.com/blog/post/Optimizing-some-old-F-code.aspx
let swap fst snd i =
   if i = fst then snd else
   if i = snd then fst else
   i
let shuffle items (rng: Random) =
   let rec shuffleTo items upTo =
      match upTo with
      | 0 -> items
      | _ ->
         let fst = rng.Next(upTo)
         let shuffled = List.permute (swap fst (upTo - 1)) items
         shuffleTo shuffled (upTo - 1)
   let length = List.length items
   shuffleTo items length

let csvfile = @"/eUSB/sync/fsharp/UFLDL-tutorial-F#/housing.csv"
let housingAsLines = 
    File.ReadAllLines(csvfile)
        |> Array.map (fun t -> t.Split(',')
                            |> Array.map (fun t -> float t))
let housingAsMatrix= DenseMatrix.OfRowArrays housingAsLines
let housingAsMatrixTmp = housingAsMatrix.Transpose()
let v1 = DenseVector.Create(housingAsMatrixTmp.ColumnCount,1.0)
let housingAsMatrixT = housingAsMatrixTmp.InsertRow(0,v1)

let m = housingAsMatrixT.RowCount - 1
let listOfArray = [0..m]
let random = new Random()
let shuffled = shuffle listOfArray random

let z = [for i in shuffled -> (housingAsMatrixT.[i, *])]
let final = DenseMatrix.OfRowVectors z

Ruscio