Maximum Likelihood Estimate pseudocode

Question

I need to code a Maximum Likelihood Estimator to estimate the mean and variance of some toy data. I have a vector with 100 samples, created with numpy.random.randn(100). The data should have zero mean and unit variance Gaussian distribution.

I checked Wikipedia and some extra sources, but I am a little bit confused since I don't have a statistics background.

Is there any pseudo code for a maximum likelihood estimator? I get the intuition of MLE but I cannot figure out where to start coding.

Wiki says taking argmax of log-likelihood. What I understand is: I need to calculate log-likelihood by using different parameters and then I'll take the parameters which gave the maximum probability. What I don't get is: where will I find the parameters in the first place? If I randomly try different mean & variance to get a high probability, when should I stop trying?

Answer 1

Pseudocode for MLE:

import numpy as np

from scipy.optimize import minimize

import scipy.stats as stats

import time

# Set up your x values

x = np.linspace(0, 100, num=100)

# Set up your observed y values with a known slope (2.4), intercept (5), and sd (4)

yObs = 5 + 2.4*x + np.random.normal(0, 4, 100)

def regressLL(params):

    # Resave the initial parameter guesses

    b0 = params[0]

    b1 = params[1]

    sd = params[2]

    # Calculate the predicted values from the initial parameter guesses

    yPred = b0 + b1*x

    logLik = -np.sum( stats.norm.logpdf(yObs, loc=yPred, scale=sd) )

    # Tell the function to return the NLL (this is what will be minimized)

    return(logLik)

# Make a list of initial parameter guesses (b0, b1, sd)    

initParams = [1, 1, 1]

# Run the minimizer

results = minimize(regressLL, initParams, method='nelder-mead')

# Print the results. They should be really close to your actual values

print results.x

 

Hope this answer helps.