2 views

I need to code a Maximum Likelihood Estimator to estimate the mean and variance of some toy data. I have a vector with 100 samples, created with numpy.random.randn(100). The data should have zero mean and unit variance Gaussian distribution.

I checked Wikipedia and some extra sources, but I am a little bit confused since I don't have a statistics background.

Is there any pseudo code for a maximum likelihood estimator? I get the intuition of MLE but I cannot figure out where to start coding.

Wiki says taking argmax of log-likelihood. What I understand is: I need to calculate log-likelihood by using different parameters and then I'll take the parameters which gave the maximum probability. What I don't get is: where will I find the parameters in the first place? If I randomly try different mean & variance to get a high probability, when should I stop trying?

by (33.1k points)

Pseudocode for MLE:

import numpy as np

from scipy.optimize import minimize

import scipy.stats as stats

import time

# Set up your x values

x = np.linspace(0, 100, num=100)

# Set up your observed y values with a known slope (2.4), intercept (5), and sd (4)

yObs = 5 + 2.4*x + np.random.normal(0, 4, 100)

def regressLL(params):

# Resave the initial parameter guesses

b0 = params[0]

b1 = params[1]

sd = params[2]

# Calculate the predicted values from the initial parameter guesses

yPred = b0 + b1*x

logLik = -np.sum( stats.norm.logpdf(yObs, loc=yPred, scale=sd) )

# Tell the function to return the NLL (this is what will be minimized)

return(logLik)

# Make a list of initial parameter guesses (b0, b1, sd)

initParams = [1, 1, 1]

# Run the minimizer

# Print the results. They should be really close to your actual values

print results.x