Explore Courses Blog Tutorials Interview Questions
0 votes
in Machine Learning by (19k points)

I need to code a Maximum Likelihood Estimator to estimate the mean and variance of some toy data. I have a vector with 100 samples, created with numpy.random.randn(100). The data should have zero mean and unit variance Gaussian distribution.

I checked Wikipedia and some extra sources, but I am a little bit confused since I don't have a statistics background.

Is there any pseudo code for a maximum likelihood estimator? I get the intuition of MLE but I cannot figure out where to start coding.

Wiki says taking argmax of log-likelihood. What I understand is: I need to calculate log-likelihood by using different parameters and then I'll take the parameters which gave the maximum probability. What I don't get is: where will I find the parameters in the first place? If I randomly try different mean & variance to get a high probability, when should I stop trying?

1 Answer

0 votes
by (33.1k points)

Pseudocode for MLE:

import numpy as np

from scipy.optimize import minimize

import scipy.stats as stats

import time

# Set up your x values

x = np.linspace(0, 100, num=100)

# Set up your observed y values with a known slope (2.4), intercept (5), and sd (4)

yObs = 5 + 2.4*x + np.random.normal(0, 4, 100)

def regressLL(params):

    # Resave the initial parameter guesses

    b0 = params[0]

    b1 = params[1]

    sd = params[2]

    # Calculate the predicted values from the initial parameter guesses

    yPred = b0 + b1*x

    logLik = -np.sum( stats.norm.logpdf(yObs, loc=yPred, scale=sd) )

    # Tell the function to return the NLL (this is what will be minimized)


# Make a list of initial parameter guesses (b0, b1, sd)    

initParams = [1, 1, 1]

# Run the minimizer

results = minimize(regressLL, initParams, method='nelder-mead')

# Print the results. They should be really close to your actual values

print results.x


Hope this answer helps.

Welcome to Intellipaat Community. Get your technical queries answered by top developers!

30.5k questions

32.6k answers


108k users

Browse Categories