Pandas merge giving the error “Buffer has the wrong number of dimensions (expected 1, got 2)”

Question

I am trying to do pandas merge and get the above error from the title when I try to run it. I am using 3 columns to match on whereas just before I do a similar merge on only 2 columns and it works fine.

df = pd.merge(df, c, how="left",

        left_on=["section_term_ps_id", "section_school_id", "state"],

        right_on=["term_ps_id", "term_school_id", "state"])

columns for the two dataframes

df:

Index([u'section_ps_id', u'section_school_id', u'section_course_number', u'secti on_term_ps_id', u'section_staff_ps_id', u'section_number', u'section_expression' , u'section_grade_level', u'state', u'sections_id', u'course_ps_id', u'course_sc hool_id', u'course_number', u'course_schd_dept', u'courses_id', u'school_ps_id', u'course_school_id', u'school_name', u'school_abbr', u'school_low_grade', u'sch ool_high_grade', u'school_alt_school_number', u'school_state', u'school_phone', u'school_fax', u'school_principal', u'school_principal_phone', u'school_principa l_email', u'school_asst_principal', u'school_asst_principal_phone', u'school_ass t_principal_email'], dtype='object')

c:

Index([u'term_ps_id', u'term_school_id', u'term_portion', u'term_start_date', u' term_end_date', u'term_abbreviation', u'term_name', u'state', u'terms_id', u'sch ool_ps_id', u'term_school_id', u'school_name', u'school_abbr', u'school_low_grad e', u'school_high_grade', u'school_alt_school_number', u'school_state', u'school _phone', u'school_fax', u'school_principal', u'school_principal_phone', u'school _principal_email', u'school_asst_principal', u'school_asst_principal_phone', u's chool_asst_principal_email'], dtype='object')

Is it possible to merge on three columns like this? Is there anything wrong with the merge call here?

Answer 1

You have a dupe column. The following code will remove the duplicated columns from the Dataframe

df = df[list(df.columns[~df.columns.duplicated()])]