Using Python to append large set of data frames to an existing excel file

Question

Following is my code snippet:

writer = pd.ExcelWriter(path_file, engine= 'openpyxl')

df.to_excel(writer, index= False)

df.to_excel(writer, startrow= len(df), index= False)

writer.save()

df has all the data I need to append.

I have a master excel sheet that is linked to a Tableau workbook.

I have pulled a large amount of data into data frames, and I want to be able to append the data frames to the existing master excel sheet.

The problem right now it is just completely overwriting the file and creating a new copy.

Any Ideas??

Answer 1

• You can use csvtools. Free command-line utilities for working with CSV files.

https://csvkit.readthedocs.io

• You could use in2csv to convert Excel to CSV, and csvstack to combine multiple CSV files. Basically, csvkit is written in Python and plays well with Python, but can be used as a standalone suite of tools. It follows the same design approach as the simple Unix command-line tools.