Filtering through csv's in sub-directories with criteria

Question

Currently, I have the below code which finds all the "*_results.csv" files and copies these files to a new directory:

import os.path

import shutil

base_dir = r'\temp'

dest_dir = r'\temp1'

for root, dirs, files in os.walk(base_dir):

    for filename in [f for f in files if f.endswith("_results.csv")]:

        shutil.copy2((os.path.join(root, filename)), dest_dir)

All of these CSV files have 2 columns, but the number of rows varies. All of the CSV files have 2 rows in column A that say "Power" and "Speed", with the values for Power and Speed in column B. It is important to note that "Power" and "Speed" are sometimes in different row numbers within Column A depending on the CSV.

For example --

x       x

x       x

Power   -20

x       x

Speed   35

I am looking to find all the CSV's that have a Power >= -18 or where the Speed - Power >= 5.

Any help would be very appreciated. Thanks.

Answer 1

For each file, you can do:

df = pd.read_csv("file.csv")

powerdf = df[df.A == "Power"]

thepower = powerdf.iloc[0][B]

if thepower >= -18:

   return True

speeddf = ...

And so on.

Improve your knowledge in data science from scratch using Data science online courses