Iteratively changing dates in URLs

Question

I'm having trouble building a logic/algorithm that creates a date, adds it to the URL, and then when I create another URL, it will contain the next date.

It should iterate through every day of every month of every year (that's why I thought of the nested for loops).

Note that I only have one variable for the date because I want the start and end dates to be the same.

# Setting range of years, months and days to be iterated through in the URL later

YYYY = []

years = range(2016, 2021)

for yyyy in years:

    YYYY.append(yyyy)

MM = []

months = range(1, 13)

for mm in months:

    MM.append(mm)

DD = []

days = range (1, 32)

for dd in days:

    DD.append(dd)

  

# Create iterating logic with i, j and k t define which year, month and day will be added to the URL

for i in YYYY:

    for j in MM:

        for k in DD:

            True

# start_date and e_date are the same, so we just define 'date'

date = str(YYYY[i]) + '-' + str(MM[j]) + '-' + str(DD[k])

print(date)

# Create URL with the date variable so it can be iterated through

URL = ('https://movement.uber.com/explore/atlanta/travel-times/query?si=1074&ti=&ag=taz&dt[tpb]=ALL_DAY&dt[wd;]=1,2,3,4,5,6,7&dt[dr][sd]=' +

       date + '&dt[dr][ed]=' + date + '&dt[dr][ed]=2016-01-19&cd=&sa;=&sdn=&lang=en-US')

Answer 1

You could use date time and time delta objects to make a generator to produce the URLs to iterate over:

from datetime import datetime, timedelta

def get_url():

    date = datetime(2016,1,1)

    while date < datetime(2020,12,31):

        yield 'https://movement.uber.com/explore/atlanta/travel-times/query?si=1074&ti=&ag=taz&dt[tpb]=ALL_DAY&dt[wd;]=1,2,3,4,5,6,7&dt[dr][sd]=' + \

               date.strftime('%Y-%m-%d') + '&dt[dr][ed]=' + date.strftime('%Y-%m-%d') + '&dt[dr][ed]=2016-01-19&cd=&sa;=&sdn=&lang=en-US'

        date += timedelta(days=1)   

i = 0

for url in get_url():

    i += 1

    if i < 3 or i > 10:

        print(url)

    if i > 12:

        break

Output:

https://movement.uber.com/explore/atlanta/travel-times/query?si=1074&ti=&ag=taz&dt[tpb]=ALL_DAY&dt[wd;]=1,2,3,4,5,6,7&dt[dr][sd]=2016-01-01&dt[dr][ed]=2016-01-01&dt[dr][ed]=2016-01-19&cd=&sa;=&sdn=&lang=en-US

https://movement.uber.com/explore/atlanta/travel-times/query?si=1074&ti=&ag=taz&dt[tpb]=ALL_DAY&dt[wd;]=1,2,3,4,5,6,7&dt[dr][sd]=2016-01-02&dt[dr][ed]=2016-01-02&dt[dr][ed]=2016-01-19&cd=&sa;=&sdn=&lang=en-US

https://movement.uber.com/explore/atlanta/travel-times/query?si=1074&ti=&ag=taz&dt[tpb]=ALL_DAY&dt[wd;]=1,2,3,4,5,6,7&dt[dr][sd]=2016-01-11&dt[dr][ed]=2016-01-11&dt[dr][ed]=2016-01-19&cd=&sa;=&sdn=&lang=en-US

https://movement.uber.com/explore/atlanta/travel-times/query?si=1074&ti=&ag=taz&dt[tpb]=ALL_DAY&dt[wd;]=1,2,3,4,5,6,7&dt[dr][sd]=2016-01-12&dt[dr][ed]=2016-01-12&dt[dr][ed]=2016-01-19&cd=&sa;=&sdn=&lang=en-US

https://movement.uber.com/explore/atlanta/travel-times/query?si=1074&ti=&ag=taz&dt[tpb]=ALL_DAY&dt[wd;]=1,2,3,4,5,6,7&dt[dr][sd]=2016-01-13&dt[dr][ed]=2016-01-13&dt[dr][ed]=2016-01-19&cd=&sa;=&sdn=&lang=en-US

If you want to know more about the Data Science then do check out the following Data Science which will help you in understanding Data Science from scratch