This is a general guideline that you can follow to change an existing ETL schedule to daily. This should work for most of the scripts but maybe not all.
You can first check out the example diff provided and that should give the gist on what changes are necessary. If you need further explanation then read the explanation following that.

--

Part 1: Modifying the Spark Script (5 minutes)
Example diff

https://phabricator.noc.tvlk.cloud/D31805?vs=on&id=86882&whitespace=ignore-most

The goal is to allow the script's time granularity to actually be controlled by --time-granularity parameter by ensuring that time windows or any time related parameters are not hardcoded.

You generally want to modify the part of code that is hardcoded but actually should be relative to the time_granularity.
You can look for these parameters:

the loader time_granularity: Ensure that time_granularity is supplied to S3FileHook when loading. In some scripts you'll find that the value is hardcoded, in which case change it to time_granularity. For example:

sources time_window: Whether the time window need changing or not depends on what data is it exactly. For example, the time window for a dimension data might be fixed regardless of the scheduling scheme. For time windows that do, modify it such that the window value is relative to whatever time_granularity value is (hour_1, hour_6, day_1, ...).

For example if the granularity is day_1 but the source data is available in HOUR duration, we need to specify the time window n value to be the number of hours in one day (24). To convert how many n time unit are there in another time unit, you can use helper function DatetimeHelper.convert_duration. For example:

Part 2: Testing the (Now More Generic) Spark Script (15-30 minutes)

This is to ensure that that the --time-granularity parameter is indeed effective and correct. That is, simply specifying --time-granularity hour_6 does make the script produce six hourly data and specifying --time-granularity day_1 does make the script produce daily data, and so on.

Ideally done in dev environment, but depending on how complicated the source data is, it can be time consuming to copy all the source data to dev. In which case do test in prod (*plakkk*) but ensure that the original data is not broken or can restored after testing.
Ensure that the script does not break with current scheduling. You can try:

Ssh to data-airiflow-worker-01. Go to /data/to_daily. Paste your modified script there.
Test run a spark-submit.sh command both with the original script and the modified script. You can find a sample command in the Airflow log of a to_avro task. For example:

spark-submit.sh --master yarn-cluster --jars /home/ubuntu/spark-jars/com.amazonaws_aws-java-sdk-1.7.4.jar,/home/ubuntu/spark-jars/org.apache.hadoop_hadoop-aws-2.7.1.jar,/home/ubuntu/spark-jars/com.databricks_spark-avro_2.10-2.0.1.jar edw_fact_flight_price_accuracy.py --dt "2016-06-01 00:00:00" --env dev --task_id track.flight.searchAccuracy_to_avro --event-name track.flight.searchAccuracy --time-granularity hour_1 --duration hour --table-name edw.fact_flight_price_accuracy

Compare the resulting S3 files, they should contain equivalent data (I usually just check for the number of records).
Ensure that the script does work with daily scheduling.

Test run a spark-submit.sh command, but set the parameter --time-granularity to day_1.
Compare whether the resulting S3 file is equal to the equivalent aggregate of it's original time granularity. For example the daily data should be equivalent to 24 hourly data. Note that you may need to consider duplicates if there can be data overlap in two different location (I usually distinct by some keys first then count).

Part 3: Creating the New (Daily) DAG Definition (5 minutes)
Example diff

Copy the original DAG definition file, name it <original name>_daily.py
In the daily DAG, make sure that these parameters are adjusted:

The DAG name, since there can't be multiple DAG with the same name, append _daily at the end of it too.
Bump the start_date to a recent date (no more than 3 days before expected release).
Change the cron expression to represent daily schedule. We usually use 0 17 * * *.
The sensors. If it usually just poke for one hourly files, now it might need to poke for 24 hourly files. If it usually poke for one 6-hourly files, it now needs to poke for four 6-hourly files. Etc etc.
The spark submit operator. Modify --time-granularity spark submit parameter to day_1.
The load to redshift operator. Modify time_granularity to day_1.

Part 4: Removing the Load to Redshift Task in Original DAG (1 minute)

!!! Important: do this only after Part 1-3 has been completed, released, and stable.

Example Diff

Post Release Checks

After completing the migration, don't forget to ensure that both the new daily DAG and the original (non-daily) DAG are successfully executed.
If not , find out why and ensure that it is not due to the migration (sometimes it failed for reasons not related to daily schedule).

Changing an ETL schedule to daily