ADF - Copy and delete files from File Storage to Storage Account

Question

ADF - Copy and delete files from File Storage to Storage Account

Rajesh Ambakkat 236

i am trying to copy files from Azure File Storage to Blob storage(Archieve tier) using ADF. I also need to delete the files in source once copy is done. Same source file hierarchy (multiple folders in source) should be maintained at destination too while copy.

Using ADF copy activity i am able to do everything except delete files. How do i delete the files from source once copy is done, is there an option. I need to only delete the source file which is successfully copied.

Instead of copy activity do i have to use any other option? Please help

also i need to copy only files which are not updated since last 60 days...so how do i filter only files while copy?

PRADEEPCHEEKATLA 90,821 Reputation points Moderator

2023-06-23T08:21:10.1666667+00:00

@Rajesh Ambakkat - Just checking in to see if the below answer provided by @Subashri Vasudevan helped.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.
PRADEEPCHEEKATLA 90,821 Reputation points Moderator

2023-06-29T09:36:15.5133333+00:00

@Rajesh Ambakkat - Following up to see if the below answer was helpful. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Accepted answer

0 additional answers

Your answer

PRADEEPCHEEKATLA 90,821 Reputation points Moderator

2023-06-23T08:21:10.1666667+00:00

@Rajesh Ambakkat - Just checking in to see if the below answer provided by @Subashri Vasudevan helped.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.
PRADEEPCHEEKATLA 90,821 Reputation points Moderator

2023-06-29T09:36:15.5133333+00:00

@Rajesh Ambakkat - Following up to see if the below answer was helpful. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Answer 1

Hi @RajNa

Thanks for using the MS Q&A Portal and for the question.

1. Using ADF copy activity i am able to do everything except delete files. How do i delete the files from source once copy is done, is there an option. I need to only delete the source file which is successfully copied.
- Copy activity does not have the ability to delete files after copying.. It is data flow which has the option to delete source file or move it a folder upon successful copy. But copy activity is cheaper than data flow. What you can do now is, you can use another copy activity to copy the files to archive and then a delete activity to delete the file after 2nd copy (to archive) is successful.
also i need to copy only files which are not updated since last 60 days...so how do i filter only files while copy?

You can make use of a get meta data activity and use the last modified date option there to pick the files which were last modiefied 60 days ago. Please find below screenshot, where in my case, i am filtering files older than 10 days. I have four files in Azfs container, out of which just two files were last modified 10 days ago.

Screenshot 2023-06-22 at 10.29.47 AM

Screenshot 2023-06-22 at 10.30.59 AM

Expression used

@addDays(formatDateTime(utcNow(),'MM/dd/yyyy'),-10) : Start time UTC
@formatDateTime(utcNow(),'MM/dd/yyyy') : End Time UTC

Set variable is optional, if you want to see how the expression works, you may assign the above expression and see how it brings date.

My output after execution

Screenshot 2023-06-22 at 10.33.39 AM

Hope it helps.

Please let us know for any question.

Share via

ADF - Copy and delete files from File Storage to Storage Account

0 additional answers

Your answer