Skip to content

Commit 596fbe0

Browse files
pkbullockpkbullock
authored
Merge pull request #937 from Adam-it/new-script-spo-time-based-file-reports
New script spo time based file reports
2 parents a40901d + 4b5059e commit 596fbe0

2 files changed

Lines changed: 312 additions & 11 deletions

File tree

‎scripts/spo-time-based-file-reports/README.md‎

Lines changed: 290 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -22,18 +22,22 @@ The three scripts are as follows:
2222
- Scan the specific folders in the specific libraries on the specific sites from the csv file to get files older than 4 years
2323
- Create a report with the specific name in the csv file and upload it to the specified SharePoint location
2424

25-
## Pre-requisites
25+
## Pre-requisites
26+
2627
Several things must be configured or installed ahead of time
27-
An App registration with a certificate and appropriate permissions has to be created in Azure
28-
That certificate must be installed on any machine that will run the scripts
29-
PowerShell 7 must be installed
30-
The following PowerShell modules are needed:
31-
PnP.PowerShell
32-
ImportExcel
28+
- An App registration with a certificate and appropriate permissions has to be created in Azure
29+
- That certificate must be installed on any machine that will run the scripts
30+
- PowerShell 7 must be installed
31+
- The following PowerShell modules are needed:
32+
- PnP.PowerShell
33+
- ImportExcel
34+
3335
## Setup
36+
3437
Each of these scripts runs off of a csv file that you must fill out before running. You also have to configure the scripts themselves. See below for more details.
3538

3639
### All Scripts
40+
3741
This script uses an Azure App registration for authentication. You must create the registration with certificate to get the client ID, tenantID, and certificate thumbprint. You should also make sure that app has sufficient permissions. These are the ones I used:
3842

3943
![APIPermissions.png](assets/APIPermissions.png)
@@ -58,6 +62,7 @@ Fill out the OneDriveURLs.CSV file with the URL of the OneDrive user you wish to
5862
Fill out the Script2SPURLs.csv file with the URLs for all the SharePoint sites you wish to scan. Make sure you use the complete path!
5963

6064
### Single document library and folder
65+
6166
1. Open customSPURLs.csv
6267
2. Add the URL for every site you wish to scan
6368
3. Add the library name for the title of the library you are scanning
@@ -77,10 +82,10 @@ Now that the scripts are setup, you just need to run them. All these steps are t
7782
4. Hit enter and the script will run
7883
5. Do the same thing with script 2 and 3 if you wish
7984

80-
8185
## 1. OneDrive Scan
8286

8387
# [PnP PowerShell](#tab/pnpps)
88+
8489
```powershell
8590
8691
# Declare and initialize your app-only authentication details
@@ -259,6 +264,7 @@ Stop-Transcript
259264
## 3. Individual library and folder Scan
260265

261266
# [PnP PowerShell](#tab/pnpps3)
267+
262268
```powershell
263269
# Declare and initialize your app-only authentication details
264270
$clientId = "xxxxx"
@@ -334,11 +340,287 @@ Stop-Transcript
334340
[!INCLUDE [More about PnP PowerShell](../../docfx/includes/MORE-PNPPS.md)]
335341
***
336342

343+
## All in one CLI for Microsoft 365 version
344+
345+
# [CLI for Microsoft 365](#tab/cli-m365-ps)
346+
347+
```powershell
348+
349+
[CmdletBinding(SupportsShouldProcess)]
350+
param(
351+
[Parameter(Mandatory = $false, HelpMessage = "SharePoint admin center URL (e.g., https://contoso-admin.sharepoint.com)")]
352+
[ValidatePattern('^https://')]
353+
[string]$TenantAdminUrl,
354+
355+
[Parameter(Mandatory = $false, HelpMessage = "SharePoint site URL to scan (e.g., https://contoso.sharepoint.com/sites/project)")]
356+
[ValidatePattern('^https://')]
357+
[string]$SiteUrl,
358+
359+
[Parameter(Mandatory = $false, HelpMessage = "Title of a specific document library to scan")]
360+
[string]$LibraryName,
361+
362+
[Parameter(Mandatory = $false, HelpMessage = "Server-relative URL of a specific folder to scan")]
363+
[string]$FolderUrl,
364+
365+
[Parameter(Mandatory = $false, HelpMessage = "Number of days to use as age threshold (default: 1460 = 4 years)")]
366+
[int]$DaysOld = 1460,
367+
368+
[Parameter(Mandatory = $false, HelpMessage = "Full path for the CSV report file")]
369+
[string]$OutputPath,
370+
371+
[Parameter(Mandatory = $false, HelpMessage = "Include OneDrive personal sites in scan (requires TenantAdminUrl)")]
372+
[switch]$IncludeOneDrive,
373+
374+
[Parameter(Mandatory = $false, HelpMessage = "Scan subfolders recursively")]
375+
[switch]$Recursive
376+
)
377+
378+
begin {
379+
if (-not $TenantAdminUrl -and -not $SiteUrl) {
380+
throw "You must specify either -TenantAdminUrl or -SiteUrl parameter."
381+
}
382+
383+
if ($TenantAdminUrl -and $SiteUrl) {
384+
throw "Cannot specify both -TenantAdminUrl and -SiteUrl. Choose one scan mode."
385+
}
386+
387+
if ($IncludeOneDrive -and -not $TenantAdminUrl) {
388+
throw "-IncludeOneDrive requires -TenantAdminUrl parameter."
389+
}
390+
391+
if ($LibraryName -and -not $SiteUrl) {
392+
throw "-LibraryName requires -SiteUrl parameter."
393+
}
394+
395+
if ($FolderUrl -and -not $SiteUrl) {
396+
throw "-FolderUrl requires -SiteUrl parameter."
397+
}
398+
399+
if (-not $OutputPath) {
400+
$timestamp = Get-Date -Format "yyyyMMdd-HHmmss"
401+
$OutputPath = Join-Path (Get-Location) "OldFilesReport-$timestamp.csv"
402+
} else {
403+
$parentFolder = Split-Path -Path $OutputPath -Parent
404+
if (-not (Test-Path -Path $parentFolder)) {
405+
throw "Output folder does not exist: $parentFolder"
406+
}
407+
}
408+
409+
$transcriptPath = $OutputPath -replace '\.csv$', '-Transcript.log'
410+
Start-Transcript -Path $transcriptPath
411+
412+
Write-Host "Starting old file report generation..." -ForegroundColor Cyan
413+
Write-Host "Age threshold: $DaysOld days" -ForegroundColor Cyan
414+
Write-Host "Output path: $OutputPath" -ForegroundColor Cyan
415+
416+
Write-Verbose "Ensuring CLI for Microsoft 365 login..."
417+
m365 login --ensure
418+
if ($LASTEXITCODE -ne 0) {
419+
Stop-Transcript
420+
throw "Failed to authenticate with CLI for Microsoft 365."
421+
}
422+
423+
$cutoffDate = (Get-Date).AddDays(-$DaysOld)
424+
$cutoffDateString = $cutoffDate.ToString("yyyy-MM-ddTHH:mm:ssZ")
425+
Write-Verbose "Cutoff date: $cutoffDateString"
426+
427+
$script:ReportCollection = [System.Collections.Generic.List[object]]::new()
428+
$script:Summary = @{
429+
SitesProcessed = 0
430+
LibrariesProcessed = 0
431+
OldFilesFound = 0
432+
Failures = 0
433+
}
434+
}
435+
436+
process {
437+
try {
438+
$sitesToProcess = @()
439+
440+
if ($TenantAdminUrl) {
441+
Write-Host "Retrieving sites from tenant..." -ForegroundColor Cyan
442+
Write-Verbose "Building site list command..."
443+
444+
$siteListArgs = @('spo', 'site', 'list', '--output', 'json')
445+
if ($IncludeOneDrive) {
446+
Write-Verbose "Including OneDrive sites"
447+
$siteListArgs += '--withOneDriveSites'
448+
} else {
449+
Write-Verbose "Filtering for SharePoint sites only"
450+
$siteListArgs += '--filter'
451+
$siteListArgs += "Url -like '/sites/'"
452+
}
453+
454+
$sitesJson = m365 @siteListArgs 2>&1
455+
if ($LASTEXITCODE -ne 0) {
456+
Write-Warning "Failed to retrieve sites. CLI: $sitesJson"
457+
$script:Summary.Failures++
458+
return
459+
}
460+
461+
$sitesToProcess = @($sitesJson | ConvertFrom-Json)
462+
Write-Host "Found $($sitesToProcess.Count) sites to scan" -ForegroundColor Green
463+
} else {
464+
Write-Verbose "Using single site: $SiteUrl"
465+
$sitesToProcess = @(@{ Url = $SiteUrl; Title = "" })
466+
}
467+
468+
$siteCounter = 0
469+
foreach ($site in $sitesToProcess) {
470+
$siteCounter++
471+
$siteUrl = $site.Url
472+
$siteTitle = if ($site.Title) { $site.Title } else { $siteUrl }
473+
474+
Write-Progress -Activity "Processing sites" -Status "Site $siteCounter of $($sitesToProcess.Count): $siteTitle" -PercentComplete (($siteCounter / $sitesToProcess.Count) * 100)
475+
Write-Verbose "Processing site: $siteUrl"
476+
477+
try {
478+
$librariesToProcess = @()
479+
480+
if ($LibraryName) {
481+
Write-Verbose "Using specific library: $LibraryName"
482+
$libraryListJson = m365 spo list list --webUrl $siteUrl --filter "Title eq '$LibraryName' and BaseTemplate eq 101 and Hidden eq false" --properties "Title,RootFolder/ServerRelativeUrl" --output json 2>&1
483+
if ($LASTEXITCODE -ne 0) {
484+
Write-Warning "Failed to retrieve library '$LibraryName' from site '$siteUrl'. CLI: $libraryListJson"
485+
$script:Summary.Failures++
486+
continue
487+
}
488+
$librariesToProcess = @($libraryListJson | ConvertFrom-Json)
489+
} else {
490+
Write-Verbose "Retrieving all document libraries..."
491+
$libraryListJson = m365 spo list list --webUrl $siteUrl --filter "BaseTemplate eq 101 and Hidden eq false" --properties "Title,RootFolder/ServerRelativeUrl" --output json 2>&1
492+
if ($LASTEXITCODE -ne 0) {
493+
Write-Warning "Failed to retrieve libraries from site '$siteUrl'. CLI: $libraryListJson"
494+
$script:Summary.Failures++
495+
continue
496+
}
497+
$librariesToProcess = @($libraryListJson | ConvertFrom-Json)
498+
}
499+
500+
if ($librariesToProcess.Count -eq 0) {
501+
Write-Verbose "No document libraries found in site '$siteUrl'"
502+
continue
503+
}
504+
505+
$script:Summary.SitesProcessed++
506+
507+
foreach ($library in $librariesToProcess) {
508+
$libraryTitle = $library.Title
509+
$folderPath = if ($FolderUrl) { $FolderUrl } else { $library.RootFolder.ServerRelativeUrl }
510+
511+
Write-Verbose "Scanning library: $libraryTitle (Folder: $folderPath)"
512+
513+
try {
514+
$fileListArgs = @(
515+
'spo', 'file', 'list',
516+
'--webUrl', $siteUrl,
517+
'--folderUrl', $folderPath,
518+
'--fields', 'Name,ServerRelativeUrl,TimeLastModified,TimeCreated,Length,ListItemAllFields/Author,ListItemAllFields/Editor',
519+
'--filter', "TimeLastModified lt datetime'$cutoffDateString'",
520+
'--output', 'json'
521+
)
522+
if ($Recursive) {
523+
$fileListArgs += '--recursive'
524+
}
525+
526+
$filesJson = m365 @fileListArgs 2>&1
527+
if ($LASTEXITCODE -ne 0) {
528+
Write-Warning "Failed to retrieve files from library '$libraryTitle' in site '$siteUrl'. CLI: $filesJson"
529+
$script:Summary.Failures++
530+
continue
531+
}
532+
533+
$files = @($filesJson | ConvertFrom-Json)
534+
Write-Verbose "Found $($files.Count) old files in library '$libraryTitle'"
535+
536+
foreach ($file in $files) {
537+
$lastModified = [DateTime]::Parse($file.TimeLastModified)
538+
$daysOldValue = [Math]::Round((Get-Date).Subtract($lastModified).TotalDays)
539+
540+
$script:ReportCollection.Add([PSCustomObject]@{
541+
SiteUrl = $siteUrl
542+
SiteTitle = $siteTitle
543+
LibraryTitle = $libraryTitle
544+
FileName = $file.Name
545+
FilePath = $file.ServerRelativeUrl
546+
LastModified = $file.TimeLastModified
547+
Created = $file.TimeCreated
548+
SizeBytes = $file.Length
549+
Author = if ($file.ListItemAllFields.Author) { $file.ListItemAllFields.Author.LookupValue } else { "N/A" }
550+
Editor = if ($file.ListItemAllFields.Editor) { $file.ListItemAllFields.Editor.LookupValue } else { "N/A" }
551+
DaysOld = $daysOldValue
552+
})
553+
$script:Summary.OldFilesFound++
554+
}
555+
556+
$script:Summary.LibrariesProcessed++
557+
} catch {
558+
Write-Warning "Error scanning library '$libraryTitle' in site '$siteUrl': $_"
559+
$script:Summary.Failures++
560+
continue
561+
}
562+
}
563+
} catch {
564+
Write-Warning "Error processing site '$siteUrl': $_"
565+
$script:Summary.Failures++
566+
continue
567+
}
568+
}
569+
} catch {
570+
Write-Warning "Unexpected error during processing: $_"
571+
$script:Summary.Failures++
572+
}
573+
}
574+
575+
end {
576+
Write-Progress -Activity "Processing sites" -Completed
577+
578+
if ($script:ReportCollection.Count -gt 0) {
579+
Write-Host "Exporting report to CSV..." -ForegroundColor Cyan
580+
$script:ReportCollection | Sort-Object SiteUrl, LibraryTitle, LastModified | Export-Csv -Path $OutputPath -NoTypeInformation -Encoding UTF8
581+
Write-Host "Report exported: $OutputPath" -ForegroundColor Green
582+
} else {
583+
Write-Host "No old files found matching criteria." -ForegroundColor Yellow
584+
}
585+
586+
Write-Host "`n===== Summary =====" -ForegroundColor Cyan
587+
Write-Host "Sites processed: $($script:Summary.SitesProcessed)" -ForegroundColor White
588+
Write-Host "Libraries processed: $($script:Summary.LibrariesProcessed)" -ForegroundColor White
589+
Write-Host "Old files found: $($script:Summary.OldFilesFound)" -ForegroundColor White
590+
if ($script:Summary.Failures -gt 0) {
591+
Write-Host "Failures: $($script:Summary.Failures)" -ForegroundColor Red
592+
} else {
593+
Write-Host "Failures: 0" -ForegroundColor Green
594+
}
595+
Write-Host "==================`n" -ForegroundColor Cyan
596+
597+
Stop-Transcript
598+
}
599+
600+
# Scans all SharePoint sites for files older than 3 years
601+
# .\Report-OldFiles.ps1 -TenantAdminUrl "https://contoso-admin.sharepoint.com" -DaysOld 1095
602+
603+
# Scans all SharePoint AND OneDrive sites for files older than 4 years
604+
# .\Report-OldFiles.ps1 -TenantAdminUrl "https://contoso-admin.sharepoint.com" -IncludeOneDrive
605+
606+
# Scans all libraries in a specific site recursively
607+
# .\Report-OldFiles.ps1 -SiteUrl "https://contoso.sharepoint.com/sites/project" -Recursive
608+
609+
# Scans a specific folder in a specific library with verbose output
610+
# .\Report-OldFiles.ps1 -SiteUrl "https://contoso.sharepoint.com/sites/project" -LibraryName "Documents" -FolderUrl "/sites/project/Documents/Archive" -DaysOld 730 -Recursive -Verbose
611+
612+
```
613+
614+
615+
[!INCLUDE [More about CLI for Microsoft 365](../../docfx/includes/MORE-CLIM365.md)]
616+
***
617+
337618
## Contributors
338619

339620
| Author(s) |
340621
|-----------|
341622
| Nick Brattoli|
623+
| Adam Wójcik|
342624

343625

344626
[!INCLUDE [DISCLAIMER](../../docfx/includes/DISCLAIMER.md)]

0 commit comments

Comments
 (0)