-->
Update your Ford SYNC version to receive the latest infotainment software features and bug fixes. Key features vary by SYNC version, these may include: Enhanced voice recognition, Apps at your command, Easy destination entry, Automatic updates over Wi-Fi and Android Auto or Apple Carplay. Update your Ford SYNC version to receive the latest infotainment software features and bug fixes. Key features vary by SYNC version, these may include: Enhanced voice recognition, Apps at your command, Easy destination entry, Automatic updates over Wi-Fi. Introducing the Ford SYNC 3 update with voice-activated technology and advanced features for navigation, calling and entertainment.
This topic describes the built-in scheduler in Azure AD Connect sync (sync engine).
This feature was introduced with build 1.1.105.0 (released February 2016).
Overview
Azure AD Connect sync synchronize changes occurring in your on-premises directory using a scheduler. There are two scheduler processes, one for password sync and another for object/attribute sync and maintenance tasks. This topic covers the latter.
In earlier releases, the scheduler for objects and attributes was external to the sync engine. It used Windows task scheduler or a separate Windows service to trigger the synchronization process. The scheduler is with the 1.1 releases built-in to the sync engine and do allow some customization. The new default synchronization frequency is 30 minutes.
The scheduler is responsible for two tasks:
- Synchronization cycle. The process to import, sync, and export changes.
- Maintenance tasks. Renew keys and certificates for Password reset and Device Registration Service (DRS). Purge old entries in the operations log.
The scheduler itself is always running, but it can be configured to only run one or none of these tasks. For example, if you need to have your own synchronization cycle process, you can disable this task in the scheduler but still run the maintenance task.
Important
By default every 30 minutes a synchronization cycle is run. If you have modified the synchronization cycley you will need to make sure that a synchronization cycle is run at least once every 7 days.
- A delta sync needs to happen within 7 days from the last delta sync.
- A delta sync (following a full sync) needs to happen within 7 days from the time the last full sync completed.
Failure to do so may cause synchronization issues which will require you to run a full synchronization to resolve. This also applies to servers in Staging mode.
Scheduler configuration
To see your current configuration settings, go to PowerShell and run Get-ADSyncScheduler
. It shows you something like this picture:
If you see The sync command or cmdlet is not available when you run this cmdlet, then the PowerShell module is not loaded. This problem could happen if you run Azure AD Connect on a domain controller or on a server with higher PowerShell restriction levels than the default settings. If you see this error, then run Import-Module ADSync
to make the cmdlet available.
- AllowedSyncCycleInterval. The shortest time interval between synchronization cycles allowed by Azure AD. You cannot synchronize more frequently than this setting and still be supported.
- CurrentlyEffectiveSyncCycleInterval. The schedule currently in effect. It has the same value as CustomizedSyncInterval (if set) if it is not more frequent than AllowedSyncInterval. If you use a build before 1.1.281 and you change CustomizedSyncCycleInterval, this change takes effect after next synchronization cycle. From build 1.1.281 the change takes effect immediately.
- CustomizedSyncCycleInterval. If you want the scheduler to run at any other frequency than the default 30 minutes, then you configure this setting. In the picture above, the scheduler has been set to run every hour instead. If you set this setting to a value lower than AllowedSyncInterval, then the latter is used.
- NextSyncCyclePolicyType. Either Delta or Initial. Defines if the next run should only process delta changes, or if the next run should do a full import and sync. The latter would also reprocess any new or changed rules.
- NextSyncCycleStartTimeInUTC. Next time the scheduler starts the next sync cycle.
- PurgeRunHistoryInterval. The time operation logs should be kept. These logs can be reviewed in the synchronization service manager. The default is to keep these logs for 7 days.
- SyncCycleEnabled. Indicates if the scheduler is running the import, sync, and export processes as part of its operation.
- MaintenanceEnabled. Shows if the maintenance process is enabled. It updates the certificates/keys and purges the operations log.
- StagingModeEnabled. Shows if staging mode is enabled. If this setting is enabled, then it suppresses the exports from running but still run import and synchronization.
- SchedulerSuspended. Set by Connect during an upgrade to temporarily block the scheduler from running.
You can change some of these settings with Set-ADSyncScheduler
. The following parameters can be modified:
- CustomizedSyncCycleInterval
- NextSyncCyclePolicyType
- PurgeRunHistoryInterval
- SyncCycleEnabled
- MaintenanceEnabled
In earlier builds of Azure AD Connect, isStagingModeEnabled was exposed in Set-ADSyncScheduler. It is unsupported to set this property. The property SchedulerSuspended should only be modified by Connect. It is unsupported to set this with PowerShell directly.
The scheduler configuration is stored in Azure AD. If you have a staging server, any change on the primary server also affects the staging server (except IsStagingModeEnabled).
CustomizedSyncCycleInterval
Syntax: Set-ADSyncScheduler -CustomizedSyncCycleInterval d.HH:mm:ss
d - days, HH - hours, mm - minutes, ss - seconds
Example: Set-ADSyncScheduler -CustomizedSyncCycleInterval 03:00:00
Changes the scheduler to run every 3 hours.
Example: Set-ADSyncScheduler -CustomizedSyncCycleInterval 1.0:0:0
Changes change the scheduler to run daily.
Disable the scheduler
If you need to make configuration changes, then you want to disable the scheduler. For example, when you configure filtering or make changes to synchronization rules.
To disable the scheduler, run Set-ADSyncScheduler -SyncCycleEnabled $false
.
When you've made your changes, do not forget to enable the scheduler again with Set-ADSyncScheduler -SyncCycleEnabled $true
.
Start the scheduler
The scheduler is by default run every 30 minutes. In some cases, you might want to run a sync cycle in between the scheduled cycles or you need to run a different type.
Delta sync cycle
A delta sync cycle includes the following steps:
- Delta import on all Connectors
- Delta sync on all Connectors
- Export on all Connectors
Full sync cycle
A full sync cycle includes the following steps:
- Full Import on all Connectors
- Full Sync on all Connectors
- Export on all Connectors
It could be that you have an urgent change that must be synchronized immediately, which is why you need to manually run a cycle.
If you need to manually run a sync cycle, then from PowerShell run Start-ADSyncSyncCycle -PolicyType Delta
.
To initiate a full sync cycle, run Start-ADSyncSyncCycle -PolicyType Initial
from a PowerShell prompt.
Running a full sync cycle can be very time consuming, read the next section to read how to optimize this process.
Sync steps required for different configuration changes
Different configuration changes require different sync steps to ensure the changes are correctly applied to all objects.
- Added more objects or attributes to be imported from a source directory (by adding/modifying the sync rules)
- A Full Import is required on the Connector for that source directory
- Made changes to the Synchronization rules
- A Full Sync is required on the Connector for the changed Synchronization rules
- Changed filtering so a different number of objects should be included
- A Full Import is required on the Connector for each AD Connector UNLESS you are using Attribute-based filtering based on attributes that are already being imported into the sync engine
Customizing a sync cycle run the right mix of Delta and Full sync steps
To avoid running a full sync cycle you can mark specific Connectors to run a Full step using the following cmdlets.
Set-ADSyncSchedulerConnectorOverride -Connector -FullImportRequired $true
Set-ADSyncSchedulerConnectorOverride -Connector -FullSyncRequired $true
Get-ADSyncSchedulerConnectorOverride -Connector
Example: If you made changes to the synchronization rules for Connector 'AD Forest A' that don't require any new attributes to be imported you would run the following cmdlets to run a delta sync cycle which also did a Full Sync step for that Connector.
Set-ADSyncSchedulerConnectorOverride -ConnectorName 'AD Forest A' -FullSyncRequired $true
Start-ADSyncSyncCycle -PolicyType Delta
Example: If you made changes to the synchronization rules for Connector 'AD Forest A' so that they now require a new attribute to be imported you would run the following cmdlets to run a delta sync cycle which also did a Full Import, Full Sync step for that Connector.
Set-ADSyncSchedulerConnectorOverride -ConnectorName 'AD Forest A' -FullImportRequired $true
Set-ADSyncSchedulerConnectorOverride -ConnectorName 'AD Forest A' -FullSyncRequired $true
Start-ADSyncSyncCycle -PolicyType Delta
Stop the scheduler
If the scheduler is currently running a synchronization cycle, you might need to stop it. For example if you start the installation wizard and you get this error:
When a sync cycle is running, you cannot make configuration changes. You could wait until the scheduler has finished the process, but you can also stop it so you can make your changes immediately. Stopping the current cycle is not harmful and pending changes are processed with next run.
Start by telling the scheduler to stop its current cycle with the PowerShell cmdlet
Stop-ADSyncSyncCycle
.If you use a build before 1.1.281, then stopping the scheduler does not stop the current Connector from its current task. To force the Connector to stop, take the following actions:
- Start Synchronization Service from the start menu. Go to Connectors, highlight the Connector with the state Running, and select Stop from the Actions.
The scheduler is still active and starts again on next opportunity.
Custom scheduler
The cmdlets documented in this section are only available in build 1.1.130.0 and later.
If the built-in scheduler does not satisfy your requirements, then you can schedule the Connectors using PowerShell.
Invoke-ADSyncRunProfile
You can start a profile for a Connector in this way:
The names to use for Connector names and Run Profile Names can be found in the Synchronization Service Manager UI.
The Invoke-ADSyncRunProfile
cmdlet is synchronous, that is, it does not return control until the Connector has completed the operation, either successfully or with an error.
When you schedule your Connectors, the recommendation is to schedule them in the following order:
- (Full/Delta) Import from on-premises directories, such as Active Directory
- (Full/Delta) Import from Azure AD
- (Full/Delta) Synchronization from on-premises directories, such as Active Directory
- (Full/Delta) Synchronization from Azure AD
- Export to Azure AD
- Export to on-premises directories, such as Active Directory
This order is how the built-in scheduler runs the Connectors.
Get-ADSyncConnectorRunStatus
You can also monitor the sync engine to see if it is busy or idle. This cmdlet returns an empty result if the sync engine is idle and is not running a Connector. If a Connector is running, it returns the name of the Connector.
In the picture above, the first line is from a state where the sync engine is idle. The second line from when the Azure AD Connector is running.
Scheduler and installation wizard
If you start the installation wizard, then the scheduler is temporarily suspended. This behavior is because it is assumed you make configuration changes and these settings cannot be applied if the sync engine is actively running. For this reason, do not leave the installation wizard open since it stops the sync engine from performing any synchronization actions.
Next steps
Learn more about the Azure AD Connect sync configuration.
Learn more about Integrating your on-premises identities with Azure Active Directory.
-->Azure File Sync is a service that allows you to cache a number of Azure file shares on an on-premises Windows Server or cloud VM.
This article introduces you to Azure File Sync concepts and features. Once you are familiar with Azure File Sync, consider following the Azure File Sync deployment guide to try out this service.
The files will be stored in the cloud in Azure file shares. Azure file shares can be used in two ways: by directly mounting these serverless Azure file shares (SMB) or by caching Azure file shares on-premises using Azure File Sync. Which deployment option you choose changes the aspects you need to consider as you plan for your deployment.
Direct mount of an Azure file share: Since Azure Files provides SMB access, you can mount Azure file shares on-premises or in the cloud using the standard SMB client available in Windows, macOS, and Linux. Because Azure file shares are serverless, deploying for production scenarios does not require managing a file server or NAS device. This means you don't have to apply software patches or swap out physical disks.
Cache Azure file share on-premises with Azure File Sync: Azure File Sync enables you to centralize your organization's file shares in Azure Files, while keeping the flexibility, performance, and compatibility of an on-premises file server. Azure File Sync transforms an on-premises (or cloud) Windows Server into a quick cache of your Azure file share.
Management concepts
An Azure File Sync deployment has three fundamental management objects:
- Azure file share: An Azure file share is a serverless cloud file share, which provides the cloud endpoint of an Azure File Sync sync relationship. Files in an Azure file share can be accessed directly with SMB or the FileREST protocol, although we encourage you to primarily access the files through the Windows Server cache when the Azure file share is being used with Azure File Sync. This is because Azure Files today lacks an efficient change detection mechanism like Windows Server has, so changes to the Azure file share directly will take time to propagate back to the server endpoints.
- Server endpoint: The path on the Windows Server that is being synced to an Azure file share. This can be a specific folder on a volume or the root of the volume. Multiple server endpoints can exist on the same volume if their namespaces do not overlap.
- Sync group: The object that defines the sync relationship between a cloud endpoint, or Azure file share, and a server endpoint. Endpoints within a sync group are kept in sync with each other. If for example, you have two distinct sets of files that you want to manage with Azure File Sync, you would create two sync groups and add different endpoints to each sync group.
Azure file share management concepts
Azure file shares are deployed into storage accounts, which are top-level objects that represent a shared pool of storage. This pool of storage can be used to deploy multiple file shares, as well as other storage resources such as blob containers, queues, or tables. All storage resources that are deployed into a storage account share the limits that apply to that storage account. To see the current limits for a storage account, see Azure Files scalability and performance targets.
There are two main types of storage accounts you will use for Azure Files deployments:
- General purpose version 2 (GPv2) storage accounts: GPv2 storage accounts allow you to deploy Azure file shares on standard/hard disk-based (HDD-based) hardware. In addition to storing Azure file shares, GPv2 storage accounts can store other storage resources such as blob containers, queues, or tables.
- FileStorage storage accounts: FileStorage storage accounts allow you to deploy Azure file shares on premium/solid-state disk-based (SSD-based) hardware. FileStorage accounts can only be used to store Azure file shares; no other storage resources (blob containers, queues, tables, etc.) can be deployed in a FileStorage account. Only FileStorage accounts can deploy both SMB and NFS file shares.
There are several other storage account types you may come across in the Azure portal, PowerShell, or CLI. Two storage account types, BlockBlobStorage and BlobStorage storage accounts, cannot contain Azure file shares. The other two storage account types you may see are general purpose version 1 (GPv1) and classic storage accounts, both of which can contain Azure file shares. Although GPv1 and classic storage accounts may contain Azure file shares, most new features of Azure Files are available only in GPv2 and FileStorage storage accounts. We therefore recommend to only use GPv2 and FileStorage storage accounts for new deployments, and to upgrade GPv1 and classic storage accounts if they already exist in your environment.
Azure File Sync management concepts
Sync groups are deployed into Storage Sync Services, which are top-level objects that register servers for use with Azure File Sync and contain the sync group relationships. The Storage Sync Service resource is a peer of the storage account resource, and can similarly be deployed to Azure resource groups. A Storage Sync Service can create sync groups that contain Azure file shares across multiple storage accounts and multiple registered Windows Servers.
Before you can create a sync group in a Storage Sync Service, you must first register a Windows Server with the Storage Sync Service. This creates a registered server object, which represents a trust relationship between your server or cluster and the Storage Sync Service. To register a Storage Sync Service, you must first install the Azure File Sync agent on the server. An individual server or cluster can be registered with only one Storage Sync Service at a time.
A sync group contains one cloud endpoint, or Azure file share, and at least one server endpoint. The server endpoint object contains the settings that configure the cloud tiering capability, which provides the caching capability of Azure File Sync. In order to sync with an Azure file share, the storage account containing the Azure file share must be in the same Azure region as the Storage Sync Service.
Important
You can make changes to any cloud endpoint or server endpoint in the sync group and have your files synced to the other endpoints in the sync group. If you make a change to the cloud endpoint (Azure file share) directly, changes first need to be discovered by an Azure File Sync change detection job. A change detection job is initiated for a cloud endpoint only once every 24 hours. For more information, see Azure Files frequently asked questions.
Management guidance
When deploying Azure File Sync, we recommend:
Deploying Azure file shares 1:1 with Windows file shares. The server endpoint object gives you a great degree of flexibility on how you set up the sync topology on the server-side of the sync relationship. To simplify management, make the path of the server endpoint match the path of the Windows file share.
Use as few Storage Sync Services as possible. This will simplify management when you have sync groups that contain multiple server endpoints, since a Windows Server can only be registered to one Storage Sync Service at a time.
Paying attention to a storage account's IOPS limitations when deploying Azure file shares. Ideally, you would map file shares 1:1 with storage accounts, however this may not always be possible due to various limits and restrictions, both from your organization and from Azure. When it is not possible to have only one file share deployed in one storage account, consider which shares will be highly active and which shares will be less active to ensure that the hottest file shares don't get put in the same storage account together.
Windows file server considerations
To enable the sync capability on Windows Server, you must install the Azure File Sync downloadable agent. The Azure File Sync agent provides two main components: FileSyncSvc.exe
, the background Windows service that is responsible for monitoring changes on the server endpoints and initiating sync sessions, and StorageSync.sys
, a file system filter that enables cloud tiering and fast disaster recovery.
Operating system requirements
Azure File Sync is supported with the following versions of Windows Server:
Version | Supported SKUs | Supported deployment options |
---|---|---|
Windows Server 2019 | Datacenter, Standard, and IoT | Full and Core |
Windows Server 2016 | Datacenter, Standard, and Storage Server | Full and Core |
Windows Server 2012 R2 | Datacenter, Standard, and Storage Server | Full and Core |
Future versions of Windows Server will be added as they are released.
Important
We recommend keeping all servers that you use with Azure File Sync up to date with the latest updates from Windows Update.
Minimum system resources
Azure File Sync requires a server, either physical or virtual, with at least one CPU and a minimum of 2 GiB of memory.
Important
If the server is running in a virtual machine with dynamic memory enabled, the VM should be configured with a minimum of 2048 MiB of memory.
For most production workloads, we do not recommend configuring an Azure File Sync sync server with only the minimum requirements. See Recommended system resources for more information.
Recommended system resources
Just like any server feature or application, the system resource requirements for Azure File Sync are determined by the scale of the deployment; larger deployments on a server require greater system resources. For Azure File Sync, scale is determined by the number of objects across the server endpoints and the churn on the dataset. A single server can have server endpoints in multiple sync groups and the number of objects listed in the following table accounts for the full namespace that a server is attached to.
For example, server endpoint A with 10 million objects + server endpoint B with 10 million objects = 20 million objects. For that example deployment, we would recommend 8 CPUs, 16 GiB of memory for steady state, and (if possible) 48 GiB of memory for the initial migration.
Namespace data is stored in memory for performance reasons. Because of that, bigger namespaces require more memory to maintain good performance, and more churn requires more CPU to process.
In the following table, we have provided both the size of the namespace as well as a conversion to capacity for typical general purpose file shares, where the average file size is 512 KiB. If your file sizes are smaller, consider adding additional memory for the same amount of capacity. Base your memory configuration on the size of the namespace.
Namespace size - files & directories (millions) | Typical capacity (TiB) | CPU Cores | Recommended memory (GiB) |
---|---|---|---|
3 | 1.4 | 2 | 8 (initial sync)/ 2 (typical churn) |
5 | 2.3 | 2 | 16 (initial sync)/ 4 (typical churn) |
10 | 4.7 | 4 | 32 (initial sync)/ 8 (typical churn) |
30 | 14.0 | 8 | 48 (initial sync)/ 16 (typical churn) |
50 | 23.3 | 16 | 64 (initial sync)/ 32 (typical churn) |
100* | 46.6 | 32 | 128 (initial sync)/ 32 (typical churn) |
*Syncing more than 100 million files & directories is not recommended at this time. This is a soft limit based on our tested thresholds. For more information, see Azure Files scalability and performance targets.
Tip
Initial synchronization of a namespace is an intensive operation and we recommend allocating more memory until initial synchronization is complete. This isn't required but, may speed up initial sync.
Typical churn is 0.5% of the namespace changing per day. For higher levels of churn, consider adding more CPU.
- A locally attached volume formatted with the NTFS file system.
Evaluation cmdlet
Before deploying Azure File Sync, you should evaluate whether it is compatible with your system using the Azure File Sync evaluation cmdlet. This cmdlet checks for potential issues with your file system and dataset, such as unsupported characters or an unsupported operating system version. Its checks cover most but not all of the features mentioned below; we recommend you read through the rest of this section carefully to ensure your deployment goes smoothly.
The evaluation cmdlet can be installed by installing the Az PowerShell module, which can be installed by following the instructions here: Install and configure Azure PowerShell.
Usage
You can invoke the evaluation tool in a few different ways: you can perform the system checks, the dataset checks, or both. To perform both the system and dataset checks:
To test only your dataset:
To test system requirements only:
To display the results in CSV:
File system compatibility
Azure File Sync is only supported on directly attached, NTFS volumes. Direct attached storage, or DAS, on Windows Server means that the Windows Server operating system owns the file system. DAS can be provided through physically attaching disks to the file server, attaching virtual disks to a file server VM (such as a VM hosted by Hyper-V), or even through ISCSI.
Only NTFS volumes are supported; ReFS, FAT, FAT32, and other file systems are not supported.
The following table shows the interop state of NTFS file system features:
Feature | Support status | Notes |
---|---|---|
Access control lists (ACLs) | Fully supported | Windows-style discretionary access control lists are preserved by Azure File Sync, and are enforced by Windows Server on server endpoints. ACLs can also be enforced when directly mounting the Azure file share, however this requires additional configuration. See the Identity section for more information. |
Hard links | Skipped | |
Symbolic links | Skipped | |
Mount points | Partially supported | Mount points might be the root of a server endpoint, but they are skipped if they are contained in a server endpoint's namespace. |
Junctions | Skipped | For example, Distributed File System DfrsrPrivate and DFSRoots folders. |
Reparse points | Skipped | |
NTFS compression | Fully supported | |
Sparse files | Fully supported | Sparse files sync (are not blocked), but they sync to the cloud as a full file. If the file contents change in the cloud (or on another server), the file is no longer sparse when the change is downloaded. |
Alternate Data Streams (ADS) | Preserved, but not synced | For example, classification tags created by the File Classification Infrastructure are not synced. Existing classification tags on files on each of the server endpoints are left untouched. |
Azure File Sync will also skip certain temporary files and system folders:
File/folder | Note |
---|---|
pagefile.sys | File specific to system |
Desktop.ini | File specific to system |
thumbs.db | Temporary file for thumbnails |
ehthumbs.db | Temporary file for media thumbnails |
~$*.* | Office temporary file |
*.tmp | Temporary file |
*.laccdb | Access DB locking file |
635D02A9D91C401B97884B82B3BCDAEA.* | Internal Sync file |
System Volume Information | Folder specific to volume |
$RECYCLE.BIN | Folder |
SyncShareState | Folder for Sync |
Failover Clustering
Windows Server Failover Clustering is supported by Azure File Sync for the 'File Server for general use' deployment option. Failover Clustering is not supported on 'Scale-Out File Server for application data' (SOFS) or on Clustered Shared Volumes (CSVs). Direct mail 4 3 5 download free.
Note
The Azure File Sync agent must be installed on every node in a Failover Cluster for sync to work correctly.
Data Deduplication
Windows Server 2016 and Windows Server 2019
Data Deduplication is supported irrespective of whether cloud tiering is enabled or disabled on one or more server endpoints on the volume for Windows Server 2016 and Windows Server 2019. Enabling Data Deduplication on a volume with cloud tiering enabled lets you cache more files on-premises without provisioning more storage.
When Data Deduplication is enabled on a volume with cloud tiering enabled, Dedup optimized files within the server endpoint location will be tiered similar to a normal file based on the cloud tiering policy settings. Once the Dedup optimized files have been tiered, the Data Deduplication garbage collection job will run automatically to reclaim disk space by removing unnecessary chunks that are no longer referenced by other files on the volume.
Note the volume savings only apply to the server; your data in the Azure file share will not be deduped.
Note
To support Data Deduplication on volumes with cloud tiering enabled on Windows Server 2019, Windows update KB4520062 must be installed and Azure File Sync agent version 9.0.0.0 or newer is required.
Windows Server 2012 R2
Azure File Sync does not support Data Deduplication and cloud tiering on the same volume on Windows Server 2012 R2. If Data Deduplication is enabled on a volume, cloud tiering must be disabled.
Notes
If Data Deduplication is installed prior to installing the Azure File Sync agent, a restart is required to support Data Deduplication and cloud tiering on the same volume.
If Data Deduplication is enabled on a volume after cloud tiering is enabled, the initial Deduplication optimization job will optimize files on the volume that are not already tiered and will have the following impact on cloud tiering:
- Free space policy will continue to tier files as per the free space on the volume by using the heatmap.
- Date policy will skip tiering of files that may have been otherwise eligible for tiering due to the Deduplication optimization job accessing the files.
For ongoing Deduplication optimization jobs, cloud tiering with date policy will get delayed by the Data Deduplication MinimumFileAgeDays setting, if the file is not already tiered.
- Example: If the MinimumFileAgeDays setting is seven days and cloud tiering date policy is 30 days, the date policy will tier files after 37 days.
- Note: Once a file is tiered by Azure File Sync, the Deduplication optimization job will skip the file.
If a server running Windows Server 2012 R2 with the Azure File Sync agent installed is upgraded to Windows Server 2016 or Windows Server 2019, the following steps must be performed to support Data Deduplication and cloud tiering on the same volume:
- Uninstall the Azure File Sync agent for Windows Server 2012 R2 and restart the server.
- Download the Azure File Sync agent for the new server operating system version (Windows Server 2016 or Windows Server 2019).
- Install the Azure File Sync agent and restart the server.
Note: The Azure File Sync configuration settings on the server are retained when the agent is uninstalled and reinstalled.
Distributed File System (DFS)
Azure File Sync supports interop with DFS Namespaces (DFS-N) and DFS Replication (DFS-R).
DFS Namespaces (DFS-N): Azure File Sync is fully supported on DFS-N servers. You can install the Azure File Sync agent on one or more DFS-N members to sync data between the server endpoints and the cloud endpoint. For more information, see DFS Namespaces overview.
DFS Replication (DFS-R): Since DFS-R and Azure File Sync are both replication solutions, in most cases, we recommend replacing DFS-R with Azure File Sync. There are however several scenarios where you would want to use DFS-R and Azure File Sync together:
- You are migrating from a DFS-R deployment to an Azure File Sync deployment. For more information, see Migrate a DFS Replication (DFS-R) deployment to Azure File Sync.
- Not every on-premises server that needs a copy of your file data can be connected directly to the internet.
- Branch servers consolidate data onto a single hub server, for which you would like to use Azure File Sync.
For Azure File Sync and DFS-R to work side by side:
- Azure File Sync cloud tiering must be disabled on volumes with DFS-R replicated folders.
- Server endpoints should not be configured on DFS-R read-only replication folders.
For more information, see DFS Replication overview.
Sync Checker 3 2019 Download
Sysprep
Using sysprep on a server that has the Azure File Sync agent installed is not supported and can lead to unexpected results. Agent installation and server registration should occur after deploying the server image and completing sysprep mini-setup.
Windows Search
If cloud tiering is enabled on a server endpoint, files that are tiered are skipped and not indexed by Windows Search. Non-tiered files are indexed properly.
Other Hierarchical Storage Management (HSM) solutions
No other HSM solutions should be used with Azure File Sync.
Identity
Azure File Sync works with your standard AD-based identity without any special setup beyond setting up sync. When you are using Azure File Sync, the general expectation is that most accesses go through the Azure File Sync caching servers, rather than through the Azure file share. Since the server endpoints are located on Windows Server, and Windows Server has supported AD and Windows-style ACLs for a long time, nothing is needed beyond ensuring the Windows file servers registered with the Storage Sync Service are domain joined. Azure File Sync will store ACLs on the files in the Azure file share, and will replicate them to all server endpoints.
Even though changes made directly to the Azure file share will take longer to sync to the server endpoints in the sync group, you may also want to ensure that you can enforce your AD permissions on your file share directly in the cloud as well. To do this, you must domain join your storage account to your on-premises AD, just like how your Windows file servers are domain joined. To learn more about domain joining your storage account to a customer-owned Active Directory, see Azure Files Active Directory overview.
Important
Domain joining your storage account to Active Directory is not required to successfully deploy Azure File Sync. This is a strictly optional step that allows the Azure file share to enforce on-premises ACLs when users mount the Azure file share directly.
Networking
The Azure File Sync agent communicates with your Storage Sync Service and Azure file share using the Azure File Sync REST protocol and the FileREST protocol, both of which always use HTTPS over port 443. SMB is never used to upload or download data between your Windows Server and the Azure file share. Because most organizations allow HTTPS traffic over port 443, as a requirement for visiting most websites, special networking configuration is usually not required to deploy Azure File Sync.
Based on your organization's policy or unique regulatory requirements, you may require more restrictive communication with Azure, and therefore Azure File Sync provides several mechanisms for you configure networking. Based on your requirements, you can:
- Tunnel sync and file upload/download traffic over your ExpressRoute or Azure VPN.
- Make use of Azure Files and Azure Networking features such as service endpoints and private endpoints.
- Configure Azure File Sync to support your proxy in your environment.
- Throttle network activity from Azure File Sync.
To learn more about Azure File Sync and networking, see Azure File Sync networking considerations.
Encryption
When using Azure File Sync, there are three different layers of encryption to consider: encryption on the at-rest storage of Windows Server, encryption in transit between the Azure File Sync agent and Azure, and encryption at rest of your data in the Azure file share.
Windows Server encryption at rest
There are two strategies for encrypting data on Windows Server that work generally with Azure File Sync: encryption beneath the file system such that the file system and all of the data written to it is encrypted, and encryption within the file format itself. These methods are not mutually exclusive; they can be used together if desired since the purpose of encryption is different.
To provide encryption beneath the file system, Windows Server provides BitLocker inbox. BitLocker is fully transparent to Azure File Sync. The primary reason to use an encryption mechanism like BitLocker is to prevent physical exfiltration of data from your on-premises datacenter by someone stealing the disks and to prevent sideloading an unauthorized OS to perform unauthorized reads/writes to your data. To learn more about BitLocker, see BitLocker overview.
Third-party products that work similarly to BitLocker, in that they sit beneath the NTFS volume, should similarly work fully transparently with Azure File Sync.
The other main method for encrypting data is to encrypt the file's data stream when the application saves the file. Some applications may do this natively, however this is usually not the case. An example of a method for encrypting the file's data stream is Azure Information Protection (AIP)/Azure Rights Management Services (Azure RMS)/Active Directory RMS. The primary reason to use an encryption mechanism like AIP/RMS is to prevent data exfiltration of data from your file share by people copying it to alternate locations, like to a flash drive, or emailing it to an unauthorized person. When a file's data stream is encrypted as part of the file format, this file will continue to be encrypted on the Azure file share.
Azure File Sync does not interoperate with NTFS Encrypted File System (NTFS EFS) or third-party encryption solutions that sit above the file system but below the file's data stream.
Encryption in transit
Note
Azure File Sync service will remove support for TLS1.0 and 1.1 on August 1st, 2020. All supported Azure File Sync agent versions already use TLS1.2 by default. Using an earlier version of TLS could occur if TLS1.2 was disabled on your server or a proxy is used. If you are using a proxy, we recommend you check the proxy configuration. Azure File Sync service regions added after 5/1/2020 will only support TLS1.2 and support for TLS1.0 and 1.1 will be removed from existing regions on August 1st, 2020. For more information, see the troubleshooting guide.
Azure File Sync agent communicates with your Storage Sync Service and Azure file share using the Azure File Sync REST protocol and the FileREST protocol, both of which always use HTTPS over port 443. Azure File Sync does not send unencrypted requests over HTTP.
Azure storage accounts contain a switch for requiring encryption in transit, which is enabled by default. Even if the switch at the storage account level is disabled, meaning that unencrypted connections to your Azure file shares are possible, Azure File Sync will still only used encrypted channels to access your file share.
The primary reason to disable encryption in transit for the storage account is to support a legacy application that must be run on an older operating system, such as Windows Server 2008 R2 or older Linux distribution, talking to an Azure file share directly. If the legacy application talks to the Windows Server cache of the file share, toggling this setting will have no effect.
We strongly recommend ensuring encryption of data in-transit is enabled.
For more information about encryption in transit, see requiring secure transfer in Azure storage.
Azure file share encryption at rest
All data stored in Azure Files is encrypted at rest using Azure storage service encryption (SSE). Storage service encryption works similarly to BitLocker on Windows: data is encrypted beneath the file system level. Because data is encrypted beneath the Azure file share's file system, as it's encoded to disk, you don't have to have access to the underlying key on the client to read or write to the Azure file share. Encryption at rest applies to both the SMB and NFS protocols.
By default, data stored in Azure Files is encrypted with Microsoft-managed keys. With Microsoft-managed keys, Microsoft holds the keys to encrypt/decrypt the data, and is responsible for rotating them on a regular basis. You can also choose to manage your own keys, which gives you control over the rotation process. If you choose to encrypt your file shares with customer-managed keys, Azure Files is authorized to access your keys to fulfill read and write requests from your clients. With customer-managed keys, you can revoke this authorization at any time, but this means that your Azure file share will no longer be accessible via SMB or the FileREST API.
Azure Files uses the same encryption scheme as the other Azure storage services such as Azure Blob storage. To learn more about Azure storage service encryption (SSE), see Azure storage encryption for data at rest.
Storage tiers
Azure Files offers four different tiers of storage, premium, transaction optimized, hot, and cool to allow you to tailor your shares to the performance and price requirements of your scenario:
- Premium: Premium file shares are backed by solid-state drives (SSDs) and provide consistent high performance and low latency, within single-digit milliseconds for most IO operations, for IO-intensive workloads. Premium file shares are suitable for a wide variety of workloads like databases, web site hosting, and development environments. Premium file shares can be used with both Server Message Block (SMB) and Network File System (NFS) protocols.
- Transaction optimized: Transaction optimized file shares enable transaction heavy workloads that don't need the latency offered by premium file shares. Transaction optimized file shares are offered on the standard storage hardware backed by hard disk drives (HDDs). Transaction optimized has historically been called 'standard', however this refers to the storage media type rather than the tier itself (the hot and cool are also 'standard' tiers, because they are on standard storage hardware).
- Hot: Hot file shares offer storage optimized for general purpose file sharing scenarios such as team shares. Hot file shares are offered on the standard storage hardware backed by HDDs.
- Cool: Cool file shares offer cost-efficient storage optimized for online archive storage scenarios. Cool file shares are offered on the standard storage hardware backed by HDDs.
Premium file shares are deployed in the FileStorage storage account kind and are only available in a provisioned billing model. For more information on the provisioned billing model for premium file shares, see Understanding provisioning for premium file shares. Standard file shares, including transaction optimized, hot, and cool file shares, are deployed in the general purpose version 2 (GPv2) storage account kind, and are available through pay as you go billing. Hot and cool file shares are available in all Azure Public and Azure Government regions. Transaction optimized file shares are available in all Azure regions, including Azure China and Azure Germany regions.
When selecting a storage tier for your workload, consider your performance and usage requirements. If your workload requires single-digit latency, or you are using SSD storage media on-premises, the premium tier is probably the best fit. If low latency isn't as much of a concern, for example with team shares mounted on-premises from Azure or cached on-premises using Azure File Sync, standard storage may be a better fit from a cost perspective.
Once you've created a file share in a storage account, you cannot move it to tiers exclusive to different storage account kinds. For example, to move a transaction optimized file share to the premium tier, you must create a new file share in a FileStorage storage account and copy the data from your original share to a new file share in the FileStorage account. We recommend using AzCopy to copy data between Azure file shares, but you may also use tools like robocopy
on Windows or rsync
for macOS and Linux.
File shares deployed within GPv2 storage accounts can be moved between the standard tiers (transaction optimized, hot, and cool) without creating a new storage account and migrating data, but you will incur transaction costs when you change your tier. When you move a share from a hotter tier to a cooler tier, you will incur the cooler tier's write transaction charge for each file in the share. Moving a file share from a cooler tier to a hotter tier will incur the cool tier's read transaction charge for each file in the share.
See Understanding Azure Files billing for more information.
Enable standard file shares to span up to 100 TiB
By default, standard file shares can span only up to 5 TiB, but you can increase the share limit to 100 TiB. To increase your share limit, enable Large file share on your storage account. Premium storage accounts (FileStorage storage accounts) don't have the large file share feature flag as all premium file shares are already enabled for provisioning up to the full 100-TiB capacity.
You can only enable large file shares on locally redundant or zone redundant standard storage accounts. Once you have enabled the large file share feature flag, you can't change the redundancy level to geo-redundant or geo-zone-redundant storage.
To enable large file shares on an existing storage account, navigate to File shares in the storage account's table of contents.On this blade, select Share capacity, change the share capacity to 100 TiB and select Save.
You can also enable 100-TiB file shares through the Set-AzStorageAccount
PowerShell cmdlet and the az storage account update
Azure CLI command. For detailed instructions on enabling large files shares, see enable and create large file shares.
To learn more about how to create file shares on new storage accounts, see creating an Azure file share.
Regional availability
Standard file shares with 100 TiB capacity have certain limitations.
- Currently, only locally redundant storage (LRS) and zone redundant storage (ZRS) accounts are supported.
- Once you enable large file shares, you cannot convert storage accounts to geo-redundant storage (GRS) or geo-zone-redundant storage (GZRS) accounts.
- Once you enable large file shares, you can't disable it.
Azure file sync region availability
For regional availability, see Products available by region.
The following regions require you to request access to Azure Storage before you can use Azure File Sync with them:
- France South
- South Africa West
- UAE Central
To request access for these regions, follow the process in this document.
Redundancy
To protect the data in your Azure file shares against data loss or corruption, all Azure file shares store multiple copies of each file as they are written. Depending on the requirements of your workload, you can select additional degrees of redundancy. Azure Files currently supports the following data redundancy options:
- Locally redundant: Locally redundant storage, often referred to as LRS, means that every file is stored three times within an Azure storage cluster. This protects against loss of data due to hardware faults, such as a bad disk drive.
- Zone redundant: Zone redundant storage, often referred to as ZRS, means that every file is stored three times across three distinct Azure storage clusters. Just like with locally redundant storage, zone redundancy gives you three copies of each file, however these copies are physically isolated in three distinct storage clusters in different Azure availability zones. Availability zones are unique physical locations within an Azure region. Each zone is made up of one or more datacenters equipped with independent power, cooling, and networking. A write to storage is not accepted until it is written to the storage clusters in all three availability zones.
- Geo-redundant: Geo-redundant storage, often referred to as GRS, is like locally redundant storage, in that a file is stored three times within an Azure storage cluster in the primary region. All writes are then asynchronously replicated to a Microsoft-defined secondary region. Geo-redundant storage provides 6 copies of your data spread between two Azure regions. In the event of a major disaster such as the permanent loss of an Azure region due to a natural disaster or other similar event, Microsoft will perform a failover so that the secondary in effect becomes the primary, serving all operations. Since the replication between the primary and secondary regions are asynchronous, in the event of a major disaster, data not yet replicated to the secondary region will be lost. You can also perform a manual failover of a geo-redundant storage account.
- Geo-zone redundant: Geo-zone redundant storage, often referred to as GZRS, is like zone redundant storage, in that a file is stored three times across three distinct storage clusters in the primary region. All writes are then asynchronously replicated to a Microsoft-defined secondary region. The failover process for geo-zone-redundant storage works the same as it does for geo-redundant storage.
Standard Azure file shares support all four redundancy types, while premium Azure file shares only support locally redundant and zone redundant storage.
Sync Checker 3 2019 Online
General purpose version 2 (GPv2) storage accounts provide two additional redundancy options that are not supported by Azure Files: read accessible geo-redundant storage, often referred to as RA-GRS, and read accessible geo-zone-redundant storage, often referred to as RA-GZRS. You can provision Azure file shares in storage accounts with these options set, however Azure Files does not support reading from the secondary region. Azure file shares deployed into read-accessible geo- or geo-zone redundant storage accounts will be billed as geo-redundant or geo-zone-redundant storage, respectively.
Important
Geo-redundant and Geo-zone redundant storage have the capability to manually failover storage to the secondary region. We recommend that you do not do this outside of a disaster when you are using Azure File Sync because of the increased likelihood of data loss. In the event of a disaster where you would like to initiate a manual failover of storage, you will need to open up a support case with Microsoft to get Azure File Sync to resume sync with the secondary endpoint.
Migration
If you have an existing Windows file server, Azure File Sync can be directly installed in place, without the need to move data over to a new server. If you are planning to migrate to a new Windows file server as a part of adopting Azure File Sync, there are several possible approaches to move data over:
Create server endpoints for your old file share and your new file share and let Azure File Sync synchronize the data between the server endpoints. The advantage to this approach is that it makes it very easy to oversubscribe the storage on your new file server, since Azure File Sync is cloud tiering aware. When you are ready, you can cut over end users to the file share on the new server and remove the old file share's server endpoint.
Create a server endpoint only on the new file server, and copy data into from the old file share using
robocopy
. Depending on the topology of file shares on your new server (how many shares you have on each volume, how free each volume is, etc.), you may need to temporarily provision additional storage as it is expected thatrobocopy
from your old server to your new server within your on-premises datacenter will complete faster than Azure File Sync will move data into Azure.
It is also possible to use Data Box to migrate data into an Azure File Sync deployment. Most of the time, when customers want to use Data Box to ingest data, they do so because they think it will increase the speed of their deployment or because it will help with constrained bandwidth scenarios. While it's true that using a Data Box to ingest data into your Azure File Sync deployment will decrease bandwidth utilization, it will likely be faster for most scenarios to pursue an online data upload through one of the methods described above. To learn more about how to use Data Box to ingest data into your Azure File Sync deployment, see Migrate data into Azure File Sync with Azure Data Box.
A common mistake customers make when migrating data into their new Azure File Sync deployment is to copy data directly into the Azure file share, rather than on their Windows file servers. Although Azure File Sync will identify all of the new files on the Azure file share, and sync them back to your Windows file shares, this is generally considerably slower than loading data through the Windows file server. When using Azure copy tools, such as AzCopy, it is important to use the latest version. Check the file copy tools table to get an overview of Azure copy tools to ensure you can copy all of the important metadata of a file such as timestamps and ACLs.
Antivirus
Because antivirus works by scanning files for known malicious code, an antivirus product might cause the recall of tiered files, resulting in high egress charges. In versions 4.0 and above of the Azure File Sync agent, tiered files have the secure Windows attribute FILE_ATTRIBUTE_RECALL_ON_DATA_ACCESS set. We recommend consulting with your software vendor to learn how to configure their solution to skip reading files with this attribute set (many do it automatically).
Microsoft's in-house antivirus solutions, Windows Defender and System Center Endpoint Protection (SCEP), both automatically skip reading files that have this attribute set. We have tested them and identified one minor issue: when you add a server to an existing sync group, files smaller than 800 bytes are recalled (downloaded) on the new server. These files will remain on the new server and will not be tiered since they do not meet the tiering size requirement (>64kb).
Note
Antivirus vendors can check compatibility between their product and Azure File Sync using the Azure File Sync Antivirus Compatibility Test Suite, which is available for download on the Microsoft Download Center.
Backup
If cloud tiering is enabled, solutions that directly back up the server endpoint or a VM on which the server endpoint is located should not be used. Cloud tiering causes only a subset of your data to be stored on the server endpoint, with the full dataset residing in your Azure file share. Depending on the backup solution used, tiered files will either be skipped and not backed up (because they have the FILE_ATTRIBUTE_RECALL_ON_DATA_ACCESS attribute set), or they will be recalled to disk, resulting in high egress charges. We recommend using a cloud backup solution to back up the Azure file share directly. For more information, see About Azure file share backup or contact your backup provider to see if they support backing up Azure file shares.
If you prefer to use an on-premises backup solution, backups should be performed on a server in the sync group that has cloud tiering disabled. When performing a restore, use the volume-level or file-level restore options. Files restored using the file-level restore option will be synced to all endpoints in the sync group and existing files will be replaced with the version restored from backup. Volume-level restores will not replace newer file versions in the Azure file share or other server endpoints.
Note
Bare-metal (BMR) restore can cause unexpected results and is not currently supported.
Note
With Version 9 of the Azure File Sync agent, VSS snapshots (including Previous Versions tab) are now supported on volumes which have cloud tiering enabled. However, you must enable previous version compatibility through PowerShell. Learn how.
Azure File Sync agent update policy
The Azure File Sync agent is updated on a regular basis to add new functionality and to address issues. We recommend you configure Microsoft Update to get updates for the Azure File Sync agent as they're available.
Major vs. minor agent versions
- Major agent versions often contain new features and have an increasing number as the first part of the version number. For example: *2.*.**
- Minor agent versions are also called 'patches' and are released more frequently than major versions. They often contain bug fixes and smaller improvements but no new features. For example: **.3.**
Upgrade paths
There are four approved and tested ways to install the Azure File Sync agent updates.
- (Preferred) Configure Microsoft Update to automatically download and install agent updates.
We always recommend taking every Azure File Sync update to ensure you have access to the latest fixes for the server agent. Microsoft Update makes this process seamless, by automatically downloading and installing updates for you. - Use AfsUpdater.exe to download and install agent updates.
The AfsUpdater.exe is located in the agent installation directory. Double-click the executable to download and install agent updates. - Patch an existing Azure File Sync agent by using a Microsoft Update patch file, or a .msp executable. The latest Azure File Sync update package can be downloaded from the Microsoft Update Catalog.
Running a .msp executable will upgrade your Azure File Sync installation with the same method used automatically by Microsoft Update in the previous upgrade path. Applying a Microsoft Update patch will perform an in-place upgrade of an Azure File Sync installation. - Download the newest Azure File Sync agent installer from the Microsoft Download Center.
To upgrade an existing Azure File Sync agent installation, uninstall the older version and then install the latest version from the downloaded installer. The server registration, sync groups, and any other settings are maintained by the Azure File Sync installer.
Automatic agent lifecycle management
With agent version 6, the file sync team has introduced an agent auto-upgrade feature. You can select either of two modes and specify a maintenance window in which the upgrade shall be attempted on the server. This feature is designed to help you with the agent lifecycle management by either providing a guardrail preventing your agent from expiration or allowing for a no-hassle, stay current setting.
- The default setting will attempt to prevent the agent from expiration. Within 21 days of the posted expiration date of an agent, the agent will attempt to self-upgrade. It will start an attempt to upgrade once a week within 21 days prior to expiration and in the selected maintenance window. This option does not eliminate the need for taking regular Microsoft Update patches.
- Optionally, you can select that the agent will automatically upgrade itself as soon as a new agent version becomes available (currently not applicable to clustered servers). This update will occur during the selected maintenance window and allow your server to benefit from new features and improvements as soon as they become generally available. This is the recommended, worry-free setting that will provide major agent versions as well as regular update patches to your server. Every agent released is at GA quality. If you select this option, Microsoft will flight the newest agent version to you. Clustered servers are excluded. Once flighting is complete, the agent will also become available on Microsoft Download Center aka.ms/AFS/agent.
Changing the auto-upgrade setting
The following instructions describe how to change the settings after you've completed the installer, if you need to make changes.
Open a PowerShell console and navigate to the directory where you installed the sync agent then import the server cmdlets. By default this would look something like this:
You can run Get-StorageSyncAgentAutoUpdatePolicy
to check the current policy setting and determine if you want to change it.
To change the current policy setting to the delayed update track, you can use:
To change the current policy setting to the immediate update track, you can use:
Agent lifecycle and change management guarantees
Azure File Sync is a cloud service, which continuously introduces new features and improvements. This means that a specific Azure File Sync agent version can only be supported for a limited time. To facilitate your deployment, the following rules guarantee you have enough time and notification to accommodate agent updates/upgrades in your change management process:
- Major agent versions are supported for at least six months from the date of initial release.
- We guarantee there is an overlap of at least three months between the support of major agent versions.
- Warnings are issued for registered servers using a soon-to-be expired agent at least three months prior to expiration. You can check if a registered server is using an older version of the agent under the registered servers section of a Storage Sync Service.
- The lifetime of a minor agent version is bound to the associated major version. For example, when agent version 3.0 is released, agent versions 2.* will all be set to expire together.
Note
Installing an agent version with an expiration warning will display a warning but succeed. Attempting to install or connect with an expired agent version is not supported and will be blocked.