Data Analytics
Azure Data Factory
The ingestion and orchestration layer that moves data from ERP, databases, files, and APIs into the warehouse or the lakehouse.
Product
Azure Data FactoryAzure Data Factory is where the ingestion, orchestration, and integration layer is built in the cloud. It connects the source systems an organisation already runs, including ERP, operational databases, file locations, and APIs, and moves that data into the data warehouse or the lakehouse. Pipelines are scheduled and monitored, so the data behind a report has a known age rather than an assumed one.
How it works
- Source systems are inventoried with the teams that own them, including how often each can be read and what a safe extraction window looks like.
- Linked services and datasets are configured for each source, whether an ERP, a relational database, a file location, or an API.
- Pipelines move the data into the landing layer of the warehouse or the lakehouse, with transformation applied where the target model requires it.
- Pipelines are scheduled and their dependencies ordered, so a downstream load does not run against an upstream extract that failed.
- Runs are monitored and failures alerted, so a missing refresh is found before the dashboard is read rather than after.
A problem it solves
Data sits in an ERP, in several operational databases, in files exported by hand each month, and behind an API written for another purpose, so reporting depends on somebody remembering to collect each of them. Azure Data Factory turns that collection into scheduled pipelines with monitoring, so the work stops being manual and a failed load is visible rather than silent.