Azure Data Factory is a data-integration service based on the Microsoft Azure Cloud & Data Factory is the best ETL tool on Azure Cloud. Data Factory is designed to deliver extraction (E), transformation (T), and loading (L) processes within the cloud environment.
 |
| Azure Data Factory |
we are focusing on the below topics in this tutorial
- Azure Data Factory History
- What is Azure Data Factory?
- What Can you do in ADF?
- Why Azure Data Factory?
- What is ETL (Extract, Transfer, and Load)?
- How Does Azure Data Factory Differ from other ETL Tools?
Azure Data Factory History
we look at what ADF is today, let's take a quick look at ADF history.
1. Azure Data Factory v1
ADF went into public preview on October 28th, 2014, and became generally
available on August 6th, 2015. Back then, it was a fairly limited tool for processing time-sliced for different data. It did that part really well, but it couldn't even begin to compete with the mature and feature-rich SQL Server Integration Services (SSIS). In the early days of ADF, you developed solutions in Visual Studio, and even though they made improvements to the diagram view, there was a lot of JSON editing involved. It was a very different world just a few years ago as I have seen.
But then! Something happened at Microsoft Ignite 2017.
2. Azure Data Factory v2
went into public preview on September 25th, 2017. It was branded as v2 because it had so many unique and ned features and capabilities that it was almost a new product. You could now lift
and shift your existing SQL Server Integration Services (SSIS) solutions to Azure Cloud (into ADF). But more importantly. you could now do amazing things: like looping and branching and even running pipelines on a wall-cock schedule in addition to the regular intervals. ADF v2 become even better when the new visual tool was enabled in public preview on Jan-16-2018.
What is Azure Data Factory?
ADF: is a hybrid data integration service that enables you to quickly and efficiently create automated data pipelines - without having to write any code!: You can do many things in ADF.
ADF: completed the data integration story by adding new data transformation capabilities called Data Flows. Now you can both copy the data and transform the data in the same user interface, making ADF a complete ETL and data integration tool.
What Can you do in Azure Data Factory ?
We can do many things with ADF but I would like to simplify it to 2 main things copy the data from a different source and transform the data
Why Azure Data Factory?
to give the report to higher officials and reporting managers or to analyze the data by using tools like BI tools (Quick view, Qlicksens, PowerBI, SaS Bi, etc.).
You need all the required data in your hand to understand what has happened in the past, predict what may happen in the future, discover patterns and anomalies, and gain the insight necessary for making faster and better decisions.
But before you can do any of those things, you need to collect, store the data from different sources, transform integrate and prepare your data.
What is ETL (Extract, Transfer, and Load)?
The ETL (extract, transform, load) process is the most popular and well-known method of collecting data from multiple sources and loading it into a centralized data warehouse.
During the ETL process, information is first extracted from a source such as a database from SQL Server or DB2 or Oracle, etc, file, or spreadsheet, then transformed to comply with the data warehouse’s standards, and finally loaded into the data warehouse.
Top ETL Tools in the market.
- Informatica
- Talend
- Microsoft -SSIS
- IBM - Infosphere information server
- AWS - Glue
How Does Azure Data Factory Differ from other ETL Tools?
There are some nice features that distinguish ADF from the rest of the cloud pack. ADF has a clean user interface to define dataflows, also .NET, REST, and Python APIs if you feel the need to code your own ETL tasks. It’s also got the ability to run SSIS packages too. That’s handy if you've got some legacy SSIS integrations hanging around and are looking for a path to something more modern. Finally, there’s a windows-based
integration runtime (aka data gateway) that helps you retrieve data from “on-premises” systems
Post a Comment (0)