Microsoft Ignite Announcements Nov 2024
Announced at Microsoft Ignite last week were some new product features related to the data platform and AI. Check out the Major announcements and Book of News. Below are the ones I found most interesting:
Fabric-related:
- Fabric Databases, now in public preview and being rolled out to various regions (it will be available in all Fabric regions by early December). Fabric Databases represent a new class of cloud databases that brings a world-class transactional database natively to Microsoft Fabric. Fabric now brings together both transactional and analytical workloads, creating a truly unified data platform. SQL database, the first database available in Fabric Databases, was built on the SQL Server engine and the simple and intuitive SaaS platform of Fabric. Data professionals who’ve tried SQL database in Fabric were able to complete common database tasks up to 71% faster and with 63% more effective task completion. Data in SQL database is automatically mirrored to Fabric OneLake, making it easy to combine the SQL database data with other data and making it available to other platforms. SQL database is just the beginning for Fabric Databases, with more databases on the roadmap. Customer scenarios I can envision to use a Fabric SQL database include a: 1) Metadata driven framework, where you need a control table in a database to drive an ETL process (see build large-scale data copy pipelines with metadata-driven approach in copy data tool); 2) Data warehouse, when the scale of Fabric Data Warehouse is not needed or you need to use a T-SQL feature not available in a Fabric Data Warehouse; 3) Digital native app development by low-code or no-code developers, where a database is needed and the developers don’t know anything about managing databases and they don’t want to know; 4) Curated data/Reverse ETL/Cache, where data is copied from the data warehouse into a SQL database for analysts to query, especially when the analysts want the interface to the data to look like it did before (by using their existing tools); 5) Hybrid application, with the operational data tier in Fabric and the operational application tier in Azure, and the operational data will be a source ingested into the Fabric data warehouse; 6) Power BI Writeback Function, where you create a user data function (in preview) inside of Fabric that updates data in a SQL Database. Then call that function from a Power BI report (via a Power BI button that calls that function). Make sure to check out the limitations in SQL database in Microsoft Fabric and Features comparison: Azure SQL Database and SQL database in Microsoft Fabric (preview). SQL database in Fabric will be free until January 1, 2025, after which compute and data storage charges will begin, with backup billing starting on February 1, 2025. To learn more, read the Fabric Databases blog post, watch the Microsoft Mechanics deep dive video or sizzle video, watch these Ignite sessions: Fuel AI innovation with Azure Databases, Use AI with the latest Azure SQL innovations to transform your data, Power AI apps with insights from SQL database in Fabric, and check out the Learning pathways for SQL database in Microsoft Fabric. To compare Fabric data stores such as warehouse, lakehouse, Eventhouse, SQL database, and Power BI datamart, check out Microsoft Fabric decision guide: choose a data store.
- OneLake catalog, which is an evolution of the OneLake data hub. It is a complete solution to explore, manage, and govern your entire Fabric data estate. The OneLake catalog comes with two tabs, Explore and Govern, that can help all Fabric users discover and manage trusted data, as well as provide governance for data owners with valuable insights, recommended actions, and tooling. The Explore tab is now generally available, and the Govern tab will be coming soon in preview. Learn more about the OneLake catalog by reading this blog post and by watching the demo, or watch this Ignite session: Ingest, govern, and secure your data with OneLake.
- The preview of open mirroring, a feature that allows any application or data provider to write change data directly into a mirrored database within Fabric. Microsoft’s Open Mirroring partner ecosystem continues to grow with publicly available solutions from Striim, Oracle Golden Gate, and MongoDB, with DataStax’s solution coming soon. Watch this Ignite session: Ingest, govern, and secure your data with OneLake.
- The public preview of SQL MI mirroring, seamlessly synchronizing data from operational databases within Azure SQL Managed Instance into Microsoft Fabric’s OneLake.
- The preview of the Copilot in Fabric experience for data pipelines in Fabric Data Factory. These features function as an AI expert to help users build, troubleshoot, and maintain data pipelines.
- Coming soon, the preview of AI skill enhancements, including a more conversational experience and support for semantic models and Eventhouse KQL databases.
- Coming soon, the preview of AI skill integration with Agent Service in the newly announced Azure AI Foundry, allowing developers to use AI skills as a core knowledge source.
- The preview of workspace monitoring, which provides detailed diagnostic logs for workspaces to troubleshoot performance issues, capacity performance, and data downtime.
- The preview of further integration with Microsoft Purview including extending Protection policies to enforce access permissions to more sources and using Data Loss Prevention policies to restrict access to semantic models with sensitive data.
- Tenant switcher control – The tenant switcher is now available in the Fabric portal. Users with access to more than one Fabric tenant can easily switch between tenants directly from the account manager in the top right corner of the Fabric portal. This is in addition to the existing From External Orgs tab that can be found in the home page of the Power BI experience.
- Microsoft Fabric SKU Estimator, in private preview. This is an evolution of the Microsoft Fabric Capacity Calculator, designed to help customers and partners accurately estimate their capacity requirements and identify the appropriate SKU for their needs.
- Upcoming Changes to the Fabric Navigation Experience, designed to enhance your navigation experience with Microsoft Fabric. These updates aim to simplify your workflow and make navigation more intuitive. It removes the granular persona/workload-based experience we currently have, in favor of a simplified two-experience model: Fabric, or Power BI. The Fabric model is a new workspace-centric navigation and task-oriented item creation flow allows you to focus on your projects without the distraction of selecting a specific workload. The Power BI model is designed for users focused on exploring insights in reports, apps, and semantic models. It features an item-first approach, providing direct access to items using Power BI tools. You will also see a new option on the navigation bar called Workloads, which is now your go-to hub for discovering available workloads, along with comprehensive getting started guides and tutorials. It’s the place where you can learn how to leverage these workloads to maximize their impact on your projects. Whether you’re exploring new features or looking to deepen your expertise, you’ll find all the resources you need to get up to speed and drive better results.
- Reflex (under Real-Time Intelligence) has been renamed to Activator.
- Fabric is now FedRAMP High certified for the Azure Commercial cloud, the highest level of compliance and security standards required by the federal government for cloud service providers. Now government agencies can run Fabric on the Azure Commercial cloud while maintaining strict compliance (see: Services added in the last 90 days).
Non-Fabric related:
- Microsoft Purview Data Catalog is being renamed to Microsoft Purview Unified Catalog to better reflect the offering’s comprehensive customer benefits.
- New Purview features: integration with new OneLake catalog; a new data quality scan engine; Purview Analytics in OneLake; and expanded Data Loss Prevention (DLP) capabilities for Fabric lakehouse and semantic models.
- A new product was announced called Azure AI Foundry, but it is more of a grouping and rebranding of existing products and services – these services include the Azure AI Foundry portal, formerly known as Azure AI Studio, the Azure AI Foundry software development kit (SDK), Azure AI Agents, pre-built application templates (25 pre-built application templates at launch) and a suite of tools designed for AI-based application development. This service integrates with existing Azure AI tools, including Azure AI Search, AI Agents, AI Content Safety and Azure Machine Learning.
- SQL Server 2025, now in private preview. See Bob Ward’s Announcing SQL Server 2025 and the Ignite session SQL Server 2025: an enterprise AI-ready database platform.
More info:
Ignite video Microsoft Fabric: What’s new and what’s next with Arun Ulagaratchagan and Amir Netz
Spreading your SQL Server wings with SQL database in Fabric
SQL database in Fabric by Marco Russo
Great list and summary as always James. You might add a bullet point for Surge protection in Fabric as well, many of my customers are interested in that.