Microsoft Fabric shortcuts
I talked about Microsoft Fabric shortcuts in my blog post Microsoft Fabric – the great unifier (where I have updated the picture with the newest supported sources) and wanted to provide more details on how shortcuts work and reduce some confusion.
This is how shortcuts appear in Explorer:
Here are important key points in understanding shortcuts:
- You can create shortcuts in Fabric lakehouses and Kusto Query Language (KQL) databases, not in Fabric warehouses
- You can use the Fabric UI to create shortcuts interactively, and you can use the REST API to create shortcuts programmatically
- In the Tables section, you can only create shortcuts at the top level. Shortcuts aren’t supported in other subdirectories of the Tables section. If the target of the shortcut contains data in Delta format, the lakehouse automatically synchronizes the metadata and recognizes the folder as a table. Shortcuts in the tables section can be to anywhere and any format (technically) but only the Delta tables are seen by the SQL engine* so it’s useless to create shortcuts to any other data if you want to use them as tables. So, for non-Delta formatted files to be useful, those files should first be converted to Delta format
- In the Files section, there are no restrictions on where you can create shortcuts. You can create them at any level of the Files section folder hierarchy. Table discovery doesn’t happen in the Files section. Shortcuts in the files section can be to any supported source in any format but those will never be visible from the SQL engine, regardless of the data type. So if you create a shortcut to a file that is not in Delta format and want to use it in the Tables section, you would need to copy the data into OneLake and convert it to Delta format. If the file you want to shortcut to is already in Delta format, you should create the shortcut in the Tables section instead of the Files section
- Any Fabric or non-Fabric service that can access data in OneLake can use the shortcuts. Shortcuts just appear as another folder in the lake
- In the Tables section, you can create shortcuts to folders, but folders will show up in an Unidentified folder and not be usable by the SQL engine if the folder does not contain a Delta log. You can also create shortcuts to individual warehouse/lakehouse tables, tables in KQL databases and tables in mirrored databases (which all store data in Delta format)
- In the Files section, you can only create shortcuts to folders, not to individual files. These shortcuts can be very useful, as an example, for situations where you want to use shortcuts to data to train a ML model instead of having to copy the data
- Shortcut supported sources don’t include relational storage – for those use Fabric mirroring. Shortcuts only work against data that is in an open file format and open-source system, so use mirroring for closed systems such as Azure SQL Database
* = The SQL engine being the new Fabric unified SQL engine that can query tables in the lakehouse as well as the warehouse through the SQL Analytics Endpoint
Pingback:Shortcuts in Microsoft Fabric – Curated SQL