James works at Microsoft as a big data and data warehousing solution architect where he has been for most of the last ten years. Before that he was an independent consultant working as a Data Warehouse/Business Intelligence architect and developer. He is a prior SQL Server MVP with over 35 years of IT experience. Check out his book Deciphering Data Architectures: Choosing Between a Modern Data Warehouse, Data Fabric, Data Lakehouse, and Data Mesh.
(updated 10/28/20). Life we be so much easier if we could just trust everyone, but since we can’t we need solid security for our databases. Azure SQL Database and Azure Synapse Analytics have many security features to make you sleep … Continue reading →
Even though an Azure SQL Database stores all data on the Azure cloud, it does not mean that your options for managing and monitoring the databases are limited compared to on-prem databases. In fact, the options available are very similar to on-prem, … Continue reading →
One of the advantages Azure SQL Database has over on-prem SQL Server is the ease in which it can scale. I’ll discuss the various options for horizontal scaling, vertical scaling, and other similar features. Horizontal scaling refers to adding or removing … Continue reading →
Microsoft made available the first technical preview of its new Microsoft Azure Stack offering today. It was announced last week. Azure Stack brings the cloud model of computing to every datacenter – your own private cloud. Azure Stack is a new hybrid … Continue reading →
This blog describes the various approaches you can use to migrate an on-premises SQL Server database to Azure SQL Database. In this migration process you migrate both your schema and your data from the SQL Server database in your current … Continue reading →
Azure SQL Database is a relational database-as-a-service in the cloud. It uses a special version of Microsoft SQL Server as its backend that is nearly identical to SQL Server (see Azure SQL Database Transact-SQL differences). While there are many benefits … Continue reading →
Previously I covered what a data lake is (including the Azure Data Lake and enhancements), and now I wanted to touch on the main reason why you might want to incorporate a data lake into your overall data warehouse solution. To … Continue reading →
Microsoft Azure is a cloud computing platform and infrastructure, created by Microsoft, for building, deploying and managing applications and services through a global network of Microsoft-managed and Microsoft partner-hosted datacenters. Included in this platform are multiple ways of storing data. Below … Continue reading →
So you have data in Azure Blob Storage and are concerned about reliability. Have no fear! There are four replication options for redundancy: 1. Locally Redundant Storage (LRS): All data in the storage account is made highly durable and available within … Continue reading →
In a previous blog I talked about copying on-prem data to Azure Blob Storage (Getting data into Azure Blob Storage). Let’s say you have copied the data and it is sitting in Azure Blob Storage (or an Azure Data Lake) … Continue reading →
I first blogged about Microsoft’s new product, the Azure Data Lake, a few months back (here). There are already enhancements, as announced at Stata + Hadoop World. Here they are in brief: The Azure Data Lake has been renamed to the … Continue reading →
If you have on-prem data and want to copy it to Azure Blob Storage in the cloud, what are all the possible ways to do it? There are many, and here is a quick review of them: AzCopy: A popular command-line … Continue reading →
I see a lot of confusion about the place and purpose of the many new database solutions (“NoSQL databases”) compared to the relational databases solutions that have been around for many years. So let me try to explain the differences … Continue reading →
In my Introduction to Hadoop I talked about the basics of Hadoop. In this post, I wanted to cover some of the more common Hadoop technologies and tools and show how they work together, in addition to showing how they work … Continue reading →
The Analytics Platform System (APS), which is a renaming of the Parallel Data Warehouse (PDW), has just released an appliance update (AU4), which is sort of like a service pack, except that it includes many new features. Below is what is new in this … Continue reading →
Yesterday at the Microsoft World Wide Partner Conference in Orlando Microsoft announced the Cortana Analytics Suite, which is a new package of data storage, information management, machine learning, and business intelligence software in a single convenient monthly subscription. Microsoft’s Cortana personal … Continue reading →
Just announced is the Microsoft Azure Data Catalog, which is an enterprise metadata catalog / portal for the self-service discovery of data sources. It becomes available on Monday next week, July 13, 2015. Check out this short video on it. … Continue reading →
Polyglot Persistence is a fancy term to mean that when storing data, it is best to use multiple data storage technologies, chosen based upon the way data is being used by individual applications or components of a single application. Different kinds … Continue reading →
Microsoft Azure Stream Analytics (ASA) is a fully managed cloud service for real-time processing of streaming data. ASA makes it easy to set up real-time analytic computations on data flowing in from devices, sensors, web sites, applications and infrastructure systems. It … Continue reading →
Massive parallel processing (MPP) is the future for data warehousing. So what is MPP? SQL Server is a Symmetric Multiprocessing (SMP) solution, which essentially means it uses one server. MPP provides scalability and query performance by running independent servers in parallel. That … Continue reading →