Analytics Platform System (APS) AU2 released
The Analytics Platform System (APS), which is a renaming of the Parallel Data Warehouse (PDW), has recently released an appliance update (AU2), which is sort of like a service pack, except that it includes many new features. These appliance updates are made available frequently, about every 3-4 months. Below is what is new in this release:
TSQL Compatibility improvements to reduce migration friction from SQL SMP
- Adds full support for user defined schemas. This is one of the most requested features from customers migrating from SQL Server (CREATE SCHEMA, ALTER USER, GRANT/DENY, etc.)
- Improves statistics collection capability and supportability by introducing filtered statistics, ability to control the sample size for sampled statistics, and support for DBCC SHOW_STATISTICS and STATS_DATE intrinsics
- Adds support for a set of 51 mathematical, string ODBC, date/time, data and type analytic built in functions:
ACOS, ATN2, COT, DEGREES, PI, RADIANS, ASCII, CHAR, CONCAT, DIFFERENCE, NCHAR, REVERSE, SPACE, STUFF, UNICODE, TERTIARY_WEIGHTS, PARSENAME, BIT_LENGTH, CURRENT_DATE, CURRENT_TIME, DAYNAME, DAYOFMONTH, DAYOFWEEK, HOUR, MINUTE, SECOND, MONTHNAME, QUARTER, WEEK, DATEFROMPARTS, DATETIME2FROMPARTS, DATETIMEFROMPARTS, DATETIMEOFFSETFROMPARTS, SMALLDATETIMEFROMPARTS, TIMEFROMPARTS, SYSDATETIME, SYSDATETIMEOFFSET, SYSUTCDATETIME, ISDATE, DATENAME, GETUTCDATE, EOMONTH, SWITCHOFFSET, TODATETIMEOFFSET, HASHBYTES, DATALENGTH, TYPE_ID, TYPE_NAME, TYPEPROPERTY, SQL_VARIANT_PROPERTY - Enables broader set of 3rd party ISV tools such as SAS and Tableau, as well as unblocking scenarios around managing data and analytical functions within the appliance
HDI Region Support for HDP 2.0 and Yarn
- Enables users to write custom apps based on new data processing models (beyond MapReduce), enabling new scenarios such as interactive/ad hoc querying or search
- Improves processing on the same hardware with predictable performance and better quality of service
- On boards SCOM management pack for HDI, enabling System Center management of PDW+HDI
- Updates the Microsoft .NET SDK for Hadoop to add value for both APS and Azure HDInsight clusters
Polybase HDP 2.0 and Query Hint Support
- Enables all Polybase scenarios to work against both HDP 2.0 on Windows and Linux, pushing computation through the Yarn framework
- Enables query hint support for Polybase push-down, enabling ability to force or disable push-down on a per-query basis
Install, Upgrade, and Servicing Improvements
- Reduces end-to-end install time from 12 to under 9 hours (single scale unit)
- Reduces end-to-end install time for virtual appliances from 7.5 to 4.5 hours
- Reduces end-to-end servicing and patching time from 2 to 1 hour
- Improves supportability with fast-fail and simplified troubleshooting steps
- Improves performance by performing parallel and group patching.
Appliance Heartbeat Monitoring
- Collects critical alerts and appliance heartbeats, storing appliance health data in Azure
Self-host Appliance and Customer Workload
- On boards key first party workloads by creating APS-based internal data warehouse
Engineering Efficiency Improvements
- Introduces train model, reducing release cycle to 120 days end-to-end
- Enables more frequent test passes to reduce bug escape
- Completely automates virtual appliance allocation
TFS and MSBuild Migration
- Migrates codebase to Team Foundation Server (TFS) under PDW_Main
- Converts codebase to MSBuild
Thank you James for this insightful post. There truly are a huge number of new features that will add lots of value for us. The data types \ functions now supported are one of my favorite additions. I trust my client will be happy about Tableau support.