re:Invent 2022 – Data and AI

In nutshell: The world is asynchronous. A big focus on asynchronous and event-driven architectures. A lot of exciting announcements around serverless and software development.

To make it easier to deepen a knowledge, I have categorised the videos I am interested in according to:

  1. Innovation
  2. Infrastructure
  3. Data and AI
  4. Security
  5. Costs
  6. Use Cases

Today I am sharing the Data and AI part.

All below comes from a public AWS channel, I only share selected and classified by me, the texts come from AWS youtube channel, links in the description.

  • Building modern data architectures on AWS

The modern data architecture is an evolution from data warehouse and data lake–based solutions, and it allows you to query data across your data warehouse, data lake, and purpose-built analytics services to gain faster and deeper insights that would not be possible otherwise. In this session, learn how to design, create, and operate a modern data architecture on AWS by using AWS purpose-built data services. Also learn about modern data architecture concepts, why customers are adapting to it, modern data architecture pillars, reference architectures, and best practices while building modern data architectures on AWS for optimal performance and cost effectiveness.

The above description and video can be found here: https://www.youtube.com/watch?v=vtXKSTNHCXQ

  • Achieving your modern data architecture

A modern data strategy is a comprehensive plan or blueprint to help you manage, access, analyze, and act on your data. Many organizations are developing roadmaps to help them achieve digital transformation. This session helps you understand how deploying a modern data architecture can help you navigate your data challenges, optimize analytics processes, and deliver faster insights to the people and applications that need it.

The above description and video can be found here: https://www.youtube.com/watch?v=DNzSsiAfWpU

  • Publishing real-time financial data feeds using Kafka

This session describes how to offer a real-time financial data feed as a service on AWS. By using Amazon Managed Streaming for Apache Kafka (Amazon MSK), you can use Kafka to allow your customers to subscribe to message topics containing the financial data of interest. Consider connectivity best practices to distribute data at a global scale, security options to ensure a secure architecture, and lessons learned from customers that are using AWS to distribute financial data on AWS.

The above description and video can be found here: https://www.youtube.com/watch?v=-hR3gTXoE_Y

  • Why operationalizing data mesh is critical for operating in the cloud

As companies look to scale in the cloud, they face new and unique challenges related to data management. Data mesh offers a framework and a set of principles that companies can adopt to help them scale a well-managed cloud data ecosystem. In this session, learn how Capital One approached scaling its data ecosystem by federating data governance responsibility to data product owners within their lines of business. Also hear how companies can operate more efficiently by combining centralized tooling and policy with federated data management responsibility. This presentation is brought to you by Capital One, an AWS Partner.

The above description and video can be found here: https://www.youtube.com/watch?v=DyB2iueJa6I

  • Next-generation databases for next-generation workloads

With IoT, Web3, and social and mobile technology becoming mainstream, today’s highly distributed digital processes and interactions—and the data they create—require a new approach to data pipelines, storage, processing, and analytics. In this talk, join Brian Gilmore, Director of IoT and Emerging Technology at InfluxData, for a discussion of these emerging challenges, how organizations struggle with them today, and how new innovations within the InfluxDB Platform are used in the cloud, at the edge, and beyond to solve some of the world’s most complex technical challenges. This presentation is brought to you by InfluxData, an AWS Partner.

The above description and video can be found here: https://www.youtube.com/watch?v=ApR8uq-UQW8

  • Deploy ML models for inference at high performance & low cost, ft AT&T

High-performance, cost-effective model deployment is critical to maximize the return on your ML investments. Amazon SageMaker provides the breadth and depth of fully managed deployment features to achieve optimal inference performance and cost, while reducing operational burden. In this session, learn how to use SageMaker inference capabilities to quickly deploy ML models in production at scale. Discover SageMaker deployment options including: infrastructure choices; real-time, serverless, asynchronous, and batch inference; single-model, multi-model, and multi-container endpoints; auto scaling; SageMaker Inference Recommender; model monitoring; and SageMaker MLOps integration. Learn how AT&T used Amazon SageMaker to optimize ML model deployment at scale.

The above description and video can be found here: https://www.youtube.com/watch?v=4FqHt5bmS2o

  • The power of responsibility in AI/ML

As the world increasingly relies on artificial intelligence (AI) to inform decision-making, how can you advocate for your customers and maintain responsibility for your technology? Join this session with Aileen Smith, Head of Content Strategy, Diversity Marketing at AWS, and Diya Wynn, Senior Practice Manager, AI/ML Ethics at AWS, to explore a strategy for the practice of responsible AI. Learn how to integrate a human-centered, culturally aware approach into product development workstreams and build more trust with customers. Learn how to use big data responsibly, develop better AI systems, and build products and solutions to expand your customer base and serve people better.

The above description and video can be found here: https://www.youtube.com/watch?v=84kvSuTGY7I

  • Deep learning on AWS with NVIDIA: From training to deployment

Over the past decade, NVIDIA has been able to illustrate the effectiveness of its GPUs across the board for both deep learning training and inference. As these models become larger, the inherent need to scale up for training and scale out for deploying such large models has become a necessity. In this session, walk through a few NVIDIA software stacks for efficient distributed training as well as streamlined deployment. Dive deep into how Amazon adopts them for some of their most demanding workloads. Also, consider the use of these SDKs on AWS. This presentation is brought to you by NVIDIA, an AWS Partner.

The above description and video can be found here: https://www.youtube.com/watch?v=l8AFfaCkp0E

  • Unlock the value of your data with AWS analytics

Data fuels digital transformation and drives effective business decisions. To survive in an ever-changing world, organizations are turning to data to derive insights, create new experiences, and reinvent themselves so they can remain relevant today and in the future. AWS offers analytics services that allow organizations to gain faster and deeper insights from all their data. In this session, G2 Krishnamoorthy, VP of AWS Analytics, addresses the current state of analytics on AWS, covers the latest service innovations around data, and highlights customer successes with AWS analytics. Also, learn from organizations like FINRA and more who have turned to AWS for their digital transformation journey.

The above description and video can be found here: https://www.youtube.com/watch?v=A6DG_Afe07I

  • Democratizing your organization’s data analytics experience

AWS analytics services empower data users, such as data scientists, analysts, and business users with diverse technical expertise, across an organization to quickly access, analyze, and gain insights from their data. In this session, learn how AWS can democratize analytics for your organization with ease of use and better price performance. Dive into how serverless AWS simplifies data preparation and makes it easier for analysts and data scientists of all skill levels to use machine learning.

The above description and video can be found here: https://www.youtube.com/watch?v=sTz0aWi23PI

  • Modernize your database with Confluent and Amazon RDS

On-premises databases can be slower and more rigid, costing businesses money in both upfront and ongoing maintenance costs. That limits the speed at which businesses can scale, experiment, and drive innovation. Today’s organizations need scalable, cloud-native databases for improved agility, elasticity, and cost-efficiency. In this session, walk through how you can start building persistent pipelines for continuous migration from legacy to cloud database, connecting across multi-cloud and hybrid environments and moving from batch to real-time data. Explore how to use Confluent and Amazon RDS to create a bridge across your cloud and on-premises environments. This presentation is brought to you by Confluent, an AWS Partner.

The above description and video can be found here: https://www.youtube.com/watch?v=L4eZrvdi3ao

  • Trust your data with data integrity on AWS

According to a recent IDC report, only 27 percent of data practitioners trust the data they work with. That’s not good enough for your DataOps initiatives and the modern data stack you’re building on AWS. To build trust, you must break data out of siloes, understand its lineage and business impact, cultivate continued quality, and enrich it with context and spatial insights. You need data integrity—data that is accurate, consistent, and contextualized. In this talk, learn how the Precisely Data Integrity Suite can help you build trust in your AWS data for innovation and digital transformation. This presentation is brought to you by Precisely, an AWS Partner.

The above description and video can be found here: https://www.youtube.com/watch?v=QTDl9-H80SM

  • Build and automate for resilience with modern data protection

At any given moment ransomware incidents, catastrophic weather events, or even basic human error could result in significant downtime and substantial losses incurred. Commvault intelligent automation and API integration combine with scalable, elastic AWS resources to help your organization recover from whatever comes your way. In this talk, guests from Illinois State University discuss how Commvault and AWS have worked with them to help modernize how they protect and manage their data by simplifying operations, reducing cloud costs, and allowing them to focus on what they do best: conduct groundbreaking research and educate future generations. This presentation is brought to you by Commvault, an AWS Partner.

The above description and video can be found here: https://www.youtube.com/watch?v=OkaGvr3xYNk

  • Make ML outcomes roar with graph neural networks on TigerGraph and AWS

Your data is full of relationships—customers who are married, parts coming from the same supplier, or individual transactions that collectively are a money laundering scheme. If you don’t use these relationships in your machine learning, you may be throwing half your information away. In this lightning talk, learn about TigerGraph ML Workbench, which integrates directly with Amazon SageMaker to build ML models that are a quantum leap better in precision and recall. Customers can boost the accuracy of established machine learning models by 20%+, returning tens of millions of dollars a year. Learn about graph analytics and how you can work toward the same results for your own business. This presentation is brought to you by TigerGraph, an AWS Partner.

The above description and video can be found here: https://www.youtube.com/watch?v=QBKEeiMePDw

  • Modernize apps with purpose-built databases, featuring Liberty Mutual

Modernizing your applications is more than moving them to the cloud—that is simply the first step. By moving your applications to the cloud, you gain access to the latest database technologies, allowing you to modernize your applications to better serve your customers, stakeholders, or constituents. In this session, learn how to use open-source relational databases and new technologies such as key-value, document, and graphs to enhance performance of your applications. Learn why Liberty Mutual adopted AWS services and how they will reduce future total cost of operations while positively impacting the environment by shifting to AWS’s more energy-efficient data centers.

The above description and video can be found here: https://www.youtube.com/watch?v=V-DiplATdi0

  • Transform your business with time-series data

The world is changing at warp speed, thanks to our ability to capture and analyze more data than ever. Time-series data is at the heart of this transformation, driving innovation in everything from IoT-powered transportation networks and autonomous financial trading algorithms to SaaS application analytics. Businesses, governments, schools, and communities large and small are finding ways to use feature-rich time-series data to better deliver value to those they serve. In this talk, learn how to use time-series data to identify and measure change in your business and how Timescale Cloud can power and scale your time-series applications on AWS. This presentation is brought to you by Timescale, an AWS Partner.

The above description and video can be found here: https://www.youtube.com/watch?v=Z1U-eH3XNss

  • Best practices for managing S3 data at scale, w/Bridgewater Associates

As you grow to store millions to billions of objects and terabytes to petabytes of data on Amazon S3, tools and automation to handle that scale become increasingly valuable. In this session, dive deep on Amazon S3 Lifecycle management, S3 Batch Operations, S3 Inventory reports, S3 Replication, and other features that can help you better understand, optimize, and perform actions across storage at scale. Hear from Bridgewater Associates about how they use native Amazon S3 capabilities to facilitate global macroeconomic systemic research.

The above description and video can be found here: https://www.youtube.com/watch?v=6zT-nHRDRy0

  • Utilizing sustainability data at scale

Valuable data can be generated by massive satellites or the smallest of sensors, and organizations need to manage rapidly increasing amounts of data. Learn how to extract and handle data at scale through use cases focused on the sustainability of something that affects everyone—the food we eat. Explore data analytics pipelines that use imagery to estimate crop yield, assess plant health, and provide insights to reduce water use. Learn how satellite and sensor data is stored in data lakes and leveraged using AWS compute, analytics, and machine learning services, while exploring how technology is driving an agriculture revolution.

The above description and video can be found here: https://www.youtube.com/watch?v=S2XdVirGnVc

  • Zero-privilege operations: Running services without access to data

AWS works with organizations and regulators to host some of the most sensitive workloads in industry and government. In this session, learn how AWS secures data, even from trusted AWS operators and services. Explore the AWS Nitro System and how it provides confidential computing and a trusted runtime environment, and dive deep into the cryptographic chains of custody that are built into AWS Identity and Access Management (IAM). Finally, hear how encryption is used to provide defense in depth and why we focus on verified isolation and customer transparency at AWS.

The above description and video can be found here: https://www.youtube.com/watch?v=kNbNWxVQP4w

  • Get clarity on your data in seconds with Amazon QuickSight Q

Amazon QuickSight Q is a machine learning–powered natural language capability that empowers business users to ask questions about all of their data using everyday business language and get answers in seconds. Q interprets questions to understand their intent and generates an answer instantly in the form of a visual without requiring authors to create graphics, dashboards, or analyses. In this session, the Amazon QuickSight Q team provides an overview and demonstration of Q in action.

The above description and video can be found here: https://www.youtube.com/watch?v=BkjUSd8JK9c

  • How Riot Games processes 20 TB of analytics data daily on AWS

Riot Games ingests about 20 TB of data every day on AWS. This data powers a wide range of services including game matchmaking, in-game personalization, analytics, security, and player behavior management. Until recently, this data was only queryable for up to 6 hours after it was produced. With Amazon MSK, the team reduced time down to 5 minutes. Amazon MSK also allowed Riot Games to lower TCO and deprecate an aging map-reduce-based pipeline. In this session, Riot Games describes the before state, migration path, current state and future state of their data ingestion pipeline.

The above description and video can be found here: https://www.youtube.com/watch?v=5L1K_moG-dY

  • Scaling data processing w/Amazon EMR at the speed of market volatility

FINRA Consolidated Audit Trail (CAT) is the single largest repository for US financial market events and processes billions of events from broker-dealers and exchanges using Apache Spark on Amazon EMR. Processing over 250 billion events from broker-dealers and over 1.7 trillion events from exchanges within a mandated SLA of 4 hours presents unique scaling challenges. Moreover, data volumes change each day based on market activity. In this session, the team at FINRA talks about how they improved SLA and reduced the cost of running the system using the latest from Amazon EMR and Amazon EC2.

The above description and video can be found here: https://www.youtube.com/watch?v=g2r2-q6DpyM

  • Using Amazon AppFlow to break down data silos for analytics and ML

Data silos can create barriers to efficiency and data quality and, as a result, often hinder an organization’s ability to begin their data and ML journeys in the cloud. Manual data ingestion, data preparation, and application integration are time-consuming and expensive. In this session, learn how organizations can efficiently break down data silos with low-code and no-code integration services, such as Amazon AppFlow, and speed up innovation with the help of AWS analytics, AI/ML, and integration services. Also get prescriptive guidance on how to select the best tool for your integration strategy.

The above description and video can be found here: https://www.youtube.com/watch?v=NF7rEo8CzpA

  • Build your data lakehouse with Starburst Galaxy

Organizations across the world strive to reveal business insight from their data. However, many struggle to truly capitalize on the valuable information stored within their data lakes. In this session, learn about the Starburst lakehouse and how Starburst’s scalable query engine provides easy and secure access to your data lake, resulting in reduced data management costs and faster time to insights for critical business decisions. Join this session to learn how to use Starburst Galaxy to implement a data lakehouse architecture and operationalize your data lake in only a few minutes. This presentation is brought to you by Starburst Data, an AWS Partner.

The above description and video can be found here: https://www.youtube.com/watch?v=RwubsWC2wEY

  • Reinvent how you derive value from your data with Amazon QuickSight

In this session, learn how you can use AWS native business analytics to provide your users with machine learning–powered interactive dashboards, natural language query (NLQ), and embedded analytics to provide insights to users at scale, when and where they need it.

The above description and video can be found here: https://www.youtube.com/watch?v=l_Go9fMtYdc

  • Scale analytics faster with Tableau Cloud

Accelerate your modern cloud analytics with security and manageability at scale. In this talk, learn how global organizations drive mission-critical analytics throughout their enterprise using the power of Tableau Cloud hosted on AWS. This presentation is brought to you by Tableau, an AWS Partner.

The above description and video can be found here: https://www.youtube.com/watch?v=AUm4x3U-mg0

  • Boost user engagement with Amazon Personalize

Many consumers today expect a tailored offering as they consider, purchase, and use products and services. Every digital touchpoint is an opportunity for businesses to deliver on that expectation. In this session, learn how Chewy, an AWS customer, used Amazon Personalize to improve user engagement by providing a dynamic, high-quality shopping experience.

The above description and video can be found here: https://www.youtube.com/watch?v=y7crGW6zLao

  • Modern data platforms: AWS and MariaDB SkySQL

Outmoded data platforms impede innovation. A modern cloud data platform turns your business into a data-centric force—leveraging data for business growth and innovation. A modern database service scales to handle extraordinary transactional volume, variety, and velocity and provide real-time operational analytics—without sacrificing performance, availability, security, or consistency. In this lightning talk, explore how your data can power unparalleled customer experiences, unleash operational efficiencies, automate administrative tasks, enable smart business decisions, and drive revenue—making your business virtually unstoppable. Learn why data-centric businesses such as FanGraphs, Virgin Media O2, and Financial Network, Inc., rely on MariaDB SkySQL fully managed cloud databases. This presentation is brought to you by MariaDB, an AWS Partner.

The above description and video can be found here: https://www.youtube.com/watch?v=DfYuf-MVa9w

  • Build scalable multi-tenant databases with Amazon Aurora

In this session, dive deep into Amazon Aurora capabilities, including Amazon Aurora Serverless v2, read replicas, Aurora global databases, and Amazon RDS Proxy, and learn how it can help you build cost-efficient and scalable multi-tenant databases. Organizations often struggle to strike the balance between operational complexity and cost. Provisioning a separate database for each workload provides the best level of isolation and most granular operational controls, but it may not be the most cost-effective or operationally manageable. Discover how Amazon Aurora helped VMware Aria Cost powered by CloudHealth (formerly CloudHealth) consolidate 166 self-managed MySQL databases on 62 Aurora clusters and achieve operational and cost efficiency along the way.

The above description and video can be found here: https://www.youtube.com/watch?v=tWRhDxcMLYo