This can be a collaborative publish from Databricks and Microsoft Azure. We thank Rajeev Jain, Senior Product Advertising Supervisor at Microsoft, for his contributions.
Knowledge + AI Summit 2023: Register now to hitch this in-person and digital occasion June 26-29 and be taught from the worldwide information group.
Microsoft is a Platinum Sponsor of Knowledge + AI Summit 2023, the premier occasion for the worldwide information group. Satya Nadella, Chairman and CEO of Microsoft, shall be talking on the Knowledge + AI Summit Keynote Day 1. Be a part of Knowledge + AI Summit to be taught from joint Databricks and Microsoft Azure prospects like Akamai, Waterstons, Windfall Well being Companies, ABN Amro, Bradesco, and Division of Veterans Affairs who’ve efficiently leveraged the Databricks Lakehouse Platform for his or her enterprise, bringing collectively information, AI and analytics on one frequent platform.
At Knowledge + AI Summit, Databricks and Microsoft prospects will take the stage for classes that can assist you see how they achieved enterprise outcomes utilizing the Databricks Lakehouse on Azure. Attendees may have the chance to listen to information leaders from Akamai and Waterstons on Tuesday, June twenty seventh, then be a part of Windfall Well being Companies, Bradesco, and ABN Amro on Wednesday, June twenty eighth and Division of Veterans Affairs on Thursday, June twenty ninth. At Knowledge + AI Summit, be taught concerning the newest improvements and applied sciences and listen to thought-provoking panel discussions together with the flexibility for networking alternatives the place you’ll be able to join with different information professionals in your business.
The classes under are a information for everybody inquisitive about Azure Databricks and span a variety of matters — from real-time medical operations monitoring to shopper analytics to IoT information gathering. When you’ve got questions on Azure Databricks or service integrations, join with Azure Databricks Options Architects at Knowledge + AI Summit on the Microsoft sales space on the Expo ground.
Keynote
Satya Nadella Keynote
Wednesday, June 28 @8:15 AM
Visionary leaders will share insights on:
Knowledge, analytics and AI panorama: Uncover what’s driving a lot concentrate on information and why information professionals are zeroing in on new methods to sort out their database challenges. Study why there may be a lot curiosity in LLMs, what is occurring throughout the information, analytics and AI panorama and the way forward for the market
Evolution of the lakehouse: Check out the bigger universe that the lakehouse lives within, be taught what’s new and discover the evolution with us
Open supply applied sciences: Hear from the open supply group about what’s new and what’s to come back for Apache Spark™, Delta Lake and MLflow and find out how this impacts the lakehouse and the general market at massive
Microsoft buyer breakout classes
Taking Your Cloud Vendor to the Subsequent Degree: Fixing Advanced Challenges with Azure Databricks
Tuesday, June 27 @1:00 PM
Akamai’s content material supply community (CDN) processes about 30% of the web’s each day visitors, leading to a large quantity of information that presents engineering challenges, each internally and with cloud distributors. On this session, we’ll talk about the limitations confronted whereas constructing an information infrastructure on Azure, Databricks, and Kafka to fulfill strict SLAs, hitting the boundaries of a few of our cloud distributors’ providers. We are going to describe the iterative means of re-architecting a large scale information platform utilizing the aforementioned applied sciences.
We will even delve into how right now, Akamai is ready to rapidly ingest and make obtainable to prospects terabytes of information, in addition to effectively question Petabytes of information and return outcomes inside 10 seconds for many queries. This dialogue will present beneficial insights for attendees and organizations looking for to successfully course of and analyze massive quantities of information.
Getting Perception From Something: Gathering Knowledge With IoT Units and Delta Reside
Tuesday, June 27 @4:00 PM
The drive to make data-driven selections shapes the technique of many firms. Nevertheless, this dream has an issue; simply because it’s worthwhile to decide doesn’t imply you have got the information to make it. Typically these selections require information from sources sometimes unconnected to the web. This may very well be an architect decided to know how the development of her constructing is progressing, or a producer desirous to dwell their greatest business 4.0 life. It is not unusual for firms to seek out this journey to grow to be “good” is an extended one; with tough moral floor to cowl, or prohibitive prices related to procuring the mandatory gear. Nevertheless, the event of low-cost and accessible IoT gadgets permits for the gathering of information from something to be extra inside attain than ever.
This session will have a look at efforts to construct proof-of-concept IoT methods. We are going to talk about our (persevering with) journey of utilizing and managing {hardware} and the issues we confronted alongside the best way. The main focus would be the software program stack. Azure IoT Hub is used for system administration, with Energy BI supporting the entrance finish. Nevertheless, the centerpiece is Delta Reside Tables – permitting us a technique of close to real-time evaluation of our information. We are going to talk about some great benefits of utilizing Delta Reside for this, and the way we want to lengthen our IoT methods sooner or later.
Bettering Hospital Operations with Streaming Knowledge and Actual Time AI/ML
Wednesday, June 28 @11:30 AM
Over the previous two years, Windfall has developed a strong streaming information platform (SDP) leveraging Databricks in Azure. The SDP allows us to ingest and course of real-time information reflecting medical operations throughout our 52 hospitals and roughly 1000 ambulatory clinics. The HL7 messages generated by Epic are parsed utilizing Databricks in our safe cloud surroundings and used to generate an up-to-the minute image of precisely what is occurring on the level of care.
We’re already leveraging this info to attenuate hospital overcrowding and have been actively integrating AI/ML to precisely forecast future circumstances (e.g., arrivals, size of keep, acuity, and discharge necessities.) This enables us to each enhance useful resource utilization (e.g., nurse staffing ranges) and to optimize affected person throughput. The result’s each improved affected person care and operational effectivity.
On this session, we’ll share how these outcomes are solely doable with the facility and class afforded by our investments in Azure, Databricks, and more and more Lakehouse. We are going to exhibit Windfall’s blueprint for enabling real-time analytics which may be generalized to different healthcare suppliers.
ABN Story: Migrating to Future Proof Knowledge Platform
Wednesday, June 28 @3:30 PM
ABN AMRO Financial institution is likely one of the high main banks within the Netherlands. It’s the third largest financial institution within the Netherlands by income and variety of mortgages held inside the Netherlands, and has high administration assist of the target to grow to be a totally data-driven financial institution. ABN AMRO began its information journey nearly seven years in the past and has constructed an information platform off-premises with Hadoop applied sciences. This information platform has been utilized by greater than 200 information suppliers, 150 information customers, and greater than 3000 datasets.
To grow to be a totally digital financial institution and handle the limitation of the on-premises platform requires a future-proof information platform DIAL (digital integration and entry layer). ABN AMRO determined to construct an Azure cloud-native information platform with the assistance of Microsoft and Databricks. Final yr this cloud-native platform was prepared for our information suppliers and information customers. Six months in the past we began the journey of migrating all of the content material from the on-premises information platform to the Azure information platform, this was a really large-scale migration and was achieved in six months.
On this session, we’ll concentrate on three issues:
- The migration technique going from on-premises to a cloud-native platform
- Which Databricks options have been used within the information platform
- How the Databricks crew assisted within the total migration
Scaling Up Healthcare Knowledge Improvements: Classes Realized from VA’s Summit Knowledge Platform Journey
Thursday, June 29 @2:30 PM
The Division of Veterans Affairs (VA) is the nation’s largest well being system with 170 VA Medical Facilities and over a 1000 outpatient clinics, serving over 9 million beneficiaries, and using nearly 400,000 staff. Three years in the past, VA Workplace of Know-how (OIT) launched into the event of the Summit Knowledge Platform (SDP) to centralize and modernize information and analytics throughout the VA enterprise. At present, SDP helps over 30 use instances from Lengthy COVID care, suicide prevention, most cancers care monitoring, medical deterioration, veteran expertise, and plenty of extra. These use instances embody a few of the most progressive technological efforts at VA with using Massive Language Fashions (LLMs), self-service analytics on the cloud, and real-time streaming analytics.
Knowledge Democratization with Lakehouse: An Open Banking Software Case
Wednesday, June 28 @11:30 AM
Banco Bradesco represents one of many largest firms within the monetary sector in Latin America. They’ve greater than 99 million prospects, 79 years of historical past, and a legacy of information distributed in a whole lot of on-premises methods. With the unfold of data-driven approaches and the expansion of cloud computing adoption, we wanted to innovate and adapt to new tendencies and allow an analytical surroundings with democratized information.
We are going to present how greater than eight enterprise departments have already engaged in utilizing the Lakehouse exploratory surroundings, with greater than 190 use instances mapped and a multi-bank monetary supervisor. In contrast to with on-premises, the price of every course of may be remoted and managed in close to real-time, permitting fast responses to value and price range deviations, whereas rising the deployment pace of latest options 36 occasions in comparison with on-premises.
The info is now used and shared safely and simply between completely different areas and corporations of the group. Additionally, the view of dashboards inside Databricks permits panels to be effectively “prototyped” with actual information, permitting a straightforward interplay of the enterprise space with its actual wants after which making a definitive view with all related factors duly careworn.
Microsoft-led breakout classes
Sponsored by: Microsoft | Subsequent-Degree Analytics with Energy BI and Databricks
Wednesday, June 28 @2:30 PM
The widely-adopted mixture of Energy BI and Databricks has been a game-changer in offering a complete resolution for contemporary information analytics. On this session, you will find out how self-service analytics mixed with the Databricks Lakehouse Platform can enable customers to make better-informed selections by unlocking insights hidden in advanced information. We’ll present sensible examples of how organizations have leveraged these applied sciences collectively to drive digital transformation, decrease complete value of possession (TCO), and improve income. By the tip of the presentation and demo, you will perceive how Energy BI and Databricks might help drive real-time insights at scale for organizations in any business.
Communications, Media & Leisure Trade Discussion board
Wednesday, June 28 @3:30 PM
Knowledge is on the core of practically each innovation within the Communications, Media & Leisure (CME) business. Leaders throughout promoting, broadcast & movie, communications, gaming, and sports activities are harnessing the facility of information and analytics to digitally remodel and make smarter selections that decrease danger, speed up innovation and drive sustainable worth creation. As a part of our business programming at Summit, we’re internet hosting a devoted Discussion board for CME. This 90-minute capstone occasion brings collectively over 250 in-person attendees and extra on-line for displays, networking and fascinating discussions on the tendencies shaping using information + AI throughout the business.
Processing Prescriptions at Scale at Walgreens
Thursday, June 29 @1:30 PM
We designed a scalable Spark Streaming job to handle 100s of hundreds of thousands of prescription-related operations per day at an end-to-end SLA of some minutes and a lookup time of 1 second utilizing CosmosDB.
On this session, we’ll share not solely the structure, however the challenges and options to utilizing the Spark Cosmos connector at scale. We are going to talk about usages of the Aggregator API, customized implementations of the CosmosDB connector, and the key roadblocks we encountered with the options we engineered. As well as, we collaborated carefully with Cosmos improvement crew at Microsoft and can share the brand new options which resulted. When you ever plan to make use of Spark with Cosmos, you will not need to miss these gotchas!
Associate-led breakout classes
Sponsored by: Avanade | Accelerating Adoption of Fashionable Analytics and Governance at Scale
Wednesday, June 28 @11:30 AM
To compress transformation, allow reinvention because the technique, and use information as the first supply of aggressive benefit, shoppers that we work with are investing into modernizing the foundational components powering their digital core. These constructing blocks supporting the enterprise reinvention are the enterprise metadata and information administration providers, information administration basis, and information providers and merchandise that allow enterprise items to totally use their information and analytics at scale.
On this session, Avanade information leaders will spotlight how Databricks fashionable information stack matches Azure information administration basis ecosystem, how Unity catalog metadata helps automated information operations eventualities, and the way we’re serving to shoppers measure fashionable analytics and governance enterprise impression and worth.
Sponsored by: Infosys | Topaz and Our AI First Method
Wednesday, June 28 @12:30 PM
A dialogue overlaying the journey of enterprise AI First adoption together with standardized framework and toolset, realized with Azure Databricks as a key framework part, for World Organizations.
How Infosys Topaz, our AI-first basis together with Databricks platform will assist our prospects navigate the AI journey for his or her organizations
Well being Care Service Company’s Cloud Migration and Knowledge Analytics Modernization Journey
Wednesday, June 28 @12:30 PM
HCSC is a number one well being care insurer that collaborated with Deloitte to modernize their information analytics and intelligence capabilities by taking the on-premise information lake and information warehouse platforms and transferring them to the cloud.
On this session, Deloitte and HCSC current tips on how to leverage Databricks and automation emigrate options from an on-premise Apache Hadoop platform to Microsoft Azure Cloud. This modernization supplies HCSC larger agility with an elastic, scalable and responsive engineered resolution strategy, which is paramount as they attempt to be the accomplice of selection for advantages and care coordination, enhancing their members’ well being and well-being by way of all phases of life.
Sponsored by: Avanade | Enabling Actual-Time Analytics with Structured Streaming and Delta Reside Tables
Wednesday, June 28 @1:00 PM
Be a part of the panel to listen to how Avanade helps shoppers allow real-time analytics and sort out the individuals and course of issues that accompany expertise, powered by Azure Databricks.
Scaling MLOps for a Demand Forecasting Throughout A number of Markets for a Massive CPG
Wednesday, June 28 @1:30 PM
On this session, we have a look at how one of many world’s largest CPG firm setup a scalable MLOps pipeline for a requirement forecasting use case that predicted demand at 100,000+ DFUs (demand forecasting items) on a weekly foundation throughout greater than 20 markets. This implementation resulted in vital value financial savings by way of improved productiveness, diminished cloud utilization and sooner time to worth amongst different advantages. You’ll go away this session with a clearer image on the next:
- Finest practices in scaling MLOps with Databricks and Azure for a requirement forecasting use case with a multi-market and multi-region roll-out.
- Finest practices associated to mannequin re-factoring and organising commonplace CI-CD pipelines for MLOps.
- What are a few of the pitfalls to keep away from in such eventualities?
Actual-Time Streaming Answer for a Name Heart Analytics: Enterprise Challenges and Technical Enablement
Wednesday, June 28 @1:30 PM
A big worldwide shopper with a enterprise footprint in North America, Europe and Africa reached out to us with an curiosity in having a real-time streaming resolution designed and carried out for its name heart dealing with incoming and outgoing shopper calls. The shopper had a earlier unhealthy expertise with one other vendor, who overpromised and underdelivered on the latency of the streaming resolution. The earlier vendor delivered an over-complex streaming information pipeline ensuing within the information taking on 5 minutes to achieve a visualization layer. The shopper felt that structure was too advanced and concerned many providers built-in collectively.
Our rapid challenges concerned gaining the shopper’s belief and proving that our design and implementation high quality would supersede a earlier expertise. To resolve a direct problem of the overly sophisticated pipeline design, we deployed a Databricks Lakehouse structure with Azure Databricks on the heart of the answer. Our reference structure built-in Genesys Cloud -> App Companies -> Occasion Hub -> Databricks <-> Knowledge Lake -> Energy BI.
The streaming resolution proved to be low latency (seconds) through the POV stage, which led to subsequent productionalization of the pipeline with deployment of jobs, DLTs pipeline, together with multi-notebook workflow and enterprise and efficiency metrics dashboarding relied on by the decision heart employees for a day-to-day efficiency monitoring and enhancements.
Sponsored by: Capgemini | Driving Velocity to Insights with an Prolonged Lakehouse Structure
Thursday, June 29 @1:30 PM
Enterprises have lengthy struggled to allow information as an enterprise asset for the complete vary of information consumption patterns. Whereas we proceed to face obstacles with the normal patterns that embody reporting, ad-hoc analytics, real-time analytics and AI/ML, we additionally now face a wholly new sample that we name autonomous analytics.
In a single instance, inside the context of the Azure Public Cloud, we illustrate how corporations can leverage the open information format of the Databricks Lakehouse, utilizing Delta Lake, mixed with different cloud native and ISV options to prepare and distribute enterprise information in an insights first structure. Discover ways to leverage this structure to offer an expanded enterprise semantic information layer with richer information in real-time that drives essential insights on the pace of enterprise.
Sponsored by: Qlik | Extracting the Full Potential of SAP Knowledge for World Automotive Manufacturing
Thursday, June 29 @1:30 PM
Yearly, organizations lose hundreds of thousands of {dollars} because of gear failure, unscheduled downtime, or unoptimized provide chains as a result of enterprise and operational information is just not built-in. Throughout this session you’ll hear from consultants at Qlik and Databricks on how world luxurious automotive producers are accelerating the invention and availability of advanced information units like SAP. Learn the way Qlik, Microsoft, and Databricks collectively are delivering an built-in resolution for world luxurious automotive producers that mixes the automated information supply capabilities of Qlik Knowledge Integration with the agility and openness of the Databricks Lakehouse platform and AI on Azure Synpase.
We’ll discover tips on how to leverage the IT and OT information convergence to extract the complete potential of business-critical SAP information, decrease IT prices and ship real-time prescriptive insights, at scale, for extra resilient, predictable, and sustainable supply-chains. Learn the way organizations can monitor and handle stock ranges, predict demand, optimize manufacturing and assist their organizations establish alternatives for enhancements.
Delta Sharing: The Key Knowledge Mesh Enabler
Thursday, June 29 @2:30 PM
Knowledge Mesh is an rising structure sample that challenges the centralized information platform strategy by empowering completely different engineering groups to personal the information merchandise in a particular enterprise area. One of many keys to the success of any Knowledge Mesh initiative is deciding on the precise protocol for Knowledge Sharing between completely different enterprise information domains that might doubtlessly be carried out by way of completely different applied sciences and cloud suppliers.
On this session you’ll study how the Delta Sharing protocol and the Delta desk format have enabled the traditionally stuck-in-the-past power and building business to be catapulted to the twenty first century by the use of a contemporary Knowledge Mesh implementation based mostly on Azure Databricks.
We additionally invite you to go to the Microsoft sales space on the Expo ground, the place you will get to speak 1:1 with Azure information engineering on tips on how to handle your hardest analytics challenges with Azure.
Register now to hitch this free digital occasion and be a part of the information and AI group. Learn the way firms are efficiently constructing their Lakehouse structure with Azure Databricks to create a unified, open, and scalable information platform. Get began utilizing Databricks with $200 in Azure credit and a free trial.