Sean Maloy: Hi, welcome everyone, and thank you for joining our webinar today. My name is Sean Maloy. I am the AWS Alliance lead here at DataArt. Today we will be discussing how DataArt, in partnership with AWS, enables clients to overcome challenges and accelerate their database migrations to AWS. Our speakers today are Lesley Ajanoh, who's a partner database leader at AWS, as well as Oleg Komissarov and Aleksei Gorbunov, who are database migration leaders here at DataArt.
Before I hand it over to our esteemed speakers, I'd like to just remind you all to ask questions. Don't be shy. Please do that in the comments section on the right side of your screen, and we will do our best to answer as many questions as possible during the dedicated Q&A at the tail end of this program. If we can't get to them all, we'll follow up offline. So with that, just wanted to say welcome again. Thanks, we appreciate your attendance. Let's get this started. Over to you, Lesley.
Lesley Ajanoh: Thank you, Sean. Before I start, I'd like to take a minute to find out from the audience how many of you are currently running databases on AWS today. You can just respond by putting plus one in the comments. If you can specify the database engine you're running today, that'll be great.
At AWS, our vision is to help you manage your data no matter where it lives. We understand that you live in a world where you have a combination of data on premises, some in one or more clouds, some in SaaS applications, and all of this data needs to be catalogued and governed, breaking down historic silos. Our goal is to meet you where you are and help you realize your target state.
It doesn't matter whether you're running applications which are monolithic, client-server, three-tier, or microservices. We can help, and we have partners like DataArt that can help you move fast. It doesn't matter whether your goal is to replace an aging server, deprecate your data center, or to build an application for 100 million plus users from day one. At AWS, we have the approaches for you, and we have partners like DataArt that we can work together with to help you get to where you want to go.
Our competitors are going to tell you that they have the technology to solve your data challenges, but we have strategies to provide you the right tool for the right job via our relational and non-relational databases. This breadth of databases has resonated with our customers so much that we've seen that the top 1,000 customers are using ten or more of our data and analytics services at AWS.
Working with customers across different industries of various sizes, what we've discovered is that they all have three core elements of their end-to-end data strategy. Modern data strategy starts with migrating to the cloud, moving towards infrastructure that enables you to achieve the scale you need at the right cost, while reducing operational burden.
The three core elements that the leaders of our customers across all verticals, especially the very large enterprise customers, implement are:
First, they always start with a comprehensive set of services. When you're thinking about what can we do with our data, how can we innovate with our data, the first element is having the right foundation. That foundation has to include a comprehensive set of services which can meet their needs now and their future needs, so that as your business scales, you don't have to re-architect your environment just because your business is scaling. If you're thinking about building a new product, you can quickly build that product without worrying about the risk and the complexities of re-architecting. Most of those foundations are built on top of our purpose-built databases, which I'll be talking about later.
The second core element of the end-to-end data strategy is to have a solution that can integrate all the data services so you can access your data from wherever you are. Our solutions give you the ability to get that integration on the foundation which you build with a comprehensive set of services.
Now that customers have built the right foundation with our services and enabled the right integrations, what they need is to have the right governance in place that will enable the team to quickly drive value from their data. Because at the end of the day, every company wants to drive value from their data. That's where we have the right governance that brings in the democratization of the data.
So the core elements of an end-to-end data strategy include leveraging the right purpose-built databases to have the right foundation, which is scalable, reliable, high-performing with faster response times, and then the right integration and governance on top. Our top customers across the biggest industries in the world are achieving that business value today on AWS.
I would like to highlight that Gartner recently recognized AWS as a leader in their Data Management for Analytics Magic Quadrant in 2022. This time we were ranked highest in execution and ability to execute, and furthest for completeness of vision. This report is based on the feedback from the top 2,000 customers for data and analytics, and the results rank AWS as a leader.
How do you build a modern data strategy? You have to start by looking at your existing infrastructure, assessing all those legacy systems, and then retiring all this technical debt by moving to databases that will help you scale your business, that will help you reimagine the customer experience, and that will help you get the performance so that your reports will be able to run in real time and deliver value for your stakeholders.
We have a breadth of databases, so it doesn't matter what kind of databases you run. There's no database which is too big or too complex for us. We have relational databases which fall under our RDS database service, and we have Amazon Aurora, which gives you one-tenth the cost of commercial databases like Oracle and SQL Server at the same performance – and even better performance, because we've seen customers running Aurora getting greater performance.
One of the values of Aurora that I'd like to highlight is that it enables you to have the serverless option. With the serverless option, you have the ability to get capacity on demand, so you don't need to worry about capacity planning. You don't need to worry about setting up for high availability, because with a single click, you can get high availability. With a single click, you can go global. So if you have customers in Indonesia and customers in Africa, with just a single click Aurora enables you to achieve that.
In addition to the relational databases, we have our NoSQL databases. These are called purpose-built databases. We actually built the very first purpose-built database, which is DynamoDB, and we moved Amazon.com from Oracle to DynamoDB. Just imagine – if we could run a big giant like Amazon.com on DynamoDB, then there's no complex workload that we can't run on AWS.
For our NoSQL databases, we have DynamoDB. We have ElastiCache that can help you get better performance. We have Neptune that helps you for managing relationships. If you have data that has to do with relationships where you want to correlate the relationships to track fraud, or to track customer experience and track recommendations to be able to personalize the experience for your customers, Amazon Neptune can do that. If you're thinking about blockchain, we have our QLDB. And if you have time series data that you want to integrate with IoT, we have our Timestream.
As customers move to the cloud, what they normally did traditionally was they retire the technical debt by lifting and shifting. But as you lift and shift your databases to EC2 on AWS, you realize that you start seeing a reduction in cost and an increase in innovation. As you move to managed databases like the Amazon RDS services that I mentioned earlier, you realize that the cost continues to reduce and your velocity to innovate continues to accelerate.
Every business wants to innovate so they can compete with their peer groups and have competitive advantage. Moving to managed databases frees you of all undifferentiated heavy lifting tasks, which Oleg is going to go into later, and helps you to drive innovation. As customers continue to move from the managed to purpose-built databases, they see the greatest savings, especially around velocity. The velocity increases – they have the highest velocity when they're running on purpose-built databases like Aurora and DynamoDB.
They're able to optimize cost to the fullest because at that point they're able to leverage serverless offerings, which you don't have to worry about capacity. It scales on demand, and you only pay for what you use for the time you're using it. You're able to get very low latency, high throughput, greater durability of their data, and excellent scalability and better resiliency.
You're not alone in thinking about migrating your database applications to AWS. We have over 800,000 databases that have been migrated, and these are databases that have migrated using only one of our tools called DMS. This number does not include other database services or other tools that customers have used to migrate.
In addition to the 800,000 databases that have migrated to AWS using AWS DMS, which is our Database Migration Service that we'll dive deep into during this session today, we have customers across every segment. If you look at the industries represented, you might find customers in your industry. This is just to let you know that you're not alone. We have every use case covered, we have the right tools for you, and we have partners like DataArt that can support you.
If you're thinking about migrating and wondering how to go about it, how to start the journey, or if you don't have the expertise, we can support you. We want to help you manage your data. We want to help you become data-driven. We want to help you reimagine the customer experience for your customers. We want to help you run your reports faster. We want to help your application team be able to have fast access to your database so they can increase their deployment frequency. We've done that for customers across different segments, and you can just come to us – we're ready to help you, and we have DataArt to help you with your strategy.
I'd like to highlight some successes that our customers have seen moving their workloads. Australia Financial Group is one of them. They were spending over 80% of their budget on operational costs, and they migrated their Oracle Exadata to Amazon RDS for Oracle and were able to save $500,000 USD per year. This was a huge saving which could be put toward building new applications to innovate the business.
Another use case is Kaplan. Kaplan was able to deprecate their data centers by moving to AWS. We see other customers like Cathay Pacific that were able to gain better performance as compared to running on premises by moving their workloads to Aurora.
Lastly, we have customers that sometimes are not ready to move in a one- or two-step approach because they're not ready to let go of the licenses for the commercial database. These customers move to RDS and then later on, once they build the capabilities and leverage partners like DataArt to support them in building their strategy and planning the roadmap for migration, they refactor from Oracle to be able to break free from the commercial database licenses, which are very expensive. They're able to be free from the frequent audits that pull money away from them and achieve full business value by leveraging our cloud-native databases like RDS Postgres and Amazon Aurora.
These are just examples of how customers are winning on AWS. There are many more examples of customers that have optimized their costs and achieved great results. You can find much more case studies on the AWS website.
We know you understand the business value of optimizing your costs and increasing the velocity to innovate. But how do you get there? We have the tools to support you. We have our AWS DMS, which as I explained, over 800,000 databases have been migrated using AWS DMS. There's no workload too complex or too big that DMS cannot help you with.
But DMS is just one of the tools. If you're thinking of modernizing, refactoring, or breaking free from the licenses of Oracle and Microsoft SQL Server, we have Schema Conversion Tool that can scan your code and give you recommendations and show you the level of effort required for you to move this workload from the current state to the target state on AWS. Partners like DataArt can help you drive that value with less risk to your business and with faster time to value.
In addition to that, we have programs like the MAP (Migration Acceleration Program), which we have used to help many customers migrate. We're willing to share the solutions and lessons learned that we got from working with hundreds of customers, modernizing Oracle, SQL Server, and mainframe applications to run on Amazon RDS, Aurora, and DynamoDB. We have solutions that we can show you to help you plan how to move.
Thank you. I'm going to hand it over to Oleg, who's going to talk about DataArt and how DataArt can help you.
Oleg Komissarov: Thank you, Lesley. There are three different migration paths or migration approaches that bring different business values.
Migration Approaches
Lift and Shift (Rehosting) is a migration approach for moving existing commercial databases like Oracle or Microsoft SQL Server to the cloud without major changes. The main business value here is infrastructure flexibility, while the database remains self-managed and operated similarly to on-premises systems.
Move to Managed (Replatforming) involves migrating databases to Amazon Relational Database Service. The key business value here is reduction of self-managed efforts, including backup, recovery, and patching. It also enables greater scalability to support the needs of growing applications and businesses. Organizations can also expect reduced licensing costs.
Modernize (Re-architecting) entails moving away from commercial databases and shifting to AWS purpose-built managed databases that Lesley mentioned, such as Amazon Aurora, DynamoDB, or Redshift. This approach delivers superior technical scalability and flexibility, allowing businesses to focus on adding unique business value. Modernizing also enables businesses to optimize costs further and leverage the extensive array of AWS database services, because these purpose-built databases are integrated with the rest of AWS services. It also frees available in-house resources that could be redirected from non-differentiated activities to innovation and implementation of business priorities.
Self-Managed vs. Managed Databases
As you can see, managed and self-managed databases provide different values. Let's understand the details better.
Self-managed databases give you complete control over the infrastructure, from patching and maintenance to backup and recovery. You are responsible for the entire database stack management, including the operating system, database software, and hardware. This requires significant expertise and resources, but it also allows for greater customization and control.
Fully managed databases such as Amazon RDS take on much of the responsibility for the database infrastructure. AWS manages infrastructure, operating system, database software, and scaling. You are managing the database schema, database configuration, and database data. Managed databases reduce the amount of maintenance and administration but still allow for some flexibility and customization.
The third type, RDS Custom, is a fully managed database service, but you can also choose your own software on top of it, or you can even bring your own license for specific database engines.
Overall, self-managed databases provide the greatest level of control and customization but also require the most resources and expertise to manage them. Fully managed databases provide a balance between convenience and control, and RDS Custom provides an even greater level of control and customization while still benefiting from infrastructure management by AWS.
AWS provides various target database migration options. RDS is a managed relational database service that supports MySQL, PostgreSQL, Oracle, and Microsoft SQL Server engines. Amazon Aurora is a managed database engine that was built by AWS for the cloud. It is compatible with MySQL and PostgreSQL.
How is Aurora different from RDS? Aurora's performance, scalability, and availability exceed traditional databases, making it a popular choice for modern applications. Aurora also provides automated failover capabilities, allowing for rapid failover in case of database outage, which is very important for enterprises.
After completion of thousands of migration projects and RDS and Aurora implementations, AWS qualified DataArt to perform RDS Service Delivery engagements.
Migrating databases can present significant challenges for businesses, including the need for time-consuming and manual discovery processes, the complexity and potential for errors during the migration process, and the high cost of refactoring legacy systems. Additionally, there is a high risk of not completing migrations on time and on budget, as well as creating more technical debt. Another significant challenge is a lack of internal expertise, which can impact effective analysis, planning, design, and execution. All of these have an impact on identifying the correct migration path and managed database option and target database engine.
A complete database migration solution combines several components: automated migration tools and services, professional enablement, and acceleration programs. All of them are very important.
Automated tools include Database Migration Service, Fleet Advisor, and Schema Conversion Tool. You will see a short demo of these tools later today when Aleksei presents.
Professional migration services consist of three stages: assessment, mobilization, and migration. Experienced migration engineers are trained to use migration best practices, guides, and out-of-the-box migration accelerators to enable careful analysis, planning, and execution so that businesses can reduce the risk of complications that I mentioned on the previous slide and realize the business benefits of database migration.
AWS also developed Migration Acceleration Programs, which is the third component that may help you receive funding and additional professional help to accelerate your database migration.
DataArt has completed thousands of migration projects and was qualified by AWS as a Migration Services Competency Partner. Our Migration Services Competency team developed a specialized consultative Database Migration Assessment offering that covers the assess and mobilize stages of database migration. This assessment is typically conducted in several weeks. It involves use of automated migration tools that I mentioned and accelerators, but we also tailor these tools and methodology to your specific needs and context.
The assessment provides actionable deliverables such as recommendation reports, including migration path, database engine type, managed database type, migration roadmap, and cost and time estimates.
I mentioned MAP – Migration Acceleration Programs. This is a program by AWS that provides access to funding for qualified workloads. Large workloads are eligible for up to 25% savings. Smaller workloads could be qualified for up to 15% savings, and proof of concept projects could be qualified for full funding.
As a qualified migration partner, DataArt can help identify if your migration meets qualification criteria, and we can help you apply and get approval for MAP program and realize MAP program benefits. These migration acceleration recommendations are also included in our consultative assessment offering.
If you are starting your database migration journey, executing migrations, or you need consultation or additional professional help, please feel free to contact us at partnerships@dataart.com to inquire about our Database Migration Assessment. This is a great and effective way to get on a successful migration path.
Thank you, and with that, I'm handing over to Aleksei, who will provide a demo.
Aleksei Gorbunov: Thank you, Oleg. Hello everyone. As Oleg mentioned, automated tools are a part of the complete database migration solution. Today, I will demonstrate two database migration acceleration use cases: Oracle to Postgres Aurora migration assessment with Schema Conversion Tool, and SQL Server migration assessment with DMS Fleet Advisor.
Schema Conversion Tool is a desktop application which needs to be downloaded and installed locally to access the target database. It automates several steps: analysis of database conversion complexity, discovery of migration restrictions, analysis of licensing considerations. It also automatically converts database schema from source to target, highlights migration issues, generates action items where automated conversion cannot be completed, and finally converts SQL scripts and even SQL code in applications.
The Schema Conversion Tool supports assessment for more than two dozen database engines, data warehouses, and analytical tools, including all popular commercial databases.
We will focus today on the migration assessment report, which plays a crucial role in the database migration process. To generate an assessment report, I connected to an Oracle instance which I'm planning to migrate and have chosen the specific schema that I want to assess. Now I am ready to create the report.
The assessment helps us understand the complexity of the migration and provides us with analysis of migration to all possible target database engines. Specifically, in our case, you can see several different target engines we can use for our migration with different levels of migration complexity. Migration complexity can be described by percentage of storage objects, code objects, and conversion actions which can be applied or executed with minimal or complex manual intervention.
The executive summary can help us choose the preferred migration target. We can see similar levels of complexity for Amazon RDS for PostgreSQL and Amazon Aurora PostgreSQL-compatible platform. In our case, I would prefer to use Amazon Aurora PostgreSQL for our migration. The potential driver for this decision could be the unique features of Aurora, such as global database and serverless options.
Scrolling down, we can find target-specific features available for each individual target identified during our assessment. This part provides color-coded histograms to describe items and migration efforts for specific engine targets. The colors reflect the complexity of the migration.
On my next step, I'm going to choose the target platform and connect to my Aurora instance to proceed with schema conversion. After the connection is established, we can switch to the assessment report to access the target-specific report on the action item types. Here you can find all actions required to complete migration, including description of the issue, recommended actions, occurrence statistics, and even documentation references available in the main view.
We can proceed with schema conversion. Usually schema conversion takes a very short period of time, but for large databases, it could take hours. Objects that require manual intervention are marked with an exclamation mark and small red circle. You can open the object that requires attention, find highlighted code which records the issue, and even adjust and validate code generated for the target platform.
Finally, the converted schema can be saved as a SQL script or applied to a target instance. As you can see, Schema Conversion Tool is an effective way to convert your existing database schemas from one database engine to another. It also provides several powerful features to help streamline the database conversion process.
Let me move on to the second use case. Our second use case is SQL Server Migration Assessment with DMS Fleet Advisor. DMS Fleet Advisor is a tool for organizations running large fleets of databases who need to capture the full picture of their database inventory in order to identify the databases to migrate and build a migration plan for discovery and recommendations.
The key features of Fleet Advisor: currently, Fleet Advisor supports Oracle, SQL Server, MySQL, and PostgreSQL. Please note, currently recommendations can be built for homogeneous migrations only.
For database fleet discovery, the DMS data collector agent must be installed and configured in your local data center. It collects data and securely uploads it to an S3 bucket. The agent collects server metadata, database schema, and optionally resource utilization metrics. DMS uses that information to analyze databases and schemas and build migration recommendations. Only database metadata is transferred – the data itself is never accessed by data collectors.
Let's take a look at the database inventory built by DMS. In my example, three databases were discovered during assessment. The inventory can be used for migration planning by looking at factors such as database type, complexity of the migration, and similarity of schemas. On the schema level, we can find information about object count, object size, and line of code counts. The inventory can be exported to a CSV file for further processing.
Finally, let's take a look at the recommendations. To generate recommendations, I click the Generate Recommendation button. Here I can select one or more databases from the inventory list, select target availability type and target instance sizing strategy, and click the generate button.
Let me open a report already generated to see the value this report provides to us. As you can see, this report specifically provides the RDS target option which is available for SQL Server. We can find more options for engines which have alternative implementations in AWS, like PostgreSQL and MySQL, both of which can be implemented as RDS or Aurora.
The target recommendation report contains important information about target instance type and size, information about storage, license model, and cost estimates. From here, you can use this calculator and change the recommended configuration to see how projected costs change.
The discovery and assessment can be conducted for hundreds of servers automatically, and DMS Fleet Advisor builds migration recommendations including instance type and size and provides cost estimates. The recommendation feature is brand new and became generally available as of March 2023.
Thank you. Sean, the floor is yours.
Sean Maloy: Thanks, Aleksei. Now we've come to the portion for Q&A. Let me gather some of these questions. I would say if you do have questions, it's not too late, so do enter those into LinkedIn as comments and we'll try to get to those in turn.
Question 1: Can you provide a real-life example of a migration assessment that you've done with the Schema Conversion Tool?
Oleg Komissarov: Let me take this question. One of the recent case studies that we have where we used the Schema Conversion Tool is with a large European stock exchange – one of the largest European stock exchanges. The tool helped us streamline migration of the same case that Aleksei presented: Oracle on-premises to Aurora PostgreSQL migration.
We performed analysis using Schema Conversion Tool on an Oracle database that had more than 400 tables and several thousand stored procedures. The tool did what it's supposed to do – it helped us automatically convert more than 85% of all database objects to Aurora PostgreSQL. In my estimate, that saved us about two to three months on analysis and target schema recommendations.
Since most of the conversion was performed automatically, the migration team focused mostly on database refactoring, taking care of issues highlighted by the Schema Conversion Tool, and improving target database schema structure. As of today, the client is running the migrated database in the cloud, enjoying all the benefits of Aurora that we mentioned in the presentation: disaster recovery, cross-regional data replication. So far, everything has been very stable. We have not had any production issues related to the migrated database.
Sean Maloy: Great. Maybe a follow-on to that: does DataArt offer any proprietary tools to supplement those AWS tools for migration?
Oleg Komissarov: Yes, we use proprietary tools specifically for popular database assessments – Microsoft SQL Server, Oracle SQL. These tools are not official services backed by AWS. Those tools help to perform even deeper analysis of databases, which is useful for experts.
Also consider that what we are offering in this webinar is an assessment package. After assessment, DataArt can execute this migration project. Part of this assessment includes setting up all the migration environment, which also has to be reliable and scalable. This usually takes time to configure Database Migration Services or other approaches.
Another tool that we have developed is our set of CDK (Cloud Development Kit) scripts and modules that help to automate infrastructure provisioning for migration and migration monitoring. This is another kind of proprietary accelerator that we offer and use.
Question 2: I would like to move to managed but also perform some code improvement and re-architecture. How would this impact the assessment stage?
Oleg Komissarov: The assessment stage would include, in this case, based on your business goals and objectives, specific recommendations. For example, in the case study that I mentioned earlier, the migration assessment recommendation was to move data, enable real-time replication from legacy to new database, and then continue with refactoring in the cloud gradually. We redesigned schemas and organized cloud-based data transformations to the new schema.
That was one of the approaches. The database was huge, and it's practically impossible to do a big bang migration plus perform schema transformation in one shot. It would be too much instability for the enterprise and create potential issues.
But if your workload is smaller, then potentially migration itself will include conversion. In this case, even Schema Conversion Tool will be able to help transform data to the new schema. Basically, these recommendations about your suggested migration path and how you perform re-architecture will be included after the assessment phase in the project.
Question 3: We currently have an old legacy application. Our goal is to modernize this. What are some steps we can use to securely migrate our data? We currently use MySQL databases.
Aleksei Gorbunov: I can take this one. One of the most popular approaches to data migration is using AWS DMS (Database Migration Service), so you can easily migrate data from one source to another. Specifically here, you can use homogeneous data migration and migrate your MySQL database to RDS MySQL. You can also use as a target Aurora MySQL-compatible platform for data migration.
Question 4: Can you talk more about migration toolset limitations?
Aleksei Gorbunov: Obviously, migration tools – all migration tools – have some limitations for heterogeneous database migrations because of the major different nature of data platforms we want to migrate from one engine to another. But besides that, the tools provide huge improvement in terms of how much time you need to complete assessment of your migration and help you migrate a large amount of data from on-premises to cloud, like the Database Migration Service, for example.
From my perspective, the biggest concern can be heterogeneous database migration. But even in this case, tools like Schema Conversion Tool provide you prescriptive guidance on what exactly you need to do in order to complete your migration.
Lesley Ajanoh: I just wanted to add – I explained why over 800,000 databases have migrated to AWS, and there's no workload or database which is so big or complex that we can't handle. I think when talking about limitations, we need to understand the use cases, because we've been able to migrate the biggest customers across every vertical. All we need to do is assess your workload and find the right approach for you.
If you have a scenario which you think is going to present some kind of limitation, reach out to us. We've got the DataArt team that will work with you to build a roadmap and show you how you can unblock those challenges and achieve your value by migrating to AWS.
Oleg Komissarov: Sometimes there are really challenging cases – for example, when clients want to move off Microsoft SQL Server but they have a large estate of applications that already integrate with this database. Potentially that represents a huge volume of refactoring for migration to a new engine like PostgreSQL.
In this case, my migration recommendation would include considering a new offering by AWS: Babelfish. This wraps around a PostgreSQL database, retaining all your Microsoft Transact-SQL statements and queries seamlessly. So for many limitations or challenging cases, there could be solutions – probably not even part of the migration process itself, but technology answers.
Sean Maloy: Thanks everyone for coming, especially thanks to our partners at AWS. Thank you, Lesley, for your active collaboration on this. We covered a lot of ground here in about 45 minutes, so I'm sure some of you may have some remaining questions.
If you would like to reach out and spend some time with our experts here, we're happy to schedule a call or even a workshop. Please reach out to us. You can reach us at partnerships@dataart.com. Once again, thanks a lot, and have a great rest of your day.
