At Alice&Bob we are always on the hunt for interesting partners and solutions to support our core mission of AWS Security. One of these partners is WANDisco and we are proud to share some insights on what WANDisco does, how we come into the picture and what’s in there for you.
WANDisco describes itself as the ‘leading data migration and replication platform’ and as such we had the opportunity to delve into their offerings. The key aspect of WANDisco is to enable you to run true hybrid cloud architectures. They offer the technology to move, replicate and synchronize arbitrarily large data sets focused on Hadoop, across on-premise and all major cloud providers. It’s obvious that we’re clearly focusing on AWS on this part, but we’re also certainly acknowledging the fact, that many customers are not in the position to keep their technology basis as rigorous.
There are many reasons why a customer has split their technology basis onto multiple platforms. It could be as simple as being stuck with long running investments, where the core IT Platform is bound for some more years and must be utilized. A multi-cloud architecture can also happen by accident, as the result of a series of M&A activities or simply as a result of the intricate mazes of corporate politics. But it could also be a well thought out and sophisticated strategy, combining the different abilities and properties of several providers, while keeping strategic flexibility. If the latter was not the start of your journey, it should be at least your goal.
In the following section we want to present two scenarios, where WANDisco does offer capabilities to extend on-premise and cloud services and help you to be in command of your data and its location.
Scenario 1: Rehost or Lift and shift with WANDisco LiveMigrator
If you recognized yourself or your organization in the first two situations, having a legacy infrastructure and just now starting your journey into the cloud, we recommend to read and study AWS migration guide, available as a white paper. The appropriate first step would be re-hosting and re-platforming (a.k.a lift-and-shift). The goal is to transfer application and data into the cloud with minimal or no modification as a first step. For many organizations, the effort to relocate workloads is difficult enough and can take years, so any modifications to really unlock additional cloud capabilities has to wait until this difficult task is done. The same scenario is also applicable if you want to establish an off-site backup.
In the context of Hadoop, we often talk about Petabytes of Data which would have to move, while production must keep running. AWS does offer some services which can help you with these challenges, for example AWS Storage Gateway allows you to provision a local VM which serves as a target for file transfers and can upload the content transparently to S3 in the background. For larger data movement, you can use AWS Snowball which physically ships a disc-array which you fill locally and ship back to AWS to copy it directly to S3 on-site. This sounds archaic, but depending on the data set size Internet Protocol over (avian) Carriers is still a thing!
Both solutions however have disadvantages, they both are not aware of the character and properties of your data and treat it as generic files. You have to organize the whole copy procedure yourself. You have to take care of checking consistency, keeping track of deltas during the copy operation making sure to reintegrate it into a Hadoop setup in the cloud and not negatively impacting your running production setting. Guaranteeing any consistency over this process is likely nothing you want to worry about and using AWS or Hadoop tools, you only get a one-time copy if at all.
Using WanDisco LiveMigrator, this is one less thing to worry while you bootstrap your cloud-journey or establish an off-site backup, guaranteeing business continuity and data consistency.
Scenario 2: Mix-and-Match cloud providers with WANDisco LiveData
Intended or by accident — heterogeneous environments with on-premise setups and multiple cloud provider in parallel are a reality which many organizations are faced today and will be in the future. Regardless of the reason why, the question of whether this situation is your worst nightmare or a potent hybrid cloud strategy is up to you to decide. As in all business areas, competition stimulates business. Although we are convinced of AWS as a strong and strategic partner and technology platform, there might be strong and compelling reasons and scenarios where an on-premise solution would prove superior for some parts of your data processing, while other parts would be most suited for an Azure, GCP or AWS based environment. A hybrid cloud strategy can be a very compelling vision, but the reality of maintaining multiple technology bases and designing work-flows across them proves to be nightmarish difficult. Taking this problem of maintaining data-consitency across all platforms out of the equation using WANDisco LiveData, it transforms into a potent strategy.
WANDisco LiveData guarantees live consistency and availability of your data between multiple platforms, whether its between on-premise Hadoop installations, on-premise Hadoop to cloud, or in between different clouds. Using their patented Distributed Coordination Engine (DconE) technology, all your platforms appear to work onto the same data storage.
You could easily continue running your heavy-lifting BigData infrastructure, export results to AWS to run AWS SageMaker as a managed ML/AI service on top of it while you gain BI insights using AWS QuickSight as well as additional, nearly infinite Ad-hoc querying capability with AWS Athena. Afterwards transparently re-import them into your on-premise infrastructure or even pass them along to some unique Azure or GCP services of your choosing. With WANDisco LiveData, all your platforms do appear as one single consistent data storage. This allows you unlock the unique strengths of each platform, rather then being subject to a fractured landscape of data dumps.
Alice&Bob is implementation Partner of WANDisco
Having a solution like WANDisco enables you to decide and take control of where your data resides and how it moves between your sites and platforms. Offering a 100% guarantee of consistency, you can continue to use your on-premise Hadoop for the heavy lifting, replicate the data to AWS using WANDisco, applying all managed cloud services of your choosing of all providers that you need and finally replicate its results back into your running Hadoop pipeline. But it could also serve as a tool to convince your CEO of the magnificent world of clouds start your cloud journey even today, without interrupting your production the slightest or go in the reverse direction and establish a hot off-site backup to guarantee business continuity.
We as Alice&Bob are thrilled to have someone like WANDisco as our partner and are operating as an implementation partner in the DACH-Region. WANDisco delivers the technology, while we are supporting you during preparation, setup and maintenance, as well as consultant on every stage of your journey.