Getting your data in the cloud with Snowflake and DataVirtuality

 

Cloud analytics databases are the rising stars of data centralization technologies. They combine the familiarity of a traditional data warehouse, the scalability of data platforms and the elasticity of the cloud. Combining all these advantages, cloud databases make your daily work as effective and agile as you need it to be. Learn more about the facts and technology of integrating your data into an analytical cloud database.

Your data with no limits - Snowflake

Snowflake is the only data warehouse built for the cloud. Snowflake’s high-performing cloud analytics database combines the power of data warehousing, the flexibility of big data platforms, the elasticity of the cloud and true data sharing at a fraction of the cost of traditional solutions.

Snowflake has the vision to make modern data warehousing effective, affordable and accessible to all data users.
One special thing about Snowflake is it’s ability to efficiently separate the storage capacity from computing capacity. This eliminates the limitation common to the cloud databases which does not allow growing the storage capacity without paying more for the computing capacity -- which you may not need at all.

Snowflake’s bing native cloud database has an important ability which is vital for data warehouses -- namely elastic scalability. That means that Snowflake is able to dynamically scale up resources as needed in order to load the data into the database. This way, loading of data into the data warehouse does not negatively affect the execution of user queries.

What comes first? Move and model your data!

DataVirtuality provides next-generation data integration solutions that enable detailed insights from real time and historical data with any BI tool. By combining data virtualization with an automated ETL engine, customers benefit from reducing their data integration effort by 80%.

Taking a closer look at the DataVirtuality platform, you have several options you can use together with Snowflake: Pipes and the Logical Data Warehouse.

 

Get to know Pipes - Your data movement solution

Especially young digital businesses are lacking of technical possibilities to connect heterogenous data sources for analysis. Pipes is the perfect choice when it comes to moving your data in your Snowflake cloud database. With Pipes you profit from more than 150+ connectors and and easy-to-use interface.

 

 

Pipes offers both self-service capabilities for BI starters but also the flexibility and the enterprise connectors required to handle complex enterprise scenarios.

You can use existing data extraction templates -- the intuitive interface will guide you through the process. Easy as buying a new sweater while shopping online, you can add data sources into a data warehouse. The setup only takes a couple of minutes.

At the same time, you can also configure custom data extractions, enjoy complex data transformations and connect custom systems. Datavirtuality’s solution engineers will help you connect and configure the data pipelines -- or will take over the job, so that you can concentrate on your business.

 

Move, model and access your data with a logical data warehouse

For those of you who want to go beyond the simple data movement, there is the Logical Data Warehouse.

The DataVirtuality Logical Data Warehouse connects, models, and moves data for analytics and process automation purposes. Combining data virtualization and next generation ETL enables a revolutionary agile data infrastructures with high performance. DataVirtuality can analyze data from any databases with over 150 pre-built connectors. By simply using SQL, you can work with the Logical Data Warehouse in a time and cost-saving manner (the DataVirtuality SQL processor even works for Hadoop and NoSQL databases that do not support SQL natively).

A self-learning algorithm will optimize each and every query and provide you with the high-performing solution that your business needs.

You can set up your data infrastructure in under 5 minutes, access data silos, and uses SQL to query raw data for insights.

 

 

DataVirtuality moves your data into the cloud database or builds a central data model for you. Snowflake delivers the performance, concurrency and simplicity needed to store and analyze all your data. And both solutions work together perfectly only by using SQL.

A list of all ready-to-use DataVirtuality connectors to Snowflake here.

 

What about data security?

Snowflake and DataVirtuality follow European data protection laws and enterprise requirements. You have an option of hosting both systems in a european hosting location (Frankfurt). With DataVirtuality you also have the possibility to choose between doing it on premise or in the cloud.

For those who have concerns of having your data in the cloud: Snowflake is the most secure version of the modern cloud data warehouse while using VPS.

Integrate all your data in any BI tool and get a central data model

Knowledge is the power for your business and the road to your success. This knowledge lays within your data. Having powerful solutions will be beneficial for you to help integrating your data from several sources and centralize it - ready for your preferred way of analyzing it.

Imagine having several different data sources with totally different requirements and you want to save the structured and unstructured data in your Snowflake analytical cloud database to analyze it with your BI tool, such as Tableau. DataVirtuality has ready-to-use connectors to more than 150 data sources. The logical data warehouse can now move and model the data within one day. Your data is structured and ready to load into and save it in your Snowflake cloud database. From there it is always accessible with any BI tool you might choose. Through the data virtualization layer you don’t have to run useless queries and block the capacity of your server. Users from every department can access the data that they really need, without manually starting and maintaining queries.

 

 

In our example use case above the BI Stack would be:

 

Data sources that want to get integrated in the data warehouse for data analytics:

  • Salesforce,
  • Facebook,
  • Google Analytics
  • … and all your other data sources

Data warehouse to store these data:

  • Snowflake

BI tool to visualize the data sources:

  • Tableau
  • Looker
  • … and any other BI tool of your choice

Data integration and data modelling platform to connect the data sources with the BI tool and to build a central data model:

  • DataVirtuality

Do you want to learn more and see how it works for your individual use case? Then get started here!

The hybrid approach: for cloud data warehousing plus traditional data warehouses

In case you have some parts of your data in your Snowflake cloud database and some in a traditional data warehouse, such as Oracle or Hadoop, the Data Virtuality Logical Data Warehouse functions as a homogeneous data layer that makes all your data accessible for your BI tools. You can decide where you want to store your data. Through this hybrid approach you don’t have to worry about having your data in different data warehouses or databases.

 

 

By combining the Snowflake cloud database and DataVirtuality’s data integration solution you will have a safe, flexible, agile, and high-performing data infrastructure that scales perfectly with your business needs. And everything just by using one language - SQL.  

See a comprehensive list of all connectors available to Snowflake here.

 

Find out more about Snowflake at snowflake.net