General Questions Regarding our Platform and Solutions
How does your pricing look like?
For Data Virtuality Pipes:
You can find the pricing for Pipes here.
For Data Virtuality Pipes Professional and Logical Data Warehouse:
Min. 12 months subscription.
The pricing is based on:
1. the number of server instances
2. the number of connectors
You are not being charged for data volume or user/seats. Support and connector maintenance is included in the subscription.
For an individual quote please write an email to email@example.com containing:
1. number of server instances (minimum 1)
2. number of connectors (see connector list here). You are only charged for data source connectors, not for analytical storage and front-end connections.
What is the difference between the 3 solutions: Pipes, Pipes Professional and Logical Data Warehouse?
Data Virtuality Pipes is a web-based cloud-ETL/ELT solution that helps you to move your data into your target storage - on schedule without the need to code. Pipes is often used for use cases where you would like to move data from a database or API to an analytical database with minimal to no maintenance efforts.
Data Virtuality Pipes Professional is a premium ETL/ELT solution. In comparison to Pipes it gives you complete access to the underlying SQL (allowing sophisticated transformations, stored procedures etc.) and you have diverse hosting options: on-premise, hosted in your virtual private cloud or hosted by Data Virtuality. Pipes Professional uses a desktop application that gives you many options to manage your data. Pipes Professional is often used for use cases where you would like to move data from a database or API to an analytical database with sophisticated customization options.
Data Virtuality Logical Data Warehouse (LDW) is an easy-to-use and agile data management solution. It enables real-time data access, data modeling and data movement. Combining data virtualization and ETL functionality it provides data teams a powerful interface to manage data and distribute it to internal data consumers. The Data Virtuality Logical Data Warehouse is used for various use cases that require fastest time-to-market, maximum customization cabability and highest adaptability to changing market needs.
Detailed use cases can be found here:
Where do I find the trial version?
Which data sources can you integrate?
You can find a list of more than 200 ready-to-use connectors here.
If the connector you are looking for is missing, you can request it here. With an active subscription, we develop new connectors free of charge within two to six weeks. You would pay for the subscription of the connector once it's released.
What is meant by data sources, connectors, analytical storage or BI tools?
Data sources are your source systems you would like to retrieve data from:
- classical SQL databases, like PostgreSQL, MySQL or Microsoft SQL Server
- legacy databases, data lakes or enterprise data warehouses
- any kind of web services from Salesforce over Google Analytics to Facebook or Twitter as well as Generic Web Services
- Flat files (XML, CSV etc.) are no problem as well.
You can find a complete overview of supported data sources here.
Connectors are the interfaces to the specific data sources. If you do not find the source you would like to integrate just send us a request here and we will develop new connectors within two to six weeks (free of charge with an active subscription).
The analytical storage is your Data Base Management System (DBMS) or Data Warehouse (DWH). Data Virtuality supports different databases like PostgreSQL and SQL Server as well as cloud databases like Snowflake and Redshift.
For Data Virtuality Pipes the analytical storage is the target storage where data from your data sources is being loaded to.
For Pipes Professional and the Logical Data Warehouse the analytical storages is being used to optimize query performance and as target storage for automated ETL processes.
Business Intelligence (BI) tools are the front-ends you're using to analyze your data. With Data Virtuality you have one single source of data truth. Your front-ends ideally connect directly to Data Virtuality to retrieve data from all connected data sources. Data Virtuality supports to connect front-ends via ODBC, JDBC and REST.
What kind of data can I retrieve with your connectors?
You can retrieve all data that is provided by the data source. There is no limitation from our side, whatever the source itself provides, we are able to make it accessible for you.
How long does it take to set up Data Virtuality and which costs are associated?
Connecting data sources is simple and takes minutes when data source credentials are available.
Data Virtuality Pipes:
You can connect data sources yourself. The only thing you need are the credentials of your data sources and the target storage you want to have your data loaded to. You can start your free trial here.
Data Virtuality Pipes Professional and Logical Data Warehouse (LDW):
Most connectors can be connected by yourself, some are available at request and will be installed separately.
Which data governance features does Data Virtuality provide?
Data Virtuality enables you to ensure data quality for accurate, complete, and consistent data. You can use metadata repositories to improve master data management and increase the transparency, accountability and auditability with automatic data lineage.
For a full list of features please check our Logical Data Warehouse page here where you can also download our product sheet for all technical details.
What kind of training do I need to productively work with Data Virtuality's solutions?
For Data Virtuality Pipes no specific training is required. It is a self-service solution you will be able use by point and click. Whenever you have a question you can use the chat inside the application to directly ask our experts.
Pipes Professional and the Logical Data Warehouse are advanced solutions for sophisticated data management use cases. Non-technical users are able to quickly acquire the knowledge to operate the solution.
- Understanding of the concept of data virtualization
During the 14 day Proof-of-Concept period you can schedule training sessions with our Solution Engineer (optional).
Where do I find help?
Where to learn more about Data Virtuality:
- During a Proof-of-Concept please contact your dedicated Solution Engineer directly via eMail.
- Customers can submit support tickets through our support system.
- You can participate in our community to discuss topics with us and other users, give feedback, submit feature requests and learn best practices from other users.
I am looking for a full-stack solution, including data management and dashboard creation. Can you help me with that?
Data Virtuality focuses on providing easy-to-use and agile data integration solutions which allow you to access, centralize and manage data within your organisation.
We do not have a proprietary storage, nor a tool to visualize data. We work with partners who specialize in those fields. We also work with solution partners who can help you with your specific project requirements beyond data integration and data management. For an overview of selected partners have a look here.
Are you doing ETL or data virtualization?
Data Virtuality uses the best of ETL/ELT and data virtualization.
Data Virtuality Pipes and Pipes Professional are ETL/ELT solutions.
Data Virtuality Logical Data Warehouse (LDW) uses both technologies: data virtualization and ETL/ELT. We combine the two distinct technologies, virtualization and replication, for the best possible performance. The architecture is named by Gartner as the future of data warehousing.
Interested to learn more about data integration technologies? Take a look into our free ebook here.
Are you able to access data in real-time?
Yes, with Pipes Professional and the Logical Data Warehouse you can access data in real-time. You will be able to seamlessly work with real-time and historical data just as you need it.
In cases real-time is not possible due to restrictions of the source system, we can reach near real-time with extracts scheduled in quick succession.
Does Data Virtuality store my data?
Data Virtuality acts as a middle layer processing, but not storing data. All data remains either in your data sources or in your analytical storage. You always stay in control over your data and we have no technical ability to access your data.
Does your software has a sandbox environment for staging purposes?
You can use additional instances for purposes such as QA or development. The synchronization between the different instances can be done using our sync tool. We published a blog post about our sync tool, read it here.
What kind of hardware is required to run Data Virtuality?
The minimum recommended hardware requirements are as follows:
16 GB RAM
4 Core CPU
100 GB storage
Various Linux distributions or Windows supported as operating system. Data Virtuality runs well in virtual environments. However, the system works even better when using SSD storage.
Our desktop application works on all common operating systems, such as Windows, MacOS or Linux.
Does your software run on premise or in the cloud?
Data Virtuality Pipes is cloud only (US or Germany).
Data Virtuality Pipes Professional and Logical Data Warehouse can be:
- Hosted inside your virtual private network (e.g. AWS or Microsoft Cloud)
- Hosted by Data Virtuality
How do you handle large data volumes?
For large data volumes, we use a variety of approaches to handle them efficiently. Firstly, we use streaming on all data sources supporting it - this means that we do not do a full table scan, but rather only read the first batch of the data. Additionally, we use the concept of push down, so for example in a filtered statement, the data source will deliver only the filtered data. Lastly, we have different techniques for efficient distributed joining, such as Merge Join (on sorted keys) or Dependent Join (result of smaller table transported to filter the result of the bigger table)
How does Data Virtuality handle metadata? Is a search functionality included?
The metadata from all connected data sources, as well as the logical data model, is made available in Data Virtuality. It can be searched or exposed to 3rd party tools.
Does Data Virtuality provide a multitenancy architecture?
We operate a multitenancy architecture and offer isolated user environments depending on a tenant's role.
With our multiple virtual databases and isolated user environments feature, you can now create additional databases within one system. This is especially helpful if you are working in a big company with several branches and want to use one system with multiple customized virtual databases (VDBs).
In case you are rather looking for a separation due to staging purposes you can also easily setup a dedicated staging instance and seamlessly sync with production environments - just as needed.
How does your software handle user permissions and authentication?
A role-based permission system is available that can optionally be connected to your existing LDAP repository or Active Directory (AD). Each role can be assigned a set of permissions to access data sources or the logical data model. Additional security features, such as row-level security or column masking can also be configured per role.
What kind of SQL dialect Data Virtuality is using?
Data Virtuality uses an SQL dialect based on ANSI Standard and PostgreSQL dialect with numerous extensions.
How does Data Virtuality handle CDC / incremental updates?
CDC is available for certain data sources, where Data Virtuality will use the data source's transaction logs to replicate changes.
For data sources where CDC is not available, an algorithm can be used to identify changed rows based on the data model. This requires specific columns to be present in the data model to identify changed rows (e.g. a last updated timestamp).