Google Cloud Dataproc
Amazon EMR
Google BigQuery
HortonWorks Data Platform
Databricks
Google Cloud Dataflow
MapR Converged Data Platform
Snowflake
Metaplane
Telmai
Truth{set}
Segment
Iteratively
Percival
Masthead Data
Segment Protocols
Data teams are often the last to know about data quality issues, finding out only when downstream data consumers complain about broken dashboards. Metaplane solves this problem by continuously monitoring the entire data stack, alerting teams when something goes wrong, and providing context about what caused the issue.
Metaplane is the only data observability tool that is free to try and can be setup in under 10 minutes. After connecting your warehouse, our test engine automatically adds thousands of tests for row counts, freshness, and statistical properties, all without writing a single line of code.
Using your query history, transformation tool and BI tools, Metaplane can construct lineage across your entire data stack. When an issue is spotted, Metaplane will send you an alert to Slack or email and provide context about what may have caused the issue as well as what could be impacted.
Google Cloud Dataproc MetaplaneBased on our record, Google Cloud Dataproc should be more popular than Metaplane. It has been mentiond 3 times since March 2021. We are tracking product recommendations and mentions on various public social media platforms and blogs. They can help you identify which product is more popular and what people think of it.
I have also a spark cluster created with google cloud dataproc. Source: about 1 year ago
Specifically, we heavily rely on managed services from our cloud provider, Google Cloud Platform (GCP), for hosting our data in managed databases like BigTable and Spanner. For data transformations, we initially heavily relied on DataProc - a managed service from Google to manage a Spark cluster. - Source: dev.to / about 2 years ago
With that, the best way to maximize processing and minimize time is to use Dataflow or Dataproc depending on your needs. These systems are highly parallel and clustered, which allows for much larger processing pipelines that execute quickly. Source: over 2 years ago
After evaluating few solutions in the market: We were in the market to hunt for a solution which will cost under 10k (yearly) considering the cost of opensource will be similar considering DE resource and maintenance cost etc 1. MonteCarlo - Super duper expensive - Unable to hosting in Google Cloud 2. BigEye - Good features 3. Metaplane - Overall good package but when compared to catalog and other features it... Source: about 1 year ago
Amazon EMR - Amazon Elastic MapReduce is a web service that makes it easy to quickly process vast amounts of data.
Telmai - Monitor your customer data quality in real-time
Google BigQuery - A fully managed data warehouse for large-scale data analytics.
Truth{set} - Measuring the quality of consumer data
HortonWorks Data Platform - The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly...
Segment - We make customer data simple.